Comparison of Three Deep Reinforcement Learning Algorithms for Solving the Lunar Lander Problem

DOI

10.2991/978-94-6463-370-2_21How to use a DOI?

Keywords

Reinforcement Learning; Lunar Lander Problem; Deep Learning

Abstract

The Lunar Lander problem presents a formidable challenge in the realm of reinforcement learning, necessitating the creation of autonomous spacecraft capable of safe landings on the lunar surface. In this study, three prominent reinforcement learning algorithms, namely Deep Q-Network (DQN), Double Deep Q-Network (DDQN), and Policy Gradient, are investigated and examined to address this problem. Initially, DQN algorithm, combining neural networks and Q-Learning, is leveraged to learn an optimal landing policy. By approximating Q-Values through neural network training, the spacecraft learns to make informed decisions, leading to successful landings. Subsequently, DDQN algorithm, which mitigates overestimation bias, is used. By utilizing two neural networks - one for action selection and the other for evaluation - DDQN improves stability and convergence, resulting in refined landing policies. Furthermore, this work explores the application of Policy Gradient methods for this problem. By directly optimizing the policy using gradient ascent, the spacecraft maximizes cumulative rewards, achieving efficient and accurate landings. The performance of the algorithms is assessed through extensive simulations that encompass diverse lunar surface conditions. The results demonstrate the effectiveness of these methods, showcasing their capability to facilitate successful and fuel-efficient spacecraft landings. In conclusion, this study contributes to the understanding of DQN, DDQN, and Policy Gradient algorithms for the Lunar Lander problem. The findings highlight the unique strengths of each algorithm and their potential in autonomous spacecraft landing. The insights gained from this research have implications for the development of intelligent landing systems in future lunar missions, advancing the field of reinforcement learning in aerospace applications.

Copyright

© 2024 The Author(s)

Open Access

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

TY  - CONF
AU  - Dingli Shen
PY  - 2024
DA  - 2024/02/14
TI  - Comparison of Three Deep Reinforcement Learning Algorithms for Solving the Lunar Lander Problem
BT  - Proceedings of the 2023 International Conference on Data Science, Advanced Algorithm and Intelligent Computing (DAI 2023)
PB  - Atlantis Press
SP  - 187
EP  - 199
SN  - 1951-6851
UR  - https://doi.org/10.2991/978-94-6463-370-2_21
DO  - 10.2991/978-94-6463-370-2_21
ID  - Shen2024
ER  -