Proceedings of the 2023 International Conference on Data Science, Advanced Algorithm and Intelligent Computing (DAI 2023)

<Previous Article In Volume
Next Article In Volume>

An Evaluation of DDPG, TD3, SAC, and PPO: Deep Reinforcement Learning Algorithms for Controlling Continuous System

Authors
Shijie Liu1, *
1Department of Electrical and Electronics Engineering, The Hong Kong Polytechnic University, Hong Kong SAR, 100872, China
*Corresponding author. Email: 22100838d@connect.polyu.hk
Corresponding Author
Shijie Liu
Available Online 14 February 2024.
DOI
10.2991/978-94-6463-370-2_3How to use a DOI?
Keywords
Reinforcement Learning; Deep Learning; Continuous System Controlling
Abstract

Continuous systems are physical systems that can be stimulated by continuous and analog variables. The parameters or variables are within a range of values. An excellent continuous controlling policy enables the system to act appropriately and smoothly without much intervention, which can be useful in robotics, self-driving, industries, etc. The DRL algorithm has extensive applications in continuous systems control. This essay will explore the performance of four DRL algorithms, that is the Deep Deterministic Policy Gradient (DDPG), Twin Delayed DDPG (TD3), Soft Actor-Critic (SAC), and Proximal Policy Optimization (PPO) by using environment from the four of environments in Mujoco in Gym. Comparative experiments are done, and the highest rewards and the required number of iterations to converge are compared. The result of comparative experiments illustrates that these DRL algorithms can learn relatively appropriate policies in continuous controlling tasks. In particular, TD3 and SAC were found to be able to learn the controlling policy more effectively. Further research is needed to find better ways to adjust hyperparameters.

Copyright
© 2024 The Author(s)
Open Access
Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

<Previous Article In Volume
Next Article In Volume>
Volume Title
Proceedings of the 2023 International Conference on Data Science, Advanced Algorithm and Intelligent Computing (DAI 2023)
Series
Advances in Intelligent Systems Research
Publication Date
14 February 2024
ISBN
10.2991/978-94-6463-370-2_3
ISSN
1951-6851
DOI
10.2991/978-94-6463-370-2_3How to use a DOI?
Copyright
© 2024 The Author(s)
Open Access
Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Cite this article

risenwbib
TY  - CONF
AU  - Shijie Liu
PY  - 2024
DA  - 2024/02/14
TI  - An Evaluation of DDPG, TD3, SAC, and PPO: Deep Reinforcement Learning Algorithms for Controlling Continuous System
BT  - Proceedings of the 2023 International Conference on Data Science, Advanced Algorithm and Intelligent Computing (DAI 2023)
PB  - Atlantis Press
SP  - 15
EP  - 24
SN  - 1951-6851
UR  - https://doi.org/10.2991/978-94-6463-370-2_3
DO  - 10.2991/978-94-6463-370-2_3
ID  - Liu2024
ER  -