Abstract
With the vigorous development of computer-related technology, the “perception + decision” paradigm of the combination of deep learning and reinforcement learning has become a research hotspot. Nowadays, deep reinforcement learning algorithms have been successfully applied to the fields of games, industry and commerce. However, deep reinforcement learning algorithms often fall into the dilemma of “exploration” and “exploitation”, and the effect of these algorithms is easily affected by the quality of hyperparameters. In order to make up for the defects mentioned above, this paper introduces the particle swarm based reinforcement learning framework (PRL). Compared with the standard reinforcement learning algorithms, this framework greatly improves the exploration ability and obtains better scores in a series of gym experimental tests.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Cheng, C.A., Kolobov, A., Swaminathan, A.: Heuristic-guided reinforcement learning. Adv. Neural. Inf. Process. Syst. 34, 13550–13563 (2021)
Degrave, J., et al.: Magnetic control of tokamak plasmas through deep reinforcement learning. Nature 602(7897), 414–419 (2022)
Ding, S., Du, W., Zhao, X., Wang, L., Jia, W.: A new asynchronous reinforcement learning algorithm based on improved parallel PSO. Appl. Intell. 49(12), 4211–4222 (2019)
François-Lavet, V., Henderson, P., Islam, R., Bellemare, M.G., Pineau, J.: An introduction to deep reinforcement learning. arXiv preprint arXiv:1811.12560 (2018)
Fujimoto, S., Gu, S.S.: A minimalist approach to offline reinforcement learning. In: Advances in Neural Information Processing Systems, vol. 34 (2021)
Huang, C., et al.: Multi-hop RIS-empowered terahertz communications: a DRL-based hybrid beamforming design. IEEE J. Sel. Areas Commun. 39(6), 1663–1677 (2021)
Ibarz, J., Tan, J., Finn, C., Kalakrishnan, M., Pastor, P., Levine, S.: How to train your robot with deep reinforcement learning: lessons we have learned. Int. J. Robot. Res. 40(4–5), 698–721 (2021)
Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of ICNN 1995-International Conference on Neural Networks, vol. 4, pp. 1942–1948. IEEE (1995)
Khadka, S., et al.: Collaborative evolutionary reinforcement learning. In: International Conference on Machine Learning, pp. 3341–3350. PMLR (2019)
Khadka, S., Tumer, K.: Evolution-guided policy gradient in reinforcement learning. In: Advances in Neural Information Processing Systems, vol. 31 (2018)
Kiran, B.R., et al.: Deep reinforcement learning for autonomous driving: a survey. IEEE Trans. Intell. Transp. Syst. (2021)
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)
Movahedi, Z., Bastanfard, A.: Toward competitive multi-agents in polo game based on reinforcement learning. Multimedia Tools Appl. 80(17), 26773–26793 (2021)
Pourchot, A., Sigaud, O.: CEM-RL: combining evolutionary and gradient-based methods for policy search. arXiv preprint arXiv:1810.01222 (2018)
Silver, D., et al.: Mastering the game of go with deep neural networks and tree search. Nature 529(7587), 484–489 (2016)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (2018)
Wang, B., Liu, F., Lin, W.: Energy-efficient VM scheduling based on deep reinforcement learning. Futur. Gener. Comput. Syst. 125, 616–628 (2021)
Zhan, Z.H., et al.: Matrix-based evolutionary computation. IEEE Trans. Emerg. Top. Comput. Intell. 6(2), 315–328 (2021)
Zhang, F., Li, J., Li, Z.: A TD3-based multi-agent deep reinforcement learning method in mixed cooperation-competition environment. Neurocomputing 411, 206–215 (2020)
Zhou, J., Xue, S., Xue, Y., Liao, Y., Liu, J., Zhao, W.: A novel energy management strategy of hybrid electric vehicle via an improved TD3 deep reinforcement learning. Energy 224, 120118 (2021)
Acknowledgement
This work was supported by the National Natural Science Foundation of China under Grant 61973244 and Grant 61573277. It is also supported by the open fund of CETC Key Laboratory of Data Link Technology (CLDL-20202101-1).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Duan, J., Guo, Y., Wang, Z., Ke, L. (2022). Particle Swarm Based Reinforcement Learning. In: Tan, Y., Shi, Y. (eds) Data Mining and Big Data. DMBD 2022. Communications in Computer and Information Science, vol 1744. Springer, Singapore. https://doi.org/10.1007/978-981-19-9297-1_3
Download citation
DOI: https://doi.org/10.1007/978-981-19-9297-1_3
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-19-9296-4
Online ISBN: 978-981-19-9297-1
eBook Packages: Computer ScienceComputer Science (R0)