Abstract
In order to solve the problems of low learning efficiency and slow convergence speed when mobile robot uses reinforcement learning method for path planning in complex environment, a reinforcement learning method based on each round path planning result is proposed. Firstly, the algorithm adds obstacle learning matrix to improve the success rate of path planning; and introduces heuristic reward to speed up the learning process by reducing the search space; then proposes a method of dynamically adjusting the exploration factor to balance the exploration and utilization in path planning, so as to further improve the performance of the algorithm. Finally, the simulation experiment in grid environment shows that compared with Q-learning algorithm, the improved algorithm not only shortens the average path length of the robot to reach the target position, but also speeds up the learning efficiency of the algorithm, so that the robot can find the optimal path more quickly. The code of EPRQL algorithm proposed in this paper has been published to GitHub: https://github.com/panpanpanguoguoqian/mypaper1.git.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
The data used in this study are generated by the author’s independent experiment.
References
Castillo O, Trujillo L, Melin P (2007) Multiple objective genetic algorithms for path-planning optimization in autonomous mobile robots. Soft Comput 11(3):269–279
Chu P, Vu H, Yeo D, Lee B, Cho K (2015) Robot reinforcement learning for automatically avoiding a dynamic obstacle in a virtual environment. Lecture Notes Electr Eng 352:157–164
Guo M, Liu Y, Malec J (2004) A new q-learning algorithm based on the metropolis criterion. IEEE Trans Syst Man Cybern Part B Cybern 34(5):2140–2143
Jaradat MAK, Al-Rousan M, Quadan L (2011) Reinforcement based mobile robot navigation in dynamic environment. Robot Comput-Integr Manuf 27(1):135–149
Lieping Z, Liu T, Shenglan Z, Zhengzhong W, Xianhao S, Zuqiong Z (2021) A self-adaptive reinforcement-exploration q-learning algorithm. Symmetry 13:1057
Lin L, Xie H, Zhang D, Shen L (2010) Supervised neural q-learning based motion control for bionic underwater robots. J Bionic Eng 7(1):S177–S184
Ma YN, Gong YJ, Xiao CF, Gao Y, Zhang J (2018) Path planning for autonomous underwater vehicles: an ant colony algorithm incorporating alarm pheromone. IEEE Trans Veh Technol 68:141–154
Mac TT, Copot C, Tran DT, Keyser RD (2016) Heuristic approaches in robot path planning: a survey. Robot Auton Syst 86:13–28
Polydoros AS, Nalpantidis L (2017) Survey of model-based reinforcement learning: applications on robotics. J Intell Robot Syst 86(2):1–21
Rimon E, Koditschek DE (1992) Exact robot navigation using artificial potential functions. IEEE Trans Robot Autom 8:501–518
Song Y, Li Y, Li C (2012) Initialization of reinforcement learning for mobile robot path planning. Control Theory Appl 29(012):1623–1628
Soong LE, Pauline O, Chun CK (2019) Solving the optimal path planning of a mobile robot using improved q-learning. Robot Auton Syst 115(99):143–161
Watkins CJCH, Dayan P (1992) Q-learning. Mach Learn 8(3–4):279–292
Xu X, Yuan J (2019) Path planning method for mobile robot based on improved reinforcement learning. J Chin Inert Technol 27(3):314–320
Yu J, Su Y, Liao Y (2020) The path planning of mobile robot by neural networks and hierarchical reinforcement learning. Front Neurorobot 14:63
Zhang F, Li N, Yuan R, Fu Y (2018) Robot path planning algorithm based on reinforcement learning. J Huazhong Univ Sci Technol (Nat Sci Ed) 46(12):65–70
Zhang Y, Gong DW, Zhang JH (2013) Robot path planning in uncertain environment using multi-objective particle swarm optimization. Neurocomputing 103(1):172–185
Zhao M, Lu H, Yang S, Guo F (2020) The experience-memory q-learning algorithm for robot path planning in unknown environment. IEEE Access 8:47824–47844
Funding
This work is supported by the National Natural Science Foundation of China (Grant No. U1933123).
Author information
Authors and Affiliations
Contributions
Each author had made some contribution to this article.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Informed consent
Informed consent was obtained from all individual participants included in the study
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Pan, G., Xiang, Y., Wang, X. et al. Research on path planning algorithm of mobile robot based on reinforcement learning. Soft Comput 26, 8961–8970 (2022). https://doi.org/10.1007/s00500-022-07293-4
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-022-07293-4