Abstract
Planning an obstacle-free optimal path presents great challenges for mobile robot applications, the deep deterministic policy gradient (DDPG) algorithm offers an effective solution. However, when the original DDPG is applied to robot path planning, there remains many problems such as inefficient learning and slow convergence that can adversely affect the ability to acquire optimal path. In response to these concerns, we propose an innovative framework named dueling deep deterministic policy gradient (D-DDPG) in this paper. First of all, we integrate the dueling network into the critic network to improve the estimation accuracy of Q-value. Furthermore, we design a novel reward function by combining the cosine distance with the Euclidean distance to improve learning efficiency. Our proposed model is validated by several experiments conducted in the simulation platform Gazebo. Experiments results demonstrate that our proposed model has the better path planning capability even in the unknown environment.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Bai, N., Wang, Z., Meng, F.: A stochastic attention CNN model for rumor stance classification. IEEE Access 8, 80771–80778 (2020). https://doi.org/10.1109/ACCESS.2020.2990770
Bjørlykhaug, E., Egeland, O.: Vision system for quality assessment of robotic cleaning of fish processing plants using CNN. IEEE Access 7, 71675–71685 (2019). https://doi.org/10.1109/ACCESS.2019.2919656
Capisani, L.M., Ferrara, A.: Trajectory planning and second-order sliding mode motion/interaction control for robot manipulators in unknown environments. IEEE Trans. Industr. Electron. 59(8), 3189–3198 (2012). https://doi.org/10.1109/TIE.2011.2160510
Chen, Y., Bai, G., Zhan, Y., Hu, X., Liu, J.: Path planning and obstacle avoiding of the USV based on improved ACO-APF hybrid algorithm with adaptive early-warning. IEEE Access 9, 40728–40742 (2021). https://doi.org/10.1109/ACCESS.2021.3062375
Chen, Y., Li, H., Liu, F.: An adaptive routing algorithm based on multiple-path-finding dijkstra’s and q-learning algorithm in silicon photonic interconnects on chip. In: 2020 IEEE 20th International Conference on Communication Technology (ICCT), pp. 117–120 (2020). https://doi.org/10.1109/ICCT50939.2020.9295898
Cui, Z., Wang, Y.: UAV path planning based on multi-layer reinforcement learning technique. IEEE Access 9, 59486–59497 (2021). https://doi.org/10.1109/ACCESS.2021.3073704
Drolshagen, S., Pfingsthorn, M., Gliesche, P., Hein, A.: Acceptance of industrial collaborative robots by people with disabilities in sheltered workshops. Front. Robot. AI 7, 173 (2021)
Er, M.J., Deng, C.: Obstacle avoidance of a mobile robot using hybrid learning approach. IEEE Trans. Industr. Electron. 52(3), 898–905 (2005). https://doi.org/10.1109/TIE.2005.847576
Fernandez, S.R.: Accuracy enhancement for robotic assembly of large-scale parts in the aerospace industry (2020)
Guo, K., Pan, Y., Yu, H.: Composite learning robot control with friction compensation: a neural network-based approach. IEEE Trans. Industr. Electron. 66(10), 7841–7851 (2019). https://doi.org/10.1109/TIE.2018.2886763
Hasselt, H.V., Guez, A., Silver, D.: Deep reinforcement learning with double q-learning. Computer Science (2015)
Henkemans, O., Pal, S., Werner, I., Neerincx, M.A., Looije, R.: Learning with charlie: a robot buddy for children with diabetes. In: the Companion of the 2017 ACM/IEEE International Conference (2017)
Hessel, M., et al.: Rainbow: combining improvements in deep reinforcement learning (2017)
Khatib, O.: Real-time obstacle avoidance for manipulators and mobile robots. In: Proceedings. 1985 IEEE International Conference on Robotics and Automation, vol. 2, pp. 500–505 (1985). https://doi.org/10.1109/ROBOT.1985.1087247
Lee, S.B., Hun Yoo, S.: Design of the companion robot interaction for supporting major tasks of the elderly. In: 2017 14th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI), pp. 655–659 (2017). https://doi.org/10.1109/URAI.2017.7992695
Li, Y., Zhang, D., Yin, F., Zhang, Y.: Cleaning robot operation decision based on causal reasoning and attribute learning*. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 6878–6885 (2020). https://doi.org/10.1109/IROS45743.2020.9340930
Lillicrap, T.P., et al.: Continuous control with deep reinforcement learning. Computer Science (2015)
Luo, M., Hou, X., Yang, J.: Surface optimal path planning using an extended dijkstra algorithm. IEEE Access 8, 147827–147838 (2020). https://doi.org/10.1109/ACCESS.2020.3015976
dos Santos, M.G., Petrillo, F.: Towards automated acceptance testing for industrial robots (2021)
Sutton, R., Barto, A.: Reinforcement Learning: An Introduction. An Introduction, Reinforcement Learning (1998)
Tai, L., Paolo, G., Liu, M.: Virtual-to-real deep reinforcement learning: continuous control of mobile robots for mapless navigation. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 31–36 (2017). https://doi.org/10.1109/IROS.2017.8202134
Tang, G., Tang, C., Claramunt, C., Hu, X., Zhou, P.: Geometric a-star algorithm: an improved a-star algorithm for agv path planning in a port environment. IEEE Access 9, 59196–59210 (2021). https://doi.org/10.1109/ACCESS.2021.3070054
Wang, Y.H., Li, T., Lin, C.J.: Backward q-learning: The combination of Sarsa algorithm and q-learning. Eng. Appl. Artif. Intell. 26(9), 2184–2193 (2013)
Watkins, C., Dayan, P.: Technical note: Q-learning. Mach. Learn. 8(3–4), 279–292 (1992)
Xin, J., Zhao, H., Liu, D., Li, M.: Application of deep reinforcement learning in mobile robot path planning. In: 2017 Chinese Automation Congress (CAC), pp. 7112–7116 (2017). https://doi.org/10.1109/CAC.2017.8244061
Yang, R., Cheng, L.: Path planning of restaurant service robot based on a-star algorithms with updated weights. In: 2019 12th International Symposium on Computational Intelligence and Design (ISCID), vol. 1, pp. 292–295 (2019). https://doi.org/10.1109/ISCID.2019.00074
Yang, Y., Li, J., Peng, L.: Multirobot path planning based on a deep reinforcement learning DQN algorithm. CAAI Trans. Intell. Technol. 5(3), 177–183 (2020)
Yong, T., Wei, H., Wang, T., Chen, D.: A multi-layered interaction architecture for elderly companion robot. In: International Conference on Intelligent Robotics & Applications (2008)
Yuan, J., Yang, S., Cai, J.: Consistent path planning for on-axle-hitching multisteering trailer systems. IEEE Trans. Industr. Electron. 65(12), 9625–9634 (2018). https://doi.org/10.1109/TIE.2018.2823691
Zhao, T., Li, H., Dian, S.: Multi-robot path planning based on improved artificial potential field and fuzzy inference system. J. Intell. Fuzzy Syst. 39(5), 7621–7637 (2020)
Zhu, D.D., Sun, J.Q.: A new algorithm based on dijkstra for vehicle path planning considering intersection attribute. IEEE Access 9, 19761–19775 (2021). https://doi.org/10.1109/ACCESS.2021.3053169
Acknowledgement
This work is supported by the National Natural Science Foundation of China (61976127).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhao, P., Zheng, J., Zhou, Q., Lyu, C., Lyu, L. (2021). A Dueling-DDPG Architecture for Mobile Robots Path Planning Based on Laser Range Findings. In: Pham, D.N., Theeramunkong, T., Governatori, G., Liu, F. (eds) PRICAI 2021: Trends in Artificial Intelligence. PRICAI 2021. Lecture Notes in Computer Science(), vol 13031. Springer, Cham. https://doi.org/10.1007/978-3-030-89188-6_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-89188-6_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-89187-9
Online ISBN: 978-3-030-89188-6
eBook Packages: Computer ScienceComputer Science (R0)