Abstract
Due to the complexity of modular mechanism control, the motion planning of snake robots in an unfamiliar complex environment is a long-standing issue. We propose an improved deep deterministic policy gradient (DDPG) algorithm to plan the path with the shortest time and the least collisions. Traditional robot DDPG can not make full use of previous states to make decisions. This paper uses LSTM, learns all previous hidden states through memory and reasoning, and distinguishes the importance of features through the Self-Attention mechanism, which reduces the impact of useless features on decision-making and improves decision-making accuracy. In addition, the reward function is optimized to make the snake robot reach the target point faster. Finally, we do experiments in the simulation environment. The results show that the algorithm can speed up the convergence speed of the DDPG and reduce the path planning time and collision times of the snake robot.
This work was supported by The National Natural Science Foundation of China (62072335) and The Tianjin Science and Technology Program (19PTZWHZ00020).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Hirose, S., Cave, P., Goulden, C.: Biologically Inspired Robots: Snake-Like Locomotors and Manipulators. Oxford University Press, Oxford (1993)
Tony, O.: Biologically Inspired Robots: Snake-Like Locomotors and Manipulators by Shigeo Hirose Oxford University Press, Oxford (1993). 220 pages, incl. index (£40).” Robotica 12, 282 (1994)
Miller, G.S.P.: The motion dynamics of snakes and worms. In: Proceedings of the 15th Annual Conference on Computer Graphics and Interactive Techniques (1988)
Bing, Z., et al.: Towards autonomous locomotion: CPG-based control of smooth 3D slithering gait transition of a snake-like robot. Bioinspiration & Biomimetics 12(3), 035001 (2017)
Li, D., et al.: 2D obstacle avoidance method for snake robot based on modified artificial potential field. In: 2017 IEEE International Conference on Unmanned Systems (ICUS), pp. 358–363 (2017)
Hong, J., et al.: Semantically-aware strategies for stereo-visual robotic obstacle avoidance. In: 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 2450–2456 (2021)
Kano, T., et al.: Local reflexive mechanisms essential for snakes’ scaffold-based locomotion. Bioinspiration Biomimetics 7(4), 046008 (2012)
Sartoretti, G., et al.: Distributed learning of decentralized control policies for articulated mobile robots. IEEE Trans. Robot. 35, 1109–1122 (2019)
Haghshenas-Jaryani, M., Sevil. H.E.: Autonomous navigation and obstacle avoidance of a snake robot with combined velocity-heading control. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 7507–7512 (2020)
Yang, W., et al.: Perception-aware path finding and following of snake robot in unknown environment. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5925–5930 (2020)
Yang, W., et al.: Spline based curve path following of underactuated snake robots. In: 2019 International Conference on Robotics and Automation (ICRA), pp. 5352–5358 (2019)
Greff, K., et al.: LSTM: a search space odyssey. IEEE Trans. Neural Netw. Learn. Syst. 28, 2222–2232 (2017)
Arulkumaran, K., et al.: Deep reinforcement learning: a brief survey. IEEE Signal Process. Mag. 34, 26–38 (2017)
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015)
Silver, D., et al.: Deterministic policy gradient algorithms. In: ICML (2014)
Liu, X., et al:. Learning to locomote with artificial neural-network and CPG-based control in a soft snake robot. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 7758–7765 (2020)
Vaswani, A., et al.: Attention is all you need. ArXiv abs/1706.03762 (2017)
Gong, H., et al.: Efficient path planning for mobile robot based on deep deterministic policy gradient. Sensors (Basel, Switzerland) 22, 3579 (2022)
Fujimoto, S., et al.: Addressing function approximation error in actor-critic methods. ArXiv abs/1802.09477 (2018)
Jia, Y., Ma, S.: A coach-based bayesian reinforcement learning method for snake robot control. IEEE Robot. Autom. Lett. 6, 2319–2326 (2021)
Acknowledgement
This work was supported by The National Natural Science Foundation of China (62072335) and The Tianjin Science and Technology Program (19PTZWHZ00020).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Liu, X., Wang, J., Sun, Y. (2023). Snake Robot Motion Planning Based on Improved Depth Deterministic Policy Gradient. In: Yang, S., Islam, S. (eds) Web and Big Data. APWeb-WAIM 2022 International Workshops. APWeb-WAIM 2022. Communications in Computer and Information Science, vol 1784. Springer, Singapore. https://doi.org/10.1007/978-981-99-1354-1_14
Download citation
DOI: https://doi.org/10.1007/978-981-99-1354-1_14
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-1353-4
Online ISBN: 978-981-99-1354-1
eBook Packages: Computer ScienceComputer Science (R0)