Abstract
Controllers for underwater snake-like robots are difficult to design because of their high DOF and complex motions. Additionally, because of the complex underwater environment and insufficient knowledge of hydrodynamics, the traditional control algorithms based on environment or robot modeling cannot work well. In this paper, we propose an SCRMA algorithm, which combines the characteristics of the RMA algorithm for rapid learning and adaptation to the environment, and uses curriculum learning and save &load exploration to accelerate the training speed. Experiments show that the SCRMA algorithm works better than other kinds of reinforcement learning algorithms nowadays.
This work is supported by the National Natural Science Foundation of China (62073176). All the authors are with the Institute of Robotics and Automatic Information System, College of Artificial Intelligence, Nankai University, China.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Hirose, S.: Biologically Inspired Robots: Snakelike Locomotors and Manipulators. Oxford University Press (1993)
Ma, S.: Analysis of snake movement forms for realization of snake-like robots. In: Proceedings of the 1999 IEEE International Conference on Robotics and Automation (1999)
Matsuno, F., Mogi, K.: Redundancy controllable system and control of snake robots based on kinematic model. In: IEEE Conference on Decision & Control (2000)
Mohammadi, A., Rezapour, E., Maggiore, M., Pettersen, K.Y.: Maneuvering control of planar snake robots using virtual holonomic constraints. IEEE Trans. Control Syst. Technol. 24(3), 884–899 (2016)
Ariizumi, R., Matsuno, F.: Dynamic analysis of three snake robot gaits. IEEE Trans. Rob. 33(5), 1075–1087 (2017)
Hasanzadeh, S., Tootoonchi, A.A.: Ground adaptive and optimized locomotion of snake robot moving with a novel gait. Auton. Robot. 28(4), 457–470 (2010)
Tesch, M., Schneider, J.G., Choset, H.: Using response surfaces and expected improvement to optimize snake robot gait parameters. In: IEEE/RSJ International Conference on Intelligent Robots & Systems (2011)
Ouyang, W., Liang, W., Li, C., Zheng, H., Ren, Q., Li, P.: Steering motion control of a snake robot via a biomimetic approach. Front. Inf. Technol. Electron. Eng. 20(1), 32–44 (2019)
Minh, V., et al.: Human-level control through deep reinforcement learning. nature 518, 529–533 (2015)
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms (2017)
Lillicrap, T.P., et al.: Continuous control with deep reinforcement learning. Computer Science (2015)
Okal, B., Kai, O.A.: Learning socially normative robot navigation behaviors with Bayesian inverse reinforcement learning. In: IEEE International Conference on Robotics and Automation (ICRA) (2016)
Kretzschmar, H., Spies, M., Sprunk, C., Burgard, W.: Socially compliant mobile robot navigation via inverse reinforcement learning. Int. J. Robot. Res. 35(11), 1289–1307 (2016)
Zhu, Y., et al.: Target-driven visual navigation in indoor scenes using deep reinforcement learning (2016)
Sartoretti, G., Shi, Y., Paivine, W., Travers, M., Choset, H.: Distributed learning of decentralized control policies for articulated mobile robots (2018)
Kumar, A., Fu, Z., Pathak, D., Malik, J.: RMA: rapid motor adaptation for legged robots (2021)
Bengio, Y., Louradour, J., Collobert, R., Weston, J.: Curriculum learning. In: Proceedings of the 26th Annual International Conference on Machine Learning, ICML 2009, Montreal, Quebec, Canada, 14–18 June 2009 (2009)
Taylor, G.: Analysis of the swimming of long and narrow animals. Proc. R. Soc. Lond. 214(1117), 158–183 (1952)
Bellman, R.: A Markovian decision process. Indiana Univ. Math. J. 6(4), 679–684 (1957)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, X., Sheng, F., Guo, X. (2022). SCRMA: Snake-Like Robot Curriculum Rapid Motor Adaptation. In: Liu, H., et al. Intelligent Robotics and Applications. ICIRA 2022. Lecture Notes in Computer Science(), vol 13456. Springer, Cham. https://doi.org/10.1007/978-3-031-13822-5_16
Download citation
DOI: https://doi.org/10.1007/978-3-031-13822-5_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-13821-8
Online ISBN: 978-3-031-13822-5
eBook Packages: Computer ScienceComputer Science (R0)