Abstract
In this work, reinforcement learning techniques are implemented and compared to address biped locomotion optimization. Central Pattern Generators (CPGs) and Dynamic Movement Primitives (DMPs) were combined to easily produce complex trajectories for the joints of a simulated DARwIn-OP humanoid robot. Two reinforcement learning algorithms, Policy Learning by Weighting Exploration with the Returns (PoWER) and Path Integral Policy Improvement with Covariance Matrix Adaptation (PI2-CMA) were implemented in the simulated DARwIn-OP to seek optimal DMP parameters that maximize frontal velocity when facing different situations which demand adaptation from the controller in order to successfully walk in different types of slopes. Additionally, elitism was introduced in PI2-CMA in order to improve the performance of the algorithm. Results show that these approaches enabled easy adaptation of DARwIn-OP to new situations. The results are very promising and demonstrate flexibility at generating or adapting new trajectories for locomotion.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Shimoda, S., Yoshihara, Y., Kimura, H.: Transactions on Autonomous Mental Development (2013)
Calinon, S., Kormushev, P., Caldwell, D.G.: Robot. Auton. Syst. 61(4), 369 (2013). doi:10.1016/j.robot.2012.09.012
Morimoto, J., Atkeson, C.G.: IEEE Robot. Autom. Mag. 14(2), 41 (2007)
Vukobratović, M., Borovac, B.: International Journal of Humanoid Robotics 01(01), 157 (2004). doi:10.1142/S0219843604000083
Deisenroth, M.P., Calandra, R., Seyfarth, A., Peters, J.: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1787–1792 (2012)
Schaal, S.: In: The International Symposium on Adaptive Motion of Animals and Machines (2003)
Sugimoto, N., Morimoto, J.: In: 11th IEEE-RAS International Conference on Humanoid Robots (2011)
Matos, V., Santos, C.P.: In: IEEE-RAS International Conference on Humanoid Robots (2012)
Matsubara, T., Morimoto, J., Nakanishi, J., Sato, M.A., Doya, K.: In: Proceedings of the 2005 IEEE International Conference on Robotics and Automation, 2005. ICRA 2005, pp 4164–4169. IEEE (2005)
Nakanishi, J., Morimoto, J., Endo, G., Cheng, G., Schaal, S., Kawato, M.: Robot. Auton. Syst. 47(2-3), 79 (2004). doi:10.1016/j.robot.2004.03.003
Ijspeert, A.J., Nakanishi, J., Schaal, S.: In: Becker, S., Thrun, S., Obermayer, K. (eds.) Advances in Neural Information Processing Systems, vol. 15, pp. 1547–1554. MIT Press, Cambridge (2003)
Ijspeert, A.J., Nakanishi, J., Hoffmann, H., Pastor, P., Schaal, S.: Neural Comput. 25(2), 328 (2013). doi:10.1162/NECO_a_00393
Peters, J., Schaal, S.: 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems pp. 2219–2225. doi:10.1109/IROS.2006.282564 (2006)
Pastor, P., Kalakrishnan, M., Meier, F., Stulp, F., Buchli, J., Theodorou, E., Schaal, S.: Robot. Auton. Syst. 61(4), 351 (2013). doi:10.1016/j.robot.2012.09.017
Kober, J., Bagnell, J.A., Peters, J.: International Journal of Robotics Research (accepted) (2013)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press (1998)
André, J., Santos, C., Costa, L.: In: 11th Internation Conference of Numerical Analysis and Applied Mathematics (2013)
Peters, J., Schaal, S.: Neurocomputing 71(7-9), 1180 (2008)
Kober, J., Peters, J.: Mach. Learn. 84(1-2), 171 (2011). doi:10.1007/s10994-010-5223-6
Budden, D., Walker, J., Flannery, M., Mendes, A.: In: Australasian Conference on Robotics and Automation (ACRA) (2013)
Schaal, S., Peters, J., Nakanishi, J., Ijspeert, A.: Internationa Symposium on Robotics Research pp. 1–10 (2003)
Stulp, F., Sigaud, O., et al.: HAL-UPMC (2012)
Theodorou, E.A., Buchli, J., Schaal, S.: J. Mach. Learn. Res. 11, 3137 (2010)
Stulp, F., Sigaud, O.: In: Proceedings of the 29th International Conference on Machine Learning (Edinburgh, Scotland, 2012)
Prokopenko, M., Gerasimov, V., Tanev, I.: In: Proceedings of the 10th International Conference on the Simulation and Synthesis of Living Systems (2006)
Videos. http://asbg.dei.uminho.pt/node/387 (2013)
Oliveira, M.A., Doncieux, S., Mouret, J.B., Peixoto Santos, C.: In: Proceedings of IEEE-RAS International Conference on Humanoid Robots (2013)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
André, J., Teixeira, C., Santos, C.P. et al. Adapting Biped Locomotion to Sloped Environments. J Intell Robot Syst 80, 625–640 (2015). https://doi.org/10.1007/s10846-015-0196-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10846-015-0196-0