Skip to main content
SpringerLink
Log in

Bipedal Walking Robot Control Using PMTG Architecture

  • Conference paper
  • First Online:
Synergetic Cooperation between Robots and Humans (CLAWAR 2023)

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 811))

Included in the following conference series:

Abstract

Reinforcement learning based methods can achieve excellent results for robot locomotion control. However, their serious disadvantage is the long agent training time and large number of parameters defining its behavior. In this paper, we propose a method that significantly reduces training time. It is based on the Policy Modulating Trajectory Generator (PMTG) architecture, which uses Central Pattern Generators (CPG) as a gait generator. We tested this approach on an OpenAI BipedalWalker-v3 environment. The paper presents the results of this algorithm, showing its effectiveness in solving a locomotion problem over challenging terrain.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://youtu.be/h9M4VnhJPTs.

  2. 2.

    https://youtu.be/75HU_gmHlZE.

References

  1. McGhee, R.B.: Finite state control of quadruped locomotion. Simulation 9(3), 135–140 (1967). https://doi.org/10.1177/003754976700900308

    Article  Google Scholar 

  2. Raibert, M.H.: Legged Robots that Balance. MIT Press, Cambridge (1986)

    Book  Google Scholar 

  3. Villarreal, O., Barasuol, V., Wensing, P.M., Caldwell, D.G., Semini, C.: Mpc-based controller with terrain insight for dynamic legged locomotion. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 2436–2442 (2020)

    Google Scholar 

  4. Sleiman, J.P., Farshidian, F., Minniti, M.V., Hutter, M.: A unified mpc framework for whole-body dynamic locomotion and manipulation. IEEE Robot. Autom. Lett. 6(3), 4688–4695 (2021)

    Article  Google Scholar 

  5. Bjelonic, M., Grandia, R., Harley, O., Galliard, C., Zimmermann, S., Hutter, M.: Whole-body mpc and online gait sequence generation for wheeled-legged robots. In: 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 8388–8395 (2021)

    Google Scholar 

  6. Di Carlo, J., Wensing, P.M., Katz, B., Bledt, G., Kim, S.: Dynamic locomotion in the mit cheetah 3 through convex model-predictive control. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1–9 (2018)

    Google Scholar 

  7. Iscen, A., Caluwaerts, K., Tan, J., Zhang, T., Coumans, E., Sindhwani, V., Vanhoucke, V.: Policies modulating trajectory generators. In: Conference on Robot Learning, pp. 916–926 (2018)

    Google Scholar 

  8. Tan, J., Zhang, T., Coumans, E., Iscen, A., Bai, Y., Hafner, D., Vanhoucke, V.: Sim-to-real: Learning agile locomotion for quadruped robots (2018). arXiv:1804.10332

  9. Kumar, A., Fu, Z., Pathak, D., Malik, J.: Rma: rapid motor adaptation for legged robots (2021). arXiv:2107.04034

  10. Hwangbo, J., Lee, J., Dosovitskiy, A., Bellicoso, D., Tsounis, V., Koltun, V., Hutter, M.: Learning agile and dynamic motor skills for legged robots. Sci. Robot. 4(26) (2019). https://doi.org/10.1126/scirobotics.aau5872

  11. Margolis, G.B., Yang, G., Paigwar, K., Chen, T., Agrawal, P.: Rapid locomotion via reinforcement learning (2022). arXiv:2205.02824

  12. Margolis, G.B., Chen, T., Paigwar, K., Fu, X., Kim, D., Kim, S., Agrawal, P: Learning to jump from pixels (2021). arXiv:2110.15344

  13. Alexander, R.M.: Optimization and gaits in the locomotion of vertebrates. Physiol. Rev. 69(4), 1199–1227 (1989)

    Article  MathSciNet  Google Scholar 

  14. Haarnoja, T., Zhou, A., Hartikainen, K., Tucker, G., Ha, S., Tan, J., Kumar, V., Zhu, H., Gupta, A., Abbeel, P., et al.: Soft actor-critic algorithms and applications (2018). arXiv:1812.0590

  15. Rudin, N., Hoeller, D., Reist, P., Hutter, M.: Learning to walk in minutes using massively parallel deep reinforcement learning. In: Conference on Robot Learning, pp. 91–100 (2022)

    Google Scholar 

  16. Danilov, V., Diane, S.: CPG-based gait generator for a quadruped robot with sidewalk and turning operations. In: Robotics in Natural Settings: CLAWAR 2022, pp. 276–288 (2022). https://doi.org/10.1007/978-3-031-15226-9_27

  17. Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., Zaremba, W.: Openai gym (2016). arXiv:1606.01540

  18. Yu, J., Tan, M., Chen, J., Zhang, J.: A survey on CPG-inspired control models and system implementation. IEEE Trans. Neural Netw. Learn. Syst. 25(3), 441–456 (2013)

    Article  Google Scholar 

  19. Yu, J., Tan, M., Chen, J., Zhang, J.: A survey on CPG-inspired control models and system implementation. IEEE Trans. Neural Netw. Learn. Syst. 25(3), 441–456 (2013)

    Article  Google Scholar 

  20. Akiba, T., Sano, S., Yanase, T., Ohta, T., Koyama, M.: Optuna: a next-generation hyperparameter optimization framework. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 2623–2631 (2019)

    Google Scholar 

  21. OpenAI Gym Leaderboard. https://github.com/openai/gym/wiki/Leaderboard

Download references

Author information

Authors and Affiliations

  1. Institute of Control Problems of Russian Academy of Sciences, Moscow, Russia

    Vladimir Danilov & Sekou Diane

  2. Robotics Laboratory, Institute of Mechanics Lomonosov Moscow State University, Michurinsky prosp.1, 119192, Moscow, Russia

    Konstantin Klimov & Dmitrii Kapytov

  3. Voltbro LCC, Moscow, Russia

    Vladimir Danilov, Konstantin Klimov & Dmitrii Kapytov

Authors
  1. Vladimir Danilov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Konstantin Klimov
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dmitrii Kapytov
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sekou Diane
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vladimir Danilov .

Editor information

Editors and Affiliations

  1. Universidade Federal de Santa Catarina, Blumenau, Santa Catarina, Brazil

    Ebrahim Samer El Youssef

  2. School of Engineering, London South Bank University, London, UK

    Mohammad Osman Tokhi

  3. School of Engineering, Polytechnic Institute of Porto, Porto, Portugal

    Manuel F. Silva

  4. Universidade Federal de Santa Catarina, Blumenau, Santa Catarina, Brazil

    Leonardo Mejia Rincon

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Danilov, V., Klimov, K., Kapytov, D., Diane, S. (2024). Bipedal Walking Robot Control Using PMTG Architecture. In: Youssef, E.S.E., Tokhi, M.O., Silva, M.F., Rincon, L.M. (eds) Synergetic Cooperation between Robots and Humans. CLAWAR 2023. Lecture Notes in Networks and Systems, vol 811. Springer, Cham. https://doi.org/10.1007/978-3-031-47272-5_8

Download citation

Publish with us

Policies and ethics

Search

Navigation