Skip to main content
Springer Nature Link
Log in

A motion imitation system for humanoid robots with inference-based optimization and an auditory user interface

  • Original Article
  • Published:
Artificial Life and Robotics Aims and scope Submit manuscript

Abstract

Humanoid robots are expected to perform various jobs replacing human beings, but it is difficult for non-expert users to control them. To control humanoid robots easily, various motion imitation systems, in which a humanoid robot does the same motion as a human teacher, have been studied. In the present study, we make two contributions to building easy-to-use imitation systems. The first contribution is a flexible learning system that enables both synchronous real-time imitation, in which the robot imitates human motion almost immediately, and batch imitation, in which the robot’s motion is optimized more accurately after longer measurement of human motion. We use an efficient inference-based optimization method to enable both learning modes. The other contribution is an auditory user interface that accepts voice commands from the user and at the same time identifies the target person to be imitated. This enables the user to easily use the imitation system even when other people are present. Experimental results show that the proposed system can be used in both imitation modes and that it can identify the target person.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

We’re sorry, something doesn't seem to be working properly.

Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Notes

  1. http://www.ros.org/.

  2. http://wiki.ros.org/rwt_speech_recognition/.

References

  1. Ude A, Atkeson CG, Riley M (2004) Programming full-body movements for humanoid robots by observation. Robot Auton Syst 47(2–3):93–108

    Article  Google Scholar 

  2. Elbasiony R, Gomaa W (2018) Humanoids skill learning based on real-time human motion imitation using kinect. Intel Serv Robot 11(2):149–169

    Article  Google Scholar 

  3. Dariush B, Gienger M, Arumbakkam A, Zhu Y, Jian B, Fujimura K, Goerick C (2009) Online transfer of human motion to humanoids. Int J Hum Robot 6(02):265–289

    Article  Google Scholar 

  4. Montecillo-Puente FJ, Sreenivasa M, Laumond JP (2010) On real-time whole-body human to humanoid motion transfer. In: Proceedings of the 7th international conference on informatics in control, automation and robotics (ICINCO 2010), pp 22–31

  5. Ou Y, Hu J, Wang Z, Fu Y, Wu X, Li X (2015) A real-time human imitation system using kinect. Int J Soc Robot 7(5):587–600

    Article  Google Scholar 

  6. Alibeigi M, Rabiee S, Ahmadabadi MN (2017) Inverse kinematics based human mimicking system using skeletal tracking technology. J Intell Robot Syst 85(1):27–45

    Article  Google Scholar 

  7. Nakaoka S, Nakazawa A, Kanehiro F, Kaneko K, Morisawa M, Hirukawa H, Ikeuchi K (2007) Learning from observation paradigm: leg task models for enabling a biped humanoid robot to imitate human dances. Int J Robot Res 26(8):829–844

    Article  Google Scholar 

  8. Munirathinam K, Chevallereau C, Sakka S (2014) Offline imitation of a human motion by a humanoid robot under balance constraint. Springer, Cham, pp 269–282

    Google Scholar 

  9. Toussaint M (2009) Robot trajectory optimization using approximate inference. In: Proceedings of the 26th international conference on machine learning, pp 1049–1056

  10. Itoh H, Sakai Y, Kadoya T, Fukumoto H, Wakuya H, Furukawa T (2017) Using model uncertainty for robust optimization in approximate inference control. Artif Life Robot 22(3):327–335

    Article  Google Scholar 

  11. Zarubin D, Pokorny FT, Song D, Toussaint M, Kragic D (2013) Topological synergies for grasp transfer. In: Hand Synergies-how to tame the complexity of grapsing, Workshop, IEEE international conference on robotics and automation

  12. Watter M, Springenberg J, Boedecker J, Riedmiller M (2015) Embed to control: a locally linear latent dynamics model for control from raw images. In: Advances in neural information processing systems, pp 2728–2736

  13. Rawlik K, Zarubin D, Toussaint M, Vijayakumar S (2018) An approximate inference approach to temporal optimization for robotics. In: Robotics research. Springer, pp 145–162

  14. Toussaint M (2009) Pros and cons of truncated Gaussian EP in the context of approximate inference control. NIPS workshop on probabilistic approaches for robotics and control

  15. Rüeckert E, Mindt M, Peters J, Neumann G (2014) Robust policy updates for stochastic optimal control. In: Humanoid robots (humanoids), 2014 14th IEEE-RAS international conference on, pp 388–393

  16. Zarubin D, Ivan V, Toussaint M, Komura T, Vijayakumar S (2013) Hierarchical motion planning in topological representations. Robotics, pp 465–472

  17. Schaal S (1999) Is imitation learning the route to humanoid robots? Trends Cogn Sci 3(6):233–242

    Article  Google Scholar 

  18. Breazeal C, Scassellati B (2002) Robots that imitate humans. Trends Cogn Sci 6(11):481–487

    Article  Google Scholar 

  19. Hussein A, Gaber MM, Elyan E, Jayne C (2017) Imitation learning: a survey of learning methods. ACM Comput Surv 50(2):21

    Article  Google Scholar 

  20. Grimes DB, Chalodhorn R, Rao RP (2006) Dynamic imitation in a humanoid robot through nonparametric probabilistic inference. In: Robotics: science and systems, pp 199–206

  21. Cole JB, Grimes DB, Rao RP (2007) Learning full-body motions from monocular vision: dynamic imitation in a humanoid robot. In: 2007 IEEE/RSJ international conference on intelligent robots and systems. IEEE, pp 240–246

  22. Ding J, Shi JY (2017) Kinect microphone array-based speech and speaker recognition for the exhibition control of humanoid robots. Comput Electr Eng 62:719–729

    Article  Google Scholar 

  23. Nakashima M, Itoh H, Fukumoto H, Wakuya H, Furukawa T (2016) Developing a ROS node to control ICS servo motors. In: Proceedings of the 21st international symposium on artificial life and robotics, pp 107–110

Download references

Acknowledgements

We would like to thank the anonymous reviewers for their valuable comments. This study was partially supported by the Ministry of Education, Culture, Sports, Science and Technology in Japan, Grant-in-Aid for Scientific Research (C) 15K00341 and 19K12157.

Author information

Authors and Affiliations

  1. Department of Electrical and Electronic Engineering, Faculty of Science and Engineering, Saga University, 1 Honjo-machi, Saga, 840-8502, Japan

    Hideaki Itoh, Hisao Fukumoto & Hiroshi Wakuya

  2. Department of Electrical and Electronic Engineering, Graduate School of Science and Engineering, Saga University, 1 Honjo-machi, Saga, 840-8502, Japan

    Nozomi Ihara

Authors
  1. Hideaki Itoh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nozomi Ihara
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hisao Fukumoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hiroshi Wakuya
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nozomi Ihara.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was presented in part at the 24th International Symposium on Artificial Life and Robotics, Beppu, Oita, January 23–25, 2019.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Itoh, H., Ihara, N., Fukumoto, H. et al. A motion imitation system for humanoid robots with inference-based optimization and an auditory user interface. Artif Life Robotics 25, 106–115 (2020). https://doi.org/10.1007/s10015-019-00575-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10015-019-00575-5

Keywords