Skip to main content
SpringerLink
Log in

Lower-body control of humanoid robot NAO via Kinect

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Humanoid robot has been concerned as it can perform some movements as human, especially imitating human motion in real time with motion tracking equipments. To imitate the human motion, there are still some challenges for the lower-body control of robot due to the physical difference between human and robot. In this paper, we propose a joint angle-based control (JAC) scheme for the lower-body control of humanoid robot to imitate human motion via Kinect sensor. Due to factors such as noise, tracking error and robot joint constrains, the motion information captured from the Kinect sensor applied to the robot directly will arise the problem of balance control. To overcome it, we optimize the joint angles in the lower-body of NAO, and define a gain factor to compensate the difference between the human motion and the robot so as to keep the balance of humanoid robot during imitation. Experimental results show that the proposed control scheme works efficiently even when the humanoid robot performs some complex movements such as standing on single foot.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Afthoni R, Rizal A, Susanto E (2013) Proportional derivative control based robot arm system using microsoft. In: International conference on robotics and intelligent computation systems (ROBIONETICS). IEEE, pp 24–29

  2. Aldebaran Robotics NAO. Avaliable: www.ald.softbankrobotics.com

  3. Almetwally I, Mallem M, Real-time tele-operation and tele-walking of humanoid robot Nao using Kinect depth camera (2013). In: Tenth international conference on networking, sensing and control (ICNSC). IEEE, pp 463–466

  4. Arbulu M, Padilla A, Ramirez F (2013) Geometric balancing control of humanoid robots. In: International conference on robotics and biomimetics (ROBIO). IEEE, pp 2136–2141

  5. Avalos J, Cortez S, Vasquez K, Murrary V, Ramos O (2016) Teleoperation using the Kinect sensor and NAO robot. In: VII latin american symposium on circuits and systems (LASCAS). IEEE, pp 303–306

  6. Bakker P, Kuniyoshi Y (1996) Robot see, robot do: an overview of robot imitation. In: AISB96 workshop on learning in robots and animals. Brighton, pp 3–11

  7. Cheng G, Kuniyoshi Y (2000) Real-time mimicking of human body motion by a humanoid robot. In: Proceedings of the sixth international conference on intelligent autonomous systems (IAS 2000). Venice, pp 273–280

  8. Cheng LY, Sun Q, Su H, Cong Y, Zhao SY (2012) Design and implementation of human-robot interactive demonstration system based on Kinect. In: Twenty-fourth chinese control and decision conference (CCDC). IEEE, pp 971–975

  9. Csapo A, Gilmartin E, Grizou J, Han JG, Meena R, Anastasiou D, Jokinen K, Wilcock G (2012) Speech, gaze and gesturing: multimodal conversational interaction with NAO robot. In: Third international conference on cognitive infocommunications. IEEE, pp 667–672

  10. Do M, Azad P, Asfour T, Dillmann R (2008) Imitation of human motion on a humanoid robot using non-linear optimization. In: Proceedings of 8th IEEE-RAS international conference on humanoid robots (humanoids). Daejeon, pp 545–552

  11. Igorevich R, Ismoilovich E, Min D (2011) Behavioral synchronization of human and humanoid robot. In: Eighth international conference on ubiquitous robot and ambient intelligence (URAI). IEEE, pp 655–660

  12. Indrajit W, Muis A (2013) Development of whole body motion imitation in humanoid robot. In: International conference on QiR. IEEE, pp 138–141

  13. Kim S, Kim C, You B, Oh S (2009) Stable whole-body motion generation for humanoid robots to imitate human motions. In: Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 2518–2524

  14. Koenemann J, Burget F, Bennewitz M, Real-time imitation of human whole-body motions by humanoids (2014). In: International conference on robotics & automation (ICRA). IEEE, pp 2806–2812

  15. Kofinas N, Orfanoudakis E, Lagoudakis MG (2013) Complete analytical inverse kinematics for NAO. In: Autonomous robot systems (Robotica), pp 1–6

  16. Lu H, Li Y, Chen M, Kim H, Serikawa S (2017) Brain intelligence: go beyond artificial intelligence. In: Mobile networks and application, pp 1–10

  17. Lu H, Li Y, Mu S, Wang D, Kim H, Serikawa S (2017) Motor anomaly detection for unmanned aerial vehicles using reinforcement learning. IEEE Internet of Things Journal 99:1–8

    Article  Google Scholar 

  18. Merai M, Naouar M, Slama-Belkhodja I, Monmasson E (2016) FPGA-based space vector delta modulation current controller for grid connected converters. In: IECON-42nd annual conference of the industrial electronics society. IEEE, pp 2301–2306

  19. Microsoft Kinect V2.0. Avaliable: www.kinectforwindows.com

  20. Nguyen V, Lee J (2012) Full-body imitation of human motions with Kinect and heterogeneous kinematics structure of humanoid robot. In: International symposium on system integration (SII). IEEE, pp 93–98

  21. Ou YS, Hu JB, Fu Y, Wu X (2015) A real-time human imitation system using Kinect. Int J Soc Robot, IEEE 7(5):587–600

    Article  Google Scholar 

  22. Riley M, Ude A, Wade K, Atkeson CG (2003) Enabling realtime full-body imitation: a natural way of transferring human movement to humanoids. In: Proceedings of the 2003 IEEE international conference on robotics and automation (ICRA2003). Taipei, pp 2368–2374

  23. Serikawa S, Lu H (2014) Underwater image dehazing using joint trilateral filter. Comput Electr Eng 40(1):41–50

    Article  Google Scholar 

  24. Shon A, Storz J, Rao R (2007) Towards a real-time bayesian imitation system for a humanoidrobot. In: Proceedings of the 2007 IEEE international conference on robotics and automation (ICRA 2007). Rome, pp 2847–2852

  25. Stephens B (2007) Integral control of humanoid balance. In: International conference on intelligent robots and systems. IEEE, pp 4020–4027

  26. Thobbi A, Sheng W (2010) Imitation learning of arm gestures in presence of missing data for humanoid robots. In: Proceedings of 10th IEEE-RAS international conference on humanoid robots (humanoids). Nashville, pp 92–97

  27. Tolani D, Badler NI (1996) Real-time inverse kinematics of the human arm. Massachusetts Institute of Technology, pp 393–401

  28. Wang BC, Yang CG, Xie Q (2012) Human-machine interfaces based on EMG and Kinect applied to teleoperation of a mobile humanoid robot. In: Tenth world congress on intelligent control and automation. IEEE, pp 3903–3908

  29. Wang F, Tang C, Ou YS, Xu YS (2012) A real-time human imitation system. In: Proceedings of the 10th world congress on intelligent control and automation. IEEE, pp 3692–3697

  30. Yamane K, Anderson SO, Hodgins JK (2010) Controlling humanoid robots with human motion data: experimental validation. In: IEEE-RAS international conference on humanoid robots, pp 504–510

  31. Yang NJ, Wei YD, Liu C, Xu BB, Zhang J (2013) A study of the human-robot synchronous control system based on skeletal tracking technology. In: International conference on robotics and biomimetics (ROBIO). IEEE, pp 2191–2196

  32. Zatsiorsky V, Seluyanov V (1983) The mass and inertia characteristics of the main segments of the human body. Biomechanics VIII-B. Human Kinetic. Illinois, pp 1152–1159

  33. Zhang L, Cheng Z, Gan Y et al (2016) Fast human whole body motion imitation algorithm for humanoid robots. In: Proceedings of the 2016 IEEE international conference on robotics and biomimetics. Qingdao, China, December 3–7, pp 1430–1435

  34. Zhao X, Huang Q, Peng Z, Li K (2004) Kinematics mapping and similarity evaluation of humanoid motion based on human motion capture. In: Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (IROS 2004). Sendai, pp 840–845

  35. Zuher F, Romero R (2012) Recognition of human motions for imitation and control of a humanoid robot. In: Robotics symposium and latin american robotics symposium (SBR-LARS), Brazilian, Fortaleza, pp 190–195

Download references

Acknowledgments

This work was supported by the Technique Innovation Training Program (No. 201610293001Z), Key Lab of Broadband Wireless Communication and Sensor Network Technology (Nanjing University of Posts and Telecommunications, Ministry of Education, JZNY201704), Nanjing University of Posts and Telecommunications (NY217021), Natural Science Foundation of Jiangsu Province (BK20140891), National Natural Science Foundation of China (Grant No. 61401228), China Postdoctoral Science Foundation (Grant No. 2015M581841), and Postdoctoral Science Foundation of Jiangsu Province (Grant No. 1501019A).

Author information

Authors and Affiliations

  1. Nanjing University of Posts and Telecommunications, Nanjing, China

    Jianxin Chen, Guanwen Wang, Xiao Hu & Jiayun Shen

Authors
  1. Jianxin Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guanwen Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiao Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jiayun Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jianxin Chen.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, J., Wang, G., Hu, X. et al. Lower-body control of humanoid robot NAO via Kinect. Multimed Tools Appl 77, 10883–10898 (2018). https://doi.org/10.1007/s11042-017-5332-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-5332-3

Keywords

Search

Navigation