Skip to main content
Springer Nature Link
Log in

Human Motion Reconstruction

  • Chapter
  • First Online:
Springer Handbook of Robotics

Part of the book series: Springer Handbooks ((SHB))

Abstract

This chapter presents a set of techniques for reconstructing and understanding human motions measured using current motion capture technologies. We first review modeling and computation techniques for obtaining motion and force information from human motion data (Sect. 68.2). Here we show that kinematics and dynamics algorithms for articulated rigid bodies can be applied to human motion data processing, with help from models based on knowledge in anatomy and physiology. We then describe methods for analyzing human motions so that robots can segment and categorize different behaviors and use them as the basis for human motion understanding and communication (Sect. 68.3). These methods are based on statistical techniques widely used in linguistics. The two fields share the common goal of converting continuous and noisy signal to discrete symbols, and therefore it is natural to apply similar techniques. Finally, we introduce some application examples of human motion and models ranging from simulated human control to humanoid robot motion synthesis.

figure a

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Abbreviations

3-D:

three-dimensional

6-D:

six-dimensional

CAD:

computer-aided design

CHMM:

coupled hidden Markov model

DMP:

dynamic movement primitive

DOF:

degree of freedom

EM:

expectation maximization

EMG:

electromyography

HHMM:

hierarchical hidden Markov model

HMM:

hidden Markov model

RNN:

recurrent neural network

SAI:

simulation and active interfaces

References

  1. M.J. Mataric: Getting humaniods to move and imitate, IEEE Intell. Syst. 15(4), 18–24 (2000)

    Article  Google Scholar 

  2. S. Nakaoka, A. Nakazawa, F. Kanehiro, K. Kaneko, M. Morisawa, H. Hirukawa, K. Ikeuchi: Learning from observation paradigm: Leg task models for enabling a biped humanoid robot to imitate human dances, Int. J.Robotics Res. 26(8), 829–844 (2010)

    Article  Google Scholar 

  3. Organic Motion, Inc.: OpenStage, http://organicmotion.com/mocap-for-animation

  4. Microsoft: Kinect for Xbox One, http://www.xbox.com/en-US/xbox-one/accessories/kinect-for-xbox-one

  5. M. Vondrak, L. Sigal, J. Hodgins, O. Jenkins: Video-based 3D motion capture through biped control, ACM Trans. Graph. 31(4), 24 (2012)

    Article  Google Scholar 

  6. Y. Nakamura, K. Yamane, Y. Fujita, I. Suzuki: somatosensory computation for man-machine interface from motion capture data and musculoskeletal human model, IEEE Trans.Robotics 21(1), 58–66 (2005)

    Article  Google Scholar 

  7. K. Yamane, Y. Nakamura: Natural motion animation through constraining and deconstraining at will, IEEE Trans. Vis.Comput. Graph. 9(3), 352–360 (2003)

    Article  Google Scholar 

  8. J.J. Craig: Introduction to Robotics: Mechanics and Control (Addison-Wesley, Reading 1986)

    Google Scholar 

  9. A.V. Hill: The heat of shortening and the dynamic constants of muscle, Proc. R. Soc. Lond. B 126, 136–195 (1938)

    Article  Google Scholar 

  10. S. Stroeve: Impedance chracteristics of a neuro-musculoskeletal model of the human arm I: Posture control, J.Biol. Cyberneics 81, 475–494 (1999)

    Article  MATH  Google Scholar 

  11. J. Rasmussen, M. Damsgaard, M. Voigt: Muscle recruitment by the min/max criterion—a comparative study, J.Biomech. 34(3), 409–415 (2001)

    Article  Google Scholar 

  12. K. Yamane, Y. Fujita, Y. Nakamura: Estimation of physically and physiologically valid somatosensory information, Proc.IEEE/RSJ Int. Conf.RoboticsAutom. (ICRA) (2005) pp. 2635–2641

    Google Scholar 

  13. B.W. Mooring, Z.S. Roth, M.R. Driels: Fundamentals of Manipulator Calibration (Wiley, New York 1991)

    Google Scholar 

  14. Digital Human Research Center, AIST: Human Body Properties Database, https://www.dh.aist.go.jp/database/properties/index-e.html

  15. K. Ayusawa, G. Venture, Y. Nakamura: Identification of humanoid robots dynamics using minimal set of sensors, Proc. IEEE/RSJ Int. Conf.Intell. RobotsSyst. (IROS) (2008) pp. 2854–2859

    Google Scholar 

  16. T. Kim, S.I. Park, S.Y. Shin: Nonmetric individual differences multidimensional scaling: An alternating least squres method with optimal scaling features, ACM Trans.Graph. 22(3), 392–401 (2003)

    Article  Google Scholar 

  17. T. Shiratori, A. Nakazawa, K. Ikeuchi: Detecting dance motion structure through music analysisg, Proc. 6th IEEE Int. Conf.Autom. FaceGesture Recognit. (2004) pp. 857–862

    Google Scholar 

  18. J. Kohlmorgen, S. Lemm: A dynamic HMM for on-line segmentation of sequential data, Proc.Conf.Neural Inf. Process. Syst. (2002) pp. 793–800

    Google Scholar 

  19. D. Kulic, W. Takano, Y. Nakamura: Online segmentation and clustering from continuous observation of whole body motions, IEEE Trans.Robotics 25(5), 1158–1166 (2009)

    Article  Google Scholar 

  20. J.L. Elman: Finding structure in time, Cogn. Sci. 14, 179–211 (1990)

    Article  Google Scholar 

  21. W. Takano, Y. Nakamura: Humanoid robot’s autonomous acquisition of proto-symbols through motion segmentation, Proc. IEEE-RAS Int. Conf.Humanoid Robots (2006) pp. 425–431

    Google Scholar 

  22. A.J. Ijspeert, J. Nakanishi, S. Shaal: Learning control policies for movement imitation and movement recognition, Neural Inf. Process. Syst. 15, 1547–1554 (2003)

    Google Scholar 

  23. M. Haruno, D. Wolpert, M. Kawato: MOSAIC model for sensorimotor learning and control, Neural Comput. 13, 2201–2220 (2001)

    Article  MATH  Google Scholar 

  24. T. Inamura, I. Toshima, H. Tanie, Y. Nakamura: Embodied symbol emergence based on mimesis theory, Int. J.Robotics Res. 23(4), 363–377 (2004)

    Article  Google Scholar 

  25. A. Billard, S. Calinon, F. Guenter: Discriminative and adaptive imitation in uni-manual and bi-manual tasks, RoboticsAuton. Syst. 54, 370–384 (2006)

    Article  Google Scholar 

  26. M. Okada, K. Tatani, Y. Nakamura: Polynomial design of the nonlinear dynamics for the brain-like information processing of whole body motion, Proc. IEEE Int. Conf.RoboticsAutom. (ICRA) (2002) pp. 1410–1415

    Google Scholar 

  27. A.J. Ijispeert, J. Nakanishi, T. Shibata, S. Schaal: Nonlinear dynamical systems for imitation with humanoid robots, Proc. IEEE-RAS Int. Conf.Humanoid Robots (2001)

    Google Scholar 

  28. T. Matsubara, S.H. Hyon, J. Morimoto: Learning parametric dynamic movement primitives from multiple demonstrations, Neural Netw. 24(5), 493–500 (2011)

    Article  Google Scholar 

  29. J. Tani, M. Ito: Self-organization of behavioral primitives as multiple attractor dynamics: A robot experiment, IEEE Trans. Syst. ManCybern. A 33(4), 481–488 (2003)

    Article  Google Scholar 

  30. L. Rabiner: A Tutorial on hidden Markov models and selected applications in speech recognition, Proceedings IEEE (1989) pp. 257–286

    Google Scholar 

  31. L. Kovar, M. Gleicher, F. Pighin: Motion graphs, ACM Trans.Graph. 21(3), 473–482 (2002)

    Article  Google Scholar 

  32. W. Takano, H. Imagawa, D. Kulic, Y. Nakamura: What do you expect from a robot that tells your future? The crystal ball, Proc. IEEE/RSJ Int. Conf.Intell. RobotsSyst. (IROS), Taipei (2008) pp. 1780–1785

    Google Scholar 

  33. Y. Sugita, J. Tani: Learning semantic combinatoriality from the interaction between linguistic and behavioral processes, Adapt. Behav. 3(1), 33–52 (2005)

    Article  Google Scholar 

  34. T. Ogata, M. Murase, J. Tani, K. Komatani, H.G. Okuno: Two-way translation of compound sentences and arm motions by recurrent neural networks, Proc. IEEE/RSJ Int. Conf.Intell. RobotsSyst. (IROS) (2007) pp. 1858–1863

    Google Scholar 

  35. P.F. Brown, S.A.D. Pietra, V.J.D. Pietra, R.L. Mercer: The mathematics of statistical machine translation: Parameter estimation, Comput. Linguist. 19(2), 263–311 (1993)

    Google Scholar 

  36. W. Takano, Y. Nakamura: Construction of a space of motion labels from their mapping to full-body motion symbols, Adv. Robotics 29(2), 115–126 (2015)

    Article  Google Scholar 

  37. W. Takano, Y. Nakamura: Statistical mutual conversion between whole body motion primitives and linguistic sentences for human motions, Int. J. Robot. Res. 34(10), 1314–1328 (2015)

    Article  Google Scholar 

  38. S. Fine, Y. Singer, N. Tishby: The hierarchical hidden markov model: Analysis and application, Mach. Learn. 32, 41–62 (1998)

    Article  MATH  Google Scholar 

  39. M. Brand, N. Oliver, A. Pentland: Coupled hidden Markov models for complex action recognition, Proc.IEEE Conf.Comput. Vis.Pattern Recognit. (1999) pp. 994–999

    Google Scholar 

  40. W. Takano, K. Yamane, T. Sugihara, K. Yamamoto, Y. Nakamura: Primitive communication based on motion recognition and generation with hierarchical mimesis model, Proc. IEEE Int. Conf.RoboticsAutom. (ICRA) (2006) pp. 3602–3609

    Google Scholar 

  41. D. Lee, C. Ott, Y. Nakamura: Mimetic communication model with compliant physical contact in human-humanoid interaction, Int. J.Robotics Res. 29(13), 1684–1704 (2004)

    Article  Google Scholar 

  42. E. Demerican, L. Sentis, V. De Sapio, O. Khatib: Human motion reconstruction by direct control of marker trajectories. In: Advances in Robot Kinematics: Analysis and Design, ed. by J. Lenarcic, P. Wenger (Springer, Berlin, Heidelberg 2008) pp. 263–272

    Chapter  Google Scholar 

  43. O. Khatib, O. Brock, K. Chang, F. Conti, D. Ruspini, L. Sentis: Robotics and interactive simulation, Commun. ACM 45(3), 46–51 (2002)

    Article  Google Scholar 

  44. J.M. Wang, S.R. Hamner, S.L. Delp, V. Koltun: Optimizing locomotion controllers using biologically-based actuators and objectives, ACM Trans. Robotics 31(4), 25 (2012)

    Google Scholar 

  45. V.B. Zordan, J.K. Hodgins: Motion capture-driven simulations that hit and react, Proc.ACM SIGGRAPH Symp.Comput. Animat. (2002) pp. 89–96

    Google Scholar 

  46. K.W. Sok, M.M. Kim, J.H. Lee: Simulating biped behaviors from human motion data, ACM Trans.Graph. 26(3), 107 (2007)

    Article  Google Scholar 

  47. M. Da Silva, Y. Abe, J. Popović: Interactive simulation of stylized human locomotion, ACM Trans.Graph. 27(3), 82 (2008)

    Article  Google Scholar 

  48. K. Miura, M. Morisawa, F. Kanehiro, S. Kajia, K. Kaneko, K. Yokoi: Human-like walking with toe supporting for humanoids, Proc. IEEE/RSJ Int. Conf.Intell. RobotsSyst. (IROS) (2011) pp. 4428–4435

    Google Scholar 

  49. C. Ott, D.H. Lee, Y. Nakamura: Motion capture based human motion recognition and imitation by direct marker control, Proc.IEEE-RAS Int. Conf.Humanoid Robots (2008) pp. 399–405

    Google Scholar 

  50. K. Yamane, J.K. Hodgins: Simultaneous tracking and balancing of humanoid robots for imitating human motion capture data, Proc.IEEE/RSJ Int. Conf.Intell. Robot Syst. (IROS) (2009) pp. 2510–2517

    Google Scholar 

  51. K. Yamane, S.O. Anderson, J.K. Hodgins: Controlling humanoid robots with human motion data: Experimental validation, Proc.IEEE-RAS Int. Conf.Humanoid Robots (2010) pp. 504–510

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Disney Research, 4720 Forbes Avenue, Suite 110, PA 15213, Pittsburgh, USA

    Katsu Yamane

  2. Department of Mechano-Informatics, University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, 113-8656, Tokyo, Japan

    Wataru Takano

Authors
  1. Katsu Yamane
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wataru Takano
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Katsu Yamane .

Editor information

Editors and Affiliations

  1. Department of Electrical Engineering, University of Naples Federico II, Via Claudio 21, 80125, Naples, Italy

    Bruno Siciliano

  2. Department of Computer Sciences, Artificial Intelligence Laboratory, Stanford University, 450 Serra Mall, CA 94305, Stanford, USA

    Oussama Khatib

Video-References

Video-References

:

Example of optical motion capture data converted to joint angle data available from http://handbookofrobotics.org/view-chapter/65/videodetails/762

:

Example of muscle tensions computed from motion capture data available from http://handbookofrobotics.org/view-chapter/65/videodetails/763

:

The Crystal Ball: Predicting future motions available from http://handbookofrobotics.org/view-chapter/65/videodetails/764

:

Human motion mapped to a humanoid robot available from http://handbookofrobotics.org/view-chapter/65/videodetails/765

:

Converting human motion to sentences available from http://handbookofrobotics.org/view-chapter/65/videodetails/766

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Yamane, K., Takano, W. (2016). Human Motion Reconstruction. In: Siciliano, B., Khatib, O. (eds) Springer Handbook of Robotics. Springer Handbooks. Springer, Cham. https://doi.org/10.1007/978-3-319-32552-1_68

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-32552-1_68

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-32550-7

  • Online ISBN: 978-3-319-32552-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics