Skip to main content
SpringerLink
Log in

Incremental Learning of Full Body Motion Primitives

  • Chapter
Book cover From Motor Learning to Interaction Learning in Robots

Part of the book series: Studies in Computational Intelligence ((SCI,volume 264))

Abstract

This paper describes an approach for autonomous and incremental learning of motion pattern primitives by observation of human motion. Human motion patterns are abstracted into a dynamic stochastic model, which can be used for both subsequent motion recognition and generation. As new motion patterns are observed, they are incrementally grouped together using local clustering based on their relative distance in the model space. The clustering algorithm forms a tree structure, with specialized motions at the tree leaves, and generalized motions closer to the root. The generated tree structure will depend on the type of training data provided, so that the most specialized motions will be those for which the most training has been received. A complete system for online acquisition and visualization of motion primitives from continuous observation of human motion will also be described, allowing interactive training.

This chapter is based on work by the authors first reported in the International Journal of Robotics Research [39] and the International Symposium on Robot and Human Interactive Communication (RO-MAN) 2009 [32].

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abbeel, P., Ng, A.Y.: Apprenticeship learning via inverse reinforcement learning. In: Proceedings of the International Conference on Machine Learning (2004)

    Google Scholar 

  2. Bennewitz, M., Burgard, W., Cielniak, G., Thrun, S.: Learning motion patterns of people for compliant robot motion. International Journal of Robotics Research 24(1), 31–48 (2005)

    Article  Google Scholar 

  3. Bentivegna, D.C., Atkeson, C.G., Cheng, G.: Learning similar tasks from observation and practice. In: Proceedings of the International Conference on Intelligent Robots and Systems, pp. 2677–2683 (2006)

    Google Scholar 

  4. Bernardin, K., Ogawara, K., Ikeuchi, K., Dillmann, R.: A sensor fusion approach for recognizing continuous human grasping sequences using hidden markov models. IEEE Transactions on Robotics 21(1), 47–57 (2005)

    Article  Google Scholar 

  5. Betkowska, A., Shinoda, K., Furui, S.: Fhmm for robust speech recognition in home environment. In: Symposium on Large Scale Knowledge Resources, pp. 129–132 (2006)

    Google Scholar 

  6. Billard, A., Calinon, S., Guenter, F.: Discriminative and adaptive imitation in uni-manual and bi-manual tasks. Robotics and Autonomous Systems 54, 370–384 (2006)

    Article  Google Scholar 

  7. Breazeal, C., Scassellati, B.: Robots that imitate humans. Trends in Cognitive Sciences 6(11), 481–487 (2002)

    Article  Google Scholar 

  8. Chalodhorn, R., Rao, R.P.N.: Learning to imitate human actions through eigenposes. In: Sigaud, O., Peters, J. (eds.) From Motor Learning to Interaction Learning in Robots. SCI, vol. 264, pp. 357–381. Springer, Heidelberg (2010)

    Google Scholar 

  9. Dillmann, R.: Teaching and learning of robot tasks via observation of human performance. Journal of Robotics and Autonomous Systems 47, 109–116 (2004)

    Article  Google Scholar 

  10. Dillmann, R., Rogalla, O., Ehrenmann, M., Zollner, R., Bordegoni, M.: Learning robot behaviour and skills based on human demonstration and advice: The machine learning paradigm. In: Proceedings of the International Symposium on Robotics Research, pp. 229–238 (1999)

    Google Scholar 

  11. Dixon, K.R., Dolan, J.M., Khosla, P.K.: Predictive robot programming: Theoretical and experimental analysis. International Journal of Robotics Research 23(9), 955–973 (2004)

    Article  Google Scholar 

  12. Dominey, P.F., Metta, G., Nori, F., Natale, L.: Anticipation and initiative in human-humanoid interaction. In: Proceedings of the IEEE International Conference on Humanoid Robots, pp. 693–699 (2008)

    Google Scholar 

  13. Ekvall, S., Aarno, D., Kragic, D.: Online task recognition and real-time adaptive assistance for computer-aided machine control. IEEE Transactions on Robotics 22(5), 1029–1033 (2006)

    Article  Google Scholar 

  14. Erlhagen, W., Mukovskiy, A., Bicho, E., Panin, G., Kiss, C., Knoll, A., van Schie, H., Bekkering, H.: Goal-directed imitation for robots: A bio-inspired approach to action understanding and skill learning. Robotics and Autonomous Systems 54, 353–360 (2006)

    Article  Google Scholar 

  15. Ghahramani, Z., Jordan, M.I.: Factorial hidden markov models. Machine Learning 29, 245–273 (1997)

    Article  MATH  Google Scholar 

  16. Ho, M.A.T., Yamada, Y., Umetani, Y.: An adaptive visual attentive tracker for human communicational behaviors using hmm-based td learning with new state distinction cpapbility. IEEE Transactions on Robotics 21(3), 497–504 (2005)

    Article  Google Scholar 

  17. Iba, S., Paredis, C.J.J., Khosla, P.K.: Interactive multi-modal robot programming. International Journal of Robotics Research 24(1), 83–104 (2005)

    Article  Google Scholar 

  18. Inamura, T., Toshima, I., Tanie, H., Nakamura, Y.: Embodied symbol emergence based on mimesis theory. The International Journal of Robotics Research 23(4-5), 363–377 (2004)

    Article  Google Scholar 

  19. Inamura, T., Nakamura, Y., Ezaki, H., Toshima, I.: Imitation and primitive symbol acquisition of humanoids by the integrated mimesis loop. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 4208–4213 (2001)

    Google Scholar 

  20. Inamura, T., Toshima, I., Nakamura, Y.: Acquisition and embodiment of motion elements in closed mimesis loop. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 1539–1544 (2002)

    Google Scholar 

  21. Inamura, T., Toshima, I., Nakamura, Y.: Acquisition and embodiment of motion elements in closed mimesis loop. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 1032–1037 (2002)

    Google Scholar 

  22. Jacobs, R.A., Jiang, W., Tanner, M.A.: Factorial hidden markov models and the generalized backfitting algorithm. Neural Computation 14, 2415–2437 (2002)

    Article  MATH  Google Scholar 

  23. Jain, A.K., Murty, M.N., Flynn, P.J.: Data clustering: A review. ACM Computing Surveys 31(3), 264–323 (1999)

    Article  Google Scholar 

  24. Janus, B., Nakamura, Y.: Unsupervised probabilistic segmentation of motion data for mimesis modeling. In: Proceedings of the IEEE International Conference on Advanced Robotics, pp. 411–417 (2005)

    Google Scholar 

  25. Jenkins, O.C., Matarić, M.: Performance-derived behavior vocabularies: Data-driven acquisition of skills from motion. International Journal of Humanoid Robotics 1(2), 237–288 (2004)

    Article  Google Scholar 

  26. Jenkins, O.C., Matarić, M.: A spatio-temporal extension to isomap nonlinear dimension reduction. In: Proceedings of the International Conference on Machine Learning, pp. 441–448 (2004)

    Google Scholar 

  27. Kadone, H., Nakamura, Y.: Symbolic memory for humanoid robots using hierarchical bifurcations of attractors in nonmonotonic neural networks. In: Proceedings of the International Conference on Intelligent Robots and Systems, pp. 2900–2905 (2005)

    Google Scholar 

  28. Keysers, C., Perrett, D.I.: Demystifying social cognition: a hebbian perspective. Trends in Cognitive Sciences 8(11), 501–507 (2004)

    Article  Google Scholar 

  29. Kohlmorgen, J., Lemm, S.: A dynamic hmm for on-line segmentation of sequential data. In: Dietterich, T.G., Becker, S., Ghahramani, Z. (eds.) NIPS 2001: Advances in Neural Information Processing Systems, vol. 14, pp. 793–800 (2002)

    Google Scholar 

  30. Kragic, D., Marayong, P., Li, M., Okamura, A.M., Hager, G.D.: Human-machine collaborative systems for microsurgical applications. International Journal of Robotics Research 24(9), 731–742 (2005)

    Article  Google Scholar 

  31. Krueger, V., Kragic, D., Ude, A., Geib, C.: The meaning of action: A review on action recognition and mapping. Advanced Robotics 21(13), 1473–1501 (2007)

    Google Scholar 

  32. Kulić, D., Imagawa, H., Nakamura, Y.: Online acquisition and visualization of motion primitives for humanoid robots. In: Proceedings of the IEEE International Symposium on Robot and Human Interactive Communication (to appear, 2009)

    Google Scholar 

  33. Kulić, D., Lee, D., Ott, Ch., Nakamura, Y.: Incremental learning of full body motion primitives for humanoid robots. In: Proceedings of the IEEE International Conference on Humanoid Robots, pp. 326–332 (2008)

    Google Scholar 

  34. Kulić, D., Nakamura, Y.: Incremental learning and memory consolidation of whole body motion patterns. In: Proceedings of the International Conference on Epigenetic Robotics, pp. 61–68 (2008)

    Google Scholar 

  35. Kulić, D., Nakamura, Y.: Scaffolding on-line segmentation of full body human motion patterns. In: Proceedings of the IEEE/RJS International Conference on Intelligent Robots and Systems, pp. 2860–2866 (2008)

    Google Scholar 

  36. Kulić, D., Takano, W., Nakamura, Y.: Incremental learning of full body motions via adaptive factorial hidden markov models. In: Proceedings of the International Conference on Epigenetic Robotics, pp. 69–76 (2007)

    Google Scholar 

  37. Kulić, D., Takano, W., Nakamura, Y.: Incremental on-line hierarchical clustering of whole body motion patterns. In: Proceedings of the IEEE International Symposium on Robot and Human Interactive Communication, pp. 1016–1021 (2007)

    Google Scholar 

  38. Kulić, D., Takano, W., Nakamura, Y.: Representability of human motions by factorial hidden markov models. In: Proceedings of the IEEE International Conference on Intelligent Robots and Systems, pp. 2388–2393 (2007)

    Google Scholar 

  39. Kulić, D., Takano, W., Nakamura, Y.: Incremental learning, clustering and hierarchy formation of whole body motion patterns using adaptive hidden markov chains. International Journal of Robotics Research 27(7), 761–784 (2008)

    Article  Google Scholar 

  40. Kuniyoshi, Y., Inaba, M., Inoue, H.: Teaching by showing: Generating robot programs by visual observation of human performance. In: Proceedings of the International Symposium on Industrial Robots, pp. 119–126 (1989)

    Google Scholar 

  41. Kuniyoshi, Y., Inoue, H.: Learning by watching: Extracting reusable task knowledge from visual observation of human performance. IEEE Transactions on Robotics and Automation 10(6), 799–822 (1994)

    Article  Google Scholar 

  42. Lee, D., Nakamura, Y.: Mimesis from partial observations. In: Proceedings of the International Conference on Intelligent Robots and Systems, pp. 1911–1916 (2005)

    Google Scholar 

  43. Lee, D., Nakamura, Y.: Mimesis scheme using a monocular vision system on a humanoid robot. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 2162–2167 (2007)

    Google Scholar 

  44. Liu, S., Asada, H.: Transferring manipulative skills to robots: representation and acquisition of tool manipulative skills using a process dynamics model. Journal of Dynamic Systems, Measurement and Control 114(2), 220–228 (1992)

    Article  MATH  Google Scholar 

  45. Lockerd, A., Breazeal, C.: Tutelage and socially guided robot learning. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3475–3480 (2004)

    Google Scholar 

  46. Lopes, M., Melo, F., Montesano, L., Santos-Victor, J.: Abstraction Levels for Robotic Imitation: Overview and Computational Approaches. In: Sigaud, O., Peters, J. (eds.) From Motor Learning to Interaction Learning in Robots. SCI, vol. 264, pp. 313–355. Springer, Heidelberg (2010)

    Google Scholar 

  47. Murphy, K.P.: Bayesian map learning in dynamic environments. In: Neural Information Processing Systems (1999)

    Google Scholar 

  48. Bagnell, J.A., Ratliff, N.D., Silver, D.: Learning to search: Functional gradient techniques for imitation learning. Autonomous Robots 27(1), 25–53 (2009)

    Article  Google Scholar 

  49. Nakamura, Y., Inamura, T., Tanie, H.: A stochastic model of embodied symbol emergence. In: Proceedings of the International Symposium of Robotics Research (2003)

    Google Scholar 

  50. Nakamura, Y., Takano, W., Yamane, K.: Mimetic communication theory for humanoid robots interacting with humans. In: Proceedings of the International Symposium of Robotics Research (2005)

    Google Scholar 

  51. Nakanishi, J., Morimoto, J., Endo, G., Cheng, G., Schaal, S., Kawato, M.: Learning from demonstration adn adaptation of biped locomotion. Robotics and Autonomous Systems 47, 79–91 (2004)

    Article  Google Scholar 

  52. Ng, A.Y., Russell, S.: Algorithms for inverse reinforcement learning. In: Proceedings of the International Conference on Machine Learning (2000)

    Google Scholar 

  53. Nicolescu, M.N., Matarić, M.J.: Task learning through imitation and human-robot interaction. In: Dautenhahn, K., Nehaniv, C. (eds.) Imitation and social learning in robots, humans and animals: behavioral, social and communicative dimensions. Cambridge University Press, Cambridge (2005)

    Google Scholar 

  54. Ogata, T., Sugano, S., Tani, J.: Open-end human-robot interaction from the dynamical systems perspective: mutual adaptation and incremental learning. Advanced Robotics 19, 651–670 (2005)

    Article  Google Scholar 

  55. Okada, M., Tatani, K., Nakamura, Y.: Polynomial design of the nonlinear dynamics for the brain-like information processing of whole body motion. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 1410–1415 (2002)

    Google Scholar 

  56. Peters, J., Schaal, S.: Reinforcement learning of motor skills with policy gradients. Neural Networks 21(4), 682–697 (2008)

    Article  Google Scholar 

  57. Rabiner, L.R.: A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE 77(2), 257–286 (1989)

    Article  Google Scholar 

  58. Schaal, S., Ijspeert, A., Billard, A.: Computational approaches to motor learning by imitation. Philosophical Transactions of the Royal Society of London B: Biological Sciences 358, 537–547 (2003)

    Article  Google Scholar 

  59. Startner, T., Pentland, A.: Visual recognition of american sign language using hidden markov models. In: Proceedings of the International Conference on Automatic Face and Gesture Recognition, pp. 189–194 (1995)

    Google Scholar 

  60. Takano, W.: Stochastic segmentation, proto-symbol coding and clustering of motion patterns and their application to signifiant communication between man and humanoid robot. PhD thesis, University of Tokyo (2006)

    Google Scholar 

  61. Takano, W., Nakamura, Y.: Humanoid robot’s autonomous acquisition of proto-symbols through motion segmentation. In: Proceedings of the IEEE International Conference on Humanoid Robots, pp. 425–431 (2006)

    Google Scholar 

  62. Tatani, K., Nakamura, Y.: Dimensionality reduction and reproduction with hierarchical nlpca neural networks. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 1927–1932 (2003)

    Google Scholar 

  63. Williams, B.H., Toussaint, M., Storkey, A.: Modelling motion primitives and their timing in biologically executed movements. Advances in Neural Information Processing Systems 20, 1609–1616 (2008)

    Google Scholar 

  64. Yamane, K., Nakamura, Y.: Natural motion animation through constraining and deconstraining at will. IEEE Transactions on Visualization and Computer Graphics 9(3), 352–360 (2003)

    Article  Google Scholar 

  65. Yang, J., Xu, Y., Chen, C.S.: Human action learning via hidden markov model. IEEE Transactions on Systems, Man and Cybernetics - Part A: Systems and Humans 27(1), 34–44 (1997)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Electrical and Computer Engineering Department, University of Waterloo, 200 University Avenue West, Waterloo, Ontario, Canada, N2L 3G1

    Dana Kulić

  2. Department of Mechano-Informatics, University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, 113-8656, Tokyo, Japan

    Yoshihiko Nakamura

Authors
  1. Dana Kulić
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yoshihiko Nakamura
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Institut des Systèmes Intelligents et de Robotique (CNRS UMR 7222), Université Pierre et Marie Curie Pyramide, Tour 55 Boîte courrier 173, 4 Place Jussieu, 75252, PARIS cedex 05, France

    Olivier Sigaud

  2. Dept. Schölkopf, Max-Planck Institute for Biological Cybernetics, Spemannstraße 38,Rm 223, 72076, Tübingen, Germany

    Jan Peters

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Kulić, D., Nakamura, Y. (2010). Incremental Learning of Full Body Motion Primitives. In: Sigaud, O., Peters, J. (eds) From Motor Learning to Interaction Learning in Robots. Studies in Computational Intelligence, vol 264. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-05181-4_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-05181-4_16

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-05180-7

  • Online ISBN: 978-3-642-05181-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation