Skip to main content
SpringerLink
Log in
Book cover

Dance Notations and Robot Motion pp 117–137Cite as

Bayesian Approaches for Learning of Primitive-Based Compact Representations of Complex Human Activities

  • Chapter
  • First Online:

Part of the book series: Springer Tracts in Advanced Robotics ((STAR,volume 111))

Abstract

Human full-body activities, such as choreographed dances, are comprised of sequences of individual actions. Research in motor control shows that such individual actions can be approximated by superpositions of simplified elements, called movement primitives. Such primitives can be employed to model complex coordinated movements, as occurring in martial arts or dance. In this chapter, we will briefly outline several biologically-inspired definitions of movement primitives and will discuss a new algorithm that unifies many existing models and which identifies such primitives with higher accuracy than alternative unsupervised learning techniques. We combine this algorithm with methods from Bayesian inference to optimize the complexity of the learned models and to identify automatically the best generative model underlying the identification of such primitives. We also discuss efficient probabilistic methods for the automatic segmentation of action sequences. The developed unsupervised segmentation method is based on Bayesian binning, an algorithm that models a longer data stream by the concatenation of an optimal number of segments, at the same time estimating the optimal temporal boundaries between those segments. Applying this algorithm to motion capture data from a TaeKwonDo form, and comparing the automatically generated segmentation results with human psychophysical data, we found a good agreement between automatically generated segmentations and human performance. Furthermore, the segments agree with the minimum jerk hypothesis about human movement [32]. These results suggest that a similar approach might be useful for the decomposition of dances into primitive-like movement components, providing a new approach for the derivation of compressed descriptions of dances that is based on principles from biological motor control.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    The diagrams for all Taegueks can be viewed on www.taekwondo.de.

  2. 2.

    The a priori independent gating variables and their Bernoulli priors induce a Binomial prior on the number of segments, which is a special case of the general priors on segment number boundaries which we developed previously [27]. The latter need a dependency model between the gating variables, which we do not consider here for the sake of simplicity.

  3. 3.

    Strictly speaking, any priors that allow for an evaluation of posterior expectations in closed form are suitable, but conjugate priors are particularly convenient.

References

  1. Y. Agam, R. Sekuler, Geometric structure and chunking in reproduction of motion sequences. Journal of Vision 8(1), 1–12 (2008)

    Article  Google Scholar 

  2. O. Arikan, D.A. Forsyth, Interactive motion generation from examples. ACM Trans. Graph. 21, 483–490 (2002)

    Article  MATH  Google Scholar 

  3. J. Barbiŏ, A. Safonova, J.-Y. Pan, C. Faloutsos, J.K. Hodgins, N.S. Pollard, Segmenting motion capture data into distinct behaviors, in Proceedings of Graphics Interface 2004, GI ’04, pp. 185–194, School of Computer Science, University of Waterloo, Waterloo, Ontario, Canada, 2004. Canadian Human-Computer Communications Society

    Google Scholar 

  4. L. Baum, T. Petrie, G. Soules, N. Weiss, A maximization technique occuring in the statistical analysis of probabilistic functions of markov chains. Ann. Math. Stat. 41(1), 164–171

    Google Scholar 

  5. R. Bellman, On the approximation of curves by line segments using dynamic programming. Commun. ACM. 4(6), 284 (1961)

    Google Scholar 

  6. B. Berret, F. Bonnetblanc, C. Papaxanthis, T. Pozzo, Modular control of pointing beyond arm’s length. J. Neurosci. 29(1), 191–205 (2009)

    Article  Google Scholar 

  7. C.M. Bishop, Pattern Recognition and Machine Learning. Springer, Berlin (2007)

    Google Scholar 

  8. E. Bizzi, V.C.K. Cheung, A. d’Avella, P. Saltiel, M. Tresch, Combining modules for movement. Brain Res. Rev. 57(1), 125–133 (2008)

    Article  Google Scholar 

  9. P. Bofill, Underdetermined blind separation of delayed sound sources in the frequency domain. Neurocomputing 55(34), 627–641 (2003). (Evolving Solution with Neural Networks)

    Google Scholar 

  10. P. Bottomer, Ballroom Dancing Step-By-Step. Anness Publishing, London, UK (2012)

    Google Scholar 

  11. R.B. Cattell, The scree test for the number of factors. Multivar. Behav. Res. 1(2), 245–276 (1966)

    Article  Google Scholar 

  12. V.C.K. Cheung, A. d’Avella, M.C. Tresch, E. Bizzi, Central and sensory contributions to the activation and organization of muscle synergies during natural motor behaviors. J. Neurosci. 25(27), 6419–6434 (2005)

    Article  Google Scholar 

  13. E. Chiovetto, B. Berret, I. Delis, S. Panzeri, T. Pozzo, Investigating reduction of dimensionality during single-joint elbow movements: a case study on muscle synergies. Front. Comput. Neurosci. 7, 11 (2013)

    Article  Google Scholar 

  14. E. Chiovetto, B. Berret, T. Pozzo, Tri-dimensional and triphasic muscle organization of whole-body pointing movements. Neuroscience 170(4), 1223–1238 (2010)

    Article  Google Scholar 

  15. E. Chiovetto, A. d’ Avella, M.A. Giese, A unifying algorithm for the identification of kinematic and electromyographic motor primitives. Presented at the international conference on the neural control of movement, Puerto Rico, (April 2013)

    Google Scholar 

  16. E. Chiovetto, M.A. Giese, Kinematics of the coordination of pointing during locomotion. PLoS ONE 8(11), e79555 (2013)

    Article  Google Scholar 

  17. E. Chiovetto, L. Patan, T. Pozzo, Variant and invariant features characterizing natural and reverse whole-body pointing movements. Exp. Brain Res. 218(3), 419–431 (2012)

    Article  Google Scholar 

  18. A. d’Avella, A. Portone, L. Fernandez, F. Lacquaniti, Control of fast-reaching movements by muscle synergy combinations. J. Neurosci. 26(30), 7791–7810 (2006)

    Article  Google Scholar 

  19. A. d’Avella, P. Saltiel, E. Bizzi, Combinations of muscle synergies in the construction of a natural motor behavior. Nat. Neurosci. 6(3), 300–308 (2003)

    Article  Google Scholar 

  20. A. d’Avella, M.C. Tresch, Modularity in the motor system: decomposition of muscle patterns as combinations of time-varying synergies, in Advances in Neural Information Processing Systems, vol. 14, ed. by S.A.S. Michael, I. Jordan, Michael J. Kearns (MIT Press, Cambridge, MA, 2002), pp. 141–148

    Google Scholar 

  21. I. Delis, S. Panzeri, T. Pozzo, B. Berret, A unifying model of concurrent spatial and temporal modularity in muscle activity. J. Neurophysiol. 111(3), 675–693 (2014)

    Article  Google Scholar 

  22. N. Dominici, Y.P. Ivanenko, G. Cappellini, A. d’Avella, V. Mond, M. Cicchese, A. Fabiano, T. Silei, A. Di Paolo, C. Giannini, R.E. Poppele, F. Lacquaniti, Locomotor primitives in newborn babies and their development. Science 334(6058), 997–999 (2011)

    Article  Google Scholar 

  23. B. Emile, P. Common, Estimation of time delays between unknown colored signals. Sig. Process. 68(1), 93–100 (1998)

    Article  Google Scholar 

  24. D. Endres, E. Chiovetto, M. Giese, Model selection for the extraction of movement primitives. Front. Comput. Neurosci. 7, 185 (2013)

    Google Scholar 

  25. D. Endres, A. Christensen, L. Omlor, M.A. Giese, Emulating human observers with Bayesian binning: segmentation of action streams. ACM Trans. Appl. Percept. (TAP), 8(3), 16, 1–12 (2011)

    Google Scholar 

  26. D. Endres, A. Christensen, L. Omlor, M.A. Giese, Segmentation of action streams: human observers vs. Bayesian binning, in KI 2011, LNAI 7006, ed. by S. Edelkamp, J. Bach (Springer, Berlin, 2011), pp. 75–86

    Google Scholar 

  27. D. Endres, P. Földiák, Bayesian bin distribution inference and mutual information. IEEE Trans. Inf. Theory 51(11), 3766–3779 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  28. D. Endres, M. Oram, Feature extraction from spike trains with Bayesian binning: latency is where the signal starts. J. Comput. Neurosci. 29, 149–169 (2010)

    Article  MathSciNet  Google Scholar 

  29. D. Endres, M. Oram, J. Schindelin, P. Földiák, Bayesian binning beats approximate alternatives: estimating peri-stimulus time histograms, in Advances in Neural Information Processing Systems 20, ed. by J. Platt, D. Koller, Y. Singer, S. Roweis (MIT Press, Cambridge, MA, 2008)

    Google Scholar 

  30. P. Fearnhead, Exact and efficient Bayesian inference for multiple changepoint problems. Stat. Comput. 16(2), 203–213 (2006)

    Article  MathSciNet  Google Scholar 

  31. T. Flash, B. Hochner, Motor primitives in vertebrates and invertebrates. Curr. Opin. Neurobiol. 15(6), 660–666 (2005)

    Article  Google Scholar 

  32. T. Flash, N. Hogan, The coordination of arm movements: an experimentally confirmed mathematical model. J. Neurosci. 5, 1688–1703 (1985)

    Google Scholar 

  33. E. Fuchs, T. Gruber, J. Nitschke, B. Sick, Online segmentation of time series based on polynomial least-squares approximations. IEEE Trans. Pattern Anal. Mach. Intell. 32(12), 2232–2245 (2010)

    Article  Google Scholar 

  34. A. Gensler, T. Gruber, B. Sick, Blazing fast time series segmentation based on update techniques for polynomial approximations, in 13th IEEE International Conference on Data Mining Workshops, ICDM Workshops, TX, USA, 7–10 December 2013, pp. 1002–1011

    Google Scholar 

  35. M. Giese, A. Mukovskiy, A. Park, L. Omlor, J. Slotine, Real-time synthesis of body movements based on learned primitives, in Statistical and Geometrical Approaches to Visual Motion Analysis, ed. by D. Cremers, B. Rosenhahn, A.L. Yuille (Springer, Heidelberg, 2009), pp. 107–127

    Chapter  Google Scholar 

  36. R.D. Green, Spatial and temporal segmentation of continuous human motion from monocular video images, in Proceedings of Image and Vision Computing, pp. 163–169, New Zealand (2003)

    Google Scholar 

  37. C.M. Harris, D.M. Wolpert, Signal-dependent noise determines motor planning. Nature 394(6695), 780–784 (1998)

    Article  Google Scholar 

  38. C.B. Hart, S. Giszter, Distinguishing synchronous and time varying synergies using point process interval statistics: Motor primitives in frog and rat. Front. Comput. Neurosci. 7, 52 (2013)

    Article  Google Scholar 

  39. P.A. Højen-Sørensen, O. Winther, L.K. Hansen, Mean field approaches to independent component analysis. Neural Comput. 14(4), 889–918 (2002)

    Google Scholar 

  40. M. Hutter, Exact Bayesian regression of piecewise constant functions. J. Bayesian Anal. 2(4), 635–664 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  41. W. Ilg, G. Bakir, J. Mezger, M. Giese, On the representation, learning and transfer of spatio-temporal movement characteristics. Int. J. Hum. Rob. 1(4), 613–636 (2004)

    Article  Google Scholar 

  42. Y.P. Ivanenko, G. Cappellini, N. Dominici, R.E. Poppele, F. Lacquaniti, Coordination of locomotion with voluntary movements in humans. J. Neurosci. 25(31), 7238–7253 (2005)

    Article  Google Scholar 

  43. Y.P. Ivanenko, R.E. Poppele, F. Lacquaniti, Five basic muscle activation patterns account for muscle activity during human locomotion. J. Physiol. 556(Pt 1), 267–282 (2004)

    Article  Google Scholar 

  44. T.R. Kaminski, The coupling between upper and lower extremity synergies during whole body reaching. Gait Posture 26(2), 256–262 (2007)

    Article  Google Scholar 

  45. E. Keogh, S. Chu, D. Hart, M. Pazzani, An online algorithm for segmenting time series, in Proceedings IEEE International Conference on Data Mining, 2001, ICDM 2001, pp. 289–296 (2001)

    Google Scholar 

  46. L. Kovar, M. Gleicher, F. Pighin, Motion graphs. ACM Trans. Graph. 21, 473–482 (2002)

    Article  Google Scholar 

  47. F. Kschischang, B. Frey, H.-A. Loeliger, Factor graphs and the sum-product algorithm. IEEE Trans. Inf. Theory 47(2), 498–519 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  48. D. Kulic, W. Takano, Y. Nakamura, Online segmentation and clustering from continuous observation of whole body motions. IEEE Trans. Rob. 25(5), 1158–1166 (2009)

    Article  Google Scholar 

  49. F. Lacquaniti, C. Terzuolo, P. Viviani, The law relating kinematic and figural aspects of drawing movements. Acta Psychol. 54, 115–130 (1983)

    Article  Google Scholar 

  50. D.D. Lee, H.S. Seung, Algorithms for non-negative matrix factorization, in In NIPS, pp. 556–562. MIT Press, Cambridge (2000)

    Google Scholar 

  51. D. Lemire, A better alternative to piecewise linear time series segmentation. CoRR arXiv:abs/cs/0605103 (2006)

  52. Y. Li, T. Adal, V. Calhoun, Estimating the number of independent components for functional magnetic resonance imaging data. Hum. Brain Mapp. 28(11), 1251–1266 (2007)

    Article  Google Scholar 

  53. J. Liu, Y.C. Nakamura, N.S. Pollard, Annotating everyday grasps in action, in Dance Notations and Robot Motion, chapter ZZ, pp. XX–YY. (Springer, Berlin, 2015)

    Google Scholar 

  54. T. Minka, Automatic choice of dimensionality for PCA. Technical report, M.I.T. Media Laboratory Perceptual Computing Section (2000)

    Google Scholar 

  55. T. Minka, J. Winn, Gates: a graphical notation for mixture models, in Proceedings of NIPS (2008)

    Google Scholar 

  56. A. Mukovskiy, J.J.E. Slotine, M.A. Giese, Dynamically stable control of articulated crowds. J. Comput. Sci. 4(4), 304–310 (2013)

    Google Scholar 

  57. L. Omlor, New methods for anechoic demixing with application to shift invariant feature extraction. Ph.D. in informatics, Universität Ulm. Fakultät für Ingenieurwissenschaften und Informatik, 2010. urn:nbn:de:bsz:289-vts-72431

    Google Scholar 

  58. L. Omlor, M. Giese, Blind source separation for over-determined delayed mixtures, in Advances in Neural Information Processing Systems 19, ed. by B. Schölkopf, J. Platt, T. Hoffman (MIT Press, Cambridge, MA, 2007), pp. 1049–1056

    Google Scholar 

  59. L. Omlor, M.A. Giese, Extraction of spatio-temporal primitives of emotional body expressions. Neurocomputing, 70(10–12), 1938–1942 (2007) (reviewed)

    Google Scholar 

  60. L. Omlor, M.A. Giese, Anechoic blind source separation using wigner marginals. J. Mach. Learn. Res. 12, 1111–1148 (2011)

    MathSciNet  MATH  Google Scholar 

  61. F. Polyakov, E. Stark, R. Drori, M. Abeles, T. Flash, Parabolic movement primitives and cortical states: merging optimality with geometric invariance. Biol. Cybern. 100(2), 159–184 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  62. L. Rabiner, A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE 77(2), 257–286 (1989)

    Article  Google Scholar 

  63. C.L. Roether, L. Omlor, A. Christensen, M.A. Giese, Critical features for the perception of emotion from gait. J. Vision 9(6), 1–32 (2009)

    Article  Google Scholar 

  64. C. Rose, B. Bodenheimer, M.F. Cohen, Verbs and adverbs: Multidimensional motion interpolation using radial basis functions. IEEE Comput. Graph. Appl. 18, 32–40 (1998)

    Article  Google Scholar 

  65. A. Safonova, J.K. Hodgins, Construction and optimal search of interpolated motion graphs, in ACM SIGGRAPH 2007 Papers, SIGGRAPH’07, New York, NY, USA. (ACM, New York, 2007)

    Google Scholar 

  66. A. Safonova, J.K. Hodgins, N.S. Pollard, Synthesizing physically realistic human motion in low-dimensional, behavior-specific spaces. ACM Trans. Graph. 23, 514–521 (2004)

    Article  Google Scholar 

  67. M. Santello, M. Flanders, J.F. Soechting, Postural hand synergies for tool use. J. Neurosci. 18(23), 10105–10115 (1998)

    Google Scholar 

  68. J. Segouat, A. Braffort, Toward the study of sign language coarticulation: methodology proposal, in Proceedings of the 2009 Second International Conferences on Advances in Computer-Human Interactions, ACHI ’09, pp. 369–374, Washington, DC, USA, 2009. IEEE Computer Society

    Google Scholar 

  69. T.F. Shipley, M.J. Maguire, J. Brumberg, Segmentation of event paths. J. Vision, 4(8) (2004)

    Google Scholar 

  70. A. Swindlehurst, Time delay and spatial signature estimation using known asynchronous signals. IEEE Trans. Signal Process. 46, 449–462 (1997)

    Article  Google Scholar 

  71. J.S. Thomas, D.M. Corcos, Z. Hasan, Kinematic and kinetic constraints on arm, trunk, and leg segments in target-reaching movements. J. Neurophysiol. 93(1), 352–364 (2005)

    Article  Google Scholar 

  72. L.H. Ting, J.M. Macpherson, A limited set of muscle synergies for force control during a postural task. J. Neurophysiol. 93(1), 609–613 (2005)

    Article  Google Scholar 

  73. G. Torres-Oviedo, J.M. Macpherson, L.H. Ting, Muscle synergy organization is robust across a variety of postural perturbations. J. Neurophysiol. 96(3), 1530–1546 (2006)

    Article  Google Scholar 

  74. S. Tu, L. Xu, An investigation of several typical model selection criteria for detecting the number of signals. Front. Electr. Electron. Eng. China 6, 245–255 (2011)

    Article  Google Scholar 

  75. A. Ude, C. Atkeson, M. Riley, Programming full-body movements for humanoid robots by observation. Robot. Auton. Syst. 47, 93–108 (2004)

    Article  Google Scholar 

  76. A. Ude, M. Riley, B. Nemec, T. Asfour, G. Cheng, Synthesizing goal-directed actions from a library of example movements, in IEEE/RAS International Conference on Humanoid Robots (Humanoids) (2007)

    Google Scholar 

  77. M. Wächter, S. Schulz, T. Asfour, E. Aksoy, F. Wörgötter, R. Dillmann, Action sequence reproduction based on automatic segmentation and object-action complexes, in IEEE/RAS International Conference on Humanoid Robots (Humanoids), pp. 189–195 (2013)

    Google Scholar 

  78. L. Xu, Bayesian ying yang learning. Scholarpedia 2(3), 1809 (2007)

    Google Scholar 

  79. Ö. Yilmaz, S. Rickard, Blind separation of speech mixtures via time-frequency masking. IEEE Trans. Signal Process. 52, 1830–1847 (2004)

    Article  MathSciNet  Google Scholar 

  80. J.M. Zacks, T.S. Braver, M.A. Sheridan, D.I. Donaldson, A.Z. Snyder, J.M. Ollinger, R.L. Buckner, M.E. Raichle, Human brain activity time-locked to perceptual event boundaries. Nat. Neurosci. 4(6), 651–655 (2001)

    Article  Google Scholar 

  81. J.M. Zacks, S. Kumar, R.A. Abrams, R. Mehta, Using movement and intentions to understand human activity. Cognition 112(2), 201–216 (2009)

    Article  Google Scholar 

Download references

Acknowledgments

The research leading to these results has received funding from the European Union under grant agreements Koroibot FP7-ICT-2013-10/611909, AMARSi- EC FP7-ICT-248311; FP7-PEOPLE-2011-ITN (Marie Curie): ABC PITN-GA-011-290011, 604102 (HBP), CogIMon H2020 ICT-23-2014/644727, and form the DFG under grants GI 305/4-1, DFG GZ: KA 1258/15-1, and from BMBF grant FKZ: 01GQ1002A. DE has received support from the DFG under grant IRTG-GRK 1901 “The Brain in Action”.

Author information

Authors and Affiliations

  1. Theoretical Neuroscience Group, Department of Psychology, Philipps-University Marburg, Marburg, Germany

    Dominik Endres

  2. Section Computational Sensomotorics, Department Cognitive Neurology, University Clinic Tübingen, Tübingen, Germany

    Dominik Endres, Enrico Chiovetto & Martin A. Giese

Authors
  1. Dominik Endres
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Enrico Chiovetto
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Martin A. Giese
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dominik Endres .

Editor information

Editors and Affiliations

  1. LAAS-CNRS, Toulouse Cedex 4, France

    Jean-Paul Laumond

  2. LAAS-CNRS, Toulouse Cedex 4, France

    Naoko Abe

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Endres, D., Chiovetto, E., Giese, M.A. (2016). Bayesian Approaches for Learning of Primitive-Based Compact Representations of Complex Human Activities. In: Laumond, JP., Abe, N. (eds) Dance Notations and Robot Motion. Springer Tracts in Advanced Robotics, vol 111. Springer, Cham. https://doi.org/10.1007/978-3-319-25739-6_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-25739-6_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-25737-2

  • Online ISBN: 978-3-319-25739-6

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation