Skip to main content
SpringerLink
Log in

Automated human motion segmentation via motion regularities

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Analysis and reuse of human motion capture (mocap) data play an important role in animation, games and medical rehabilitation. In various mocap-based animation techniques, motion segmentation is regarded as one of the fundamental functions. Many proposed segmentation methods utilize little or no prior knowledge. However, human motion has its own regularities, so reasonable prior assumptions on these regularities will lead to better performance. In this paper, we focus on the learning of intrinsic regularities of mocap data based on a small set of training data which only contain daily-life motions. By utilizing these learnt motion regularities, we can successfully segment long motion sequences containing motion types that not even include in the training data. First, by assuming that most types of motions can be composed of a small number of typical poses, the motion vocabulary (mo-vocabulary) can be obtained using key pose extraction and clustering analysis, which are regarded as the low-level motion regularity. By replacing each frame with the most similar pose in the mo-vocabulary, mocap data can be transformed into text-like documents. Second, we use latent Dirichlet allocation to capture the patterns of pose combinations that frequently occur in human motions, namely the motion topics (mo-topics), which are regarded as the high-level motion regularities. By representing the target motion as the distribution over the learnt mo-topics, the segmentation task can be naturally turned into a problem of detecting notable changes of this distribution. Finally, we propose local semantic coherence curve to segment motion sequences. Since mo-topics are semantically meaningful and significantly increase the abstraction-level of motion representation, logically correct results can be obtained. The experiments demonstrate that the proposed approach outperforms the available methods on CMU and Bonn mocap database.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Arikan, O., Forsyth, D.A., O’Brien, J.F.: Motion synthesis from annotations. ACM Trans. Graph. 22(3), 402–408 (2003)

    Article  MATH  Google Scholar 

  2. Barbic, J., Safonova, A., Pan, J.Y., Faloutsos, C., Hodgins, J.K., Pollard, N.S.: Segmenting motion capture data into distinct behaviors. In: Proceedings of Graphics Interface 2004 (GI’ 04), pp. 185–194. Canadian Human-Computer Communications Society School of Computer Science, University of Waterloo, Waterloo, ON, Canada (2004)

  3. Van Basten, B.J.H., Egges, A.: Evaluating distance metrics for animation blending. In: Proceedings of the 4th International Conference on Foundations of Digital Games, pp. 199–206. ACM, New York (2009)

  4. Beaudoin, P., Coros, S., van de Panne, M., Poulin, P.: Motion-motif graphs. In: Proceedings of the 2008 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA’ 08), pp. 117–126. Eurographics Association Aire-la-Ville, Switzerland, Switzerland (2008)

  5. Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)

    MATH  Google Scholar 

  6. Deng, Z., Gu, Q., Li, Q.: Perceptually consistent example-based human motion retrieval. In: Proceedings of the 2009 Symposium on Interactive 3D Graphics and Games (I3D’ 09), pp. 191–198. ACM, New York (2009)

  7. Fod, A., Matarić, M.J., Jenkins, O.C.: Automated derivation of primitives for movement classification. Autonom. Robots 12(1), 39–54 (2002)

    Article  MATH  Google Scholar 

  8. Griffiths, T.L., Steyvers, M.: Finding scientific topics. Proc. Natl. Acad. Sci. 101, 5228–5235 (2004)

    Article  Google Scholar 

  9. Heck, R., Gleicher, M.: Parametric motion graphs. In: Proceedings of the 2007 Symposium on Interactive 3D Graphics and Games, pp. 129–136. ACM, New York (2007)

  10. Heidel, A., Chang, H.A., Lee, L.S.: Language model adaptation using latent dirichlet allocation and an efficient topic inference algorithm. In: Proceedings of European Conference on Speech Communication and Technology, pp. 2361–2364 (2007)

  11. Hu, D.J., Saul, L.K.: A probabilistic topic model for unsupervised learning of musical key-profiles. In: International Society for Music Information Retrieval (ISMIR’ 09), pp. 441–446 (2009)

  12. Kovar, L., Gleicher, M., Pighin, F.: Motion graphs. ACM Trans. Graph. 21(3), 473–482 (2002)

    Article  Google Scholar 

  13. Kulic, D., Venture, G., Nakamura, Y.: Detecting changes in motion characteristics during sports training. In: Engineering in Medicine and Biology Society, 2009. EMBC 2009. Annual International Conference of the IEEE, pp. 4011–4014. IEEE (2009)

  14. Lan, R., Sun, H., Zhu, M.: Text-like motion representation for human motion retrieval. In: Intelligent Science and Intelligent Data Engineering, pp. 72–81. Springer, Berlin (2013)

  15. Li, C., Kulkarni, P.R., Prabhakaran, B.: Segmentation and recognition of motion capture data stream by classification. Multimed. Tools Appl. 35(1), 55–70 (2007)

    Article  Google Scholar 

  16. Lv, F., Nevatia, R.: Recognition and segmentation of 3-D human action using hmm and multi-class adaboost. In: ECCV’ 06, pp. 359–372 (2006)

  17. Min, J., Chen, Y.L., Chai, J.: Interactive generation of human animation with deformable motion models. ACM Trans. Graph. 29(1), No.9 (2009)

    Google Scholar 

  18. Minka, T., Lafferty, J.: Expectation-propagation for the generative aspect model. In: Proceedings of the Eighteenth conference on Uncertainty in artificial intelligence (UAI’ 02), pp. 352–359. Morgan Kaufmann Publishers Inc., San Francisco (2002)

  19. Misra, H., Cappe, O., Yvon, F.: Using LDA to detect semantically incoherent documents. In: Proceedings of the Twelfth Conference on Computational Natural Language Learning (CoNLL’ 08), pp. 41–48. Association for Computational Linguistics, Stroudsburg, PA, USA (2008)

  20. Misra, H., Yvon, F., Jose, J.M., Cappe, O.: Text segmentation via topic modeling: An analytical study. In: Proceedings of the 18th ACM Conference on Information and Knowledge Management (CIKM’ 09), pp. 1553–1556. ACM, New York (2009)

  21. Müller, M., Röder, T., Clausen, M., Eberhardt, B., Krüger, B., Weber, A.: Documentation mocap database hdm05. Tech. Rep. CG-2007-2, Universität Bonn (2007)

  22. Peng, S.J.: Motion segmentation using central distance features and low-pass filter. In: Proceedings of the 2010 International Conference on Computational Intelligence and Security (CIS’ 10), pp. 223–226. IEEE Computer Society, Washington (2010)

  23. Sivic, J., Russell, B.C., Efros, A.A., Zisserman, A., Freeman, W.T.: Discovering object categories in image collections. Tech. rep, MIT (2005)

  24. Souvenir, R., Pless, R.: Manifold clustering. In: Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV’ 05) (2005)

  25. Wang, T.S., Shum, H.Y., Xu, Y.Q., Zheng, N.N.: Unsupervised analysis of human gestures. In: Advances in Multimedia Information Processing (PCM’ 2001), pp. 174–181 (2001)

  26. Wu, S., Wang, Z., Xia, S.: Indexing and retrieval of human motion data by a hierarchical tree. In: Proceedings of the 16th ACM Symposium on Virtual Reality Software and Technology (VRST ‘09), pp. 207–214. ACM, New York (2009)

  27. Xiao, J., Zhuang, Y., Wu, F., Guo, T., Liang, Z.: A group of novel approaches and a toolkit for motion capture data reusing. Multimed. Tools Appl. 47(3), 379–408 (2010)

    Article  Google Scholar 

  28. Zhao, L., Sukthankar, G.: An active learning approach for segmenting human activity datasets. In: Proceedings of the 17th ACM International Conference on Multimedia (MM’ 09), pp. 765–768. ACM, New York (2009)

  29. Zhou, F., De la Torre, F.D., Hodgins, J.K.: Hierarchical aligned cluster analysis for temporal clustering of human motion. IEEE Trans. Pattern Recogn. Mach. Intell. 35(3), 582–596 (2013)

    Article  Google Scholar 

  30. Zhu, M., Sun, H., Lan, R., Li, B.: Human motion retrieval using topic model. Comput. Anim. Virtual Worlds 23(5), 469–476 (2012)

    Article  Google Scholar 

Download references

Acknowledgments

This research was supported by the Programme of Introducing Talents of Discipline to Universities (Grant No. B13022) and NUST Research Funding (Grant No. 2011YBXM79). We thank Zexuan Ji, Mingyang Zhu and Liang Zhou for their help and valuable suggestions. The mocap data used in this work was obtained from CMU Graphics Lab (mocap.cs.cmu.edu) and HDM05, Universität Bonn [21].

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Nanjing University of Science and Technology, Nanjing, 210094, Jiangsu, China

    Rongyi Lan & Huaijiang Sun

Authors
  1. Rongyi Lan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huaijiang Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huaijiang Sun.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lan, R., Sun, H. Automated human motion segmentation via motion regularities. Vis Comput 31, 35–53 (2015). https://doi.org/10.1007/s00371-013-0902-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-013-0902-5

Keywords

Search

Navigation