Skip to main content
SpringerLink
Log in

Action Capture: A VR-Based Method for Character Animation

  • Chapter
  • First Online:
Virtual Realities

  • 1835 Accesses

Abstract

This contribution describes a Virtual Reality (VR) based method for character animation that extends conventional motion capture by not only tracking an actor’s movements but also his or her interactions with the objects of a virtual environment. Rather than merely replaying the actor’s movements, the idea is that virtual characters learn to imitate the actor’s goal-directed behavior while interacting with the virtual scene. Following Arbib’s equation action = movement + goal we call this approach Action Capture. For this, the VR user’s body movements are analyzed and transformed into a multi-layered action representation. Behavioral animation techniques are then applied to synthesize animations which closely resemble the demonstrated action sequences. As an advantage, captured actions can often be naturally applied to virtual characters of different sizes and body proportions, thus avoiding retargeting problems of motion capture.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    According to Thorndike [Tho98] imitation is: “from an act witnessed learn to do an act.”

  2. 2.

    According to their definition in [KT02] as: “all parts, movements and descriptions of an object that have some important role when interacting with an actor.”

References

  1. M. A. Arbib. The mirror system, imitation, and the evolution of language. In Dautenhahn and Nehaniv Imitation in Animals and Artifacts. MIT, Cambridge, 2002.

    Google Scholar 

  2. H. B. Amor, G. Heumer, B. Jung, and A. Vitzthum. Grasp synthesis from low-dimensional probabilistic grasp models. Computer Animation and Virtual Worlds, 19(3–4):445–454, 2008.

    Article  Google Scholar 

  3. H. B. Amor, M. Weber, G. Heumer, and B. Jung. Coordinate system transformations for imitation of goal-directed trajectories in virtual humans. In Virtual Environments 2007. IPT EGVE 2007. 13th Eurographics Symposium on Virtual Environments. Short Papers and Posters, 2007.

    Google Scholar 

  4. N. I. Badler, R. Bindiganavale, J. Allbeck, W. Schuler, L. Zhao, and M. Palmer. Parameterized action representation for virtual human agents. In Embodied conversational agents, pages 256–284. MIT Press, Cambridge, 2000.

    Google Scholar 

  5. P. Bakker and Y. Kuniyoshi. Robot see, Robot do: An Overview of Robot Imitation. In AISB96 Workshop: Learning in Robots and Animals, pages 3–11, 1996.

    Google Scholar 

  6. N. I. Badler, C. B. Phillips, and B. L. Webber. Simulating Humans: Computer Graphics and Animation and Control. Oxford University Press, New York, 1993.

    MATH  Google Scholar 

  7. A. Billard and R. Siegwart, editors. Special Issue on Robot Learning from Demonstration, volume 47 of Robotics and Autonomous Systems, 2004.

    Google Scholar 

  8. N. I. Badler, B. L. Webber, J. Kalita, and J. Esakov. Animation from instructions. In Making them move: mechanics, control, and animation of articulated figures, pages 51–93. Morgan Kaufmann, San Francisco, 1991.

    Google Scholar 

  9. A. P. Dempster, N. M. Laird, and D. B. Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society. Series B (Methodological), 39(1):1–38, 1977.

    Google Scholar 

  10. K. Dautenhahn and C. Nehaniv, editors. Imitation in Animals and Artifacts. MIT, Cambridge, 2002.

    Google Scholar 

  11. F. Gommlich, G. Heumer, B. Jung, and A. Vitzthum. Simulation of Articulated Standard Control Actuators in Dynamic Virtual Environments. In Proceedings of IEEE Virtual Reality 2009, pages 269–270. IEEE, 2009.

    Google Scholar 

  12. M. Gleicher. Retargetting Motion to New Characters. In SIGGRAPH’98 Conference Proceedings, Computer Graphics Annual Conference Series, pages 33–42. ACM, 1998.

    Google Scholar 

  13. G. Heumer, H. B. Amor, and B. Jung. Grasp recognition for uncalibrated data gloves: A machine learning approach. Presence: Teleoperators & Virtual Environments, 17(2):121–142, 2008.

    Article  Google Scholar 

  14. G. Heumer, H. B. Amor, M. Weber, and B. Jung. Grasp Recognition with Uncalibrated Data Gloves – A Comparison Of Classification Methods. In Proceedings of IEEE Virtual Reality Conference, VR ’07, pages 19–26, March 2007.

    Google Scholar 

  15. H. Heuer and J. Sangals. Task-dependent mixtures of coordinate systems in visuomotor transformations. Experimental Brain Research, 119(2):224-236, 1998.

    Article  Google Scholar 

  16. W. L. Johnson and J. Rickel. Steve: an animated pedagogical agent for procedural training in virtual environments. SIGART Bulletin, 8(1–4):16–21, 1997.

    Article  Google Scholar 

  17. A. Kendon. Gesture: Visible Action as Utterance. Cambridge University Press, Cambridge, 2004.

    Google Scholar 

  18. J. Kuffner and J. Latombe. Interactive Manipulation Planning for Animated Characters. In Proceedings of Pacific Graphics, 2000.

    Google Scholar 

  19. M. Kallmann and D. Thalmann. Direct 3D Interaction with Smart Objects. In Proceedings ACM VRST 99, London, 1999.

    Google Scholar 

  20. M. Kallmann and D. Thalmann. Modeling behaviors of interactive objects for real-time virtual environments. Journal of Visual Languages and Computing, 13(2):177–195, 2002.

    Article  Google Scholar 

  21. A. N. Meltzoff. The human infant as imitative generalist: a 20-year progress report on infant imitation with implications for comparative psychology. In Social Learning in Animals: The Roots of Culture, pages 347–370, 1996.

    Google Scholar 

  22. A. Moon and M. Farsi. Grasp Quality Measures in the Control of Dextrous Robot Hands. Physical Modelling as a Basis for Control (Digest No: 1996/042), IEE Colloquium on, pages 6/1–6/4, 1996.

    Google Scholar 

  23. M. Möhring and B. Fröhlich. Enabling Functional Validation of Virtual Cars through Natural Interaction Metaphors. In Proceedings of IEEE Virtual Reality Conference, VR 2010, 2010.

    Google Scholar 

  24. C. L. MacKenzie and T. Iberall. The Grasping Hand. Elsevier-North Holland, 1994.

    Google Scholar 

  25. N. Magnenat-Thalmann and D. Thalmann, editors. Handbook of Virtual Humans. Wiley, 2004.

    Google Scholar 

  26. C. Nehaniv and K. Dautenhahn. The Correspondence Problem. In Dautenhahn and Nehaniv [DN02], pages 41–61.

    Google Scholar 

  27. C. Nehaniv and K. Dautenhahn, editors. Imitation and Social Learning in Robots, Humans and Animals: Behavioural, Social and Communicative Dimensions. Cambridge University Press, Cambridge, 2007.

    Google Scholar 

  28. N. S. Pollard and V. B. Zordan. Physically based Grasping Control from Example. In SCA ’05: Proceedings of the 2005 ACM SIGGRAPH/Eurographics symposium on Computer animation, pages 311–318. ACM, New York, 2005.

    Google Scholar 

  29. S. T. Roweis and L. K. Saul. Nonlinear dimensionality reduction by locally linear embedding. Science, 290(5500):2323–2326, 2000.

    Article  Google Scholar 

  30. R. Rao, A. P. Shon, and A. N. Meltzoff. A Bayesian model of imitation in infants and robots. In Imitation and Social Learning in Robots, Humans and Animals: Behavioural, Social and Communicative Dimensions, Cambridge University Press, Cambridge, 2004.

    Google Scholar 

  31. G. Schlesinger. Der Mechanische Aufbau der Künstlichen Glieder. In M. Borchardt et al., editors, Ersatzglieder und Arbeitshilfen für Kriegsbeschädigte und Unfallverletzte, pages 321–661. Springer, Berlin, 1919.

    Google Scholar 

  32. J. B. Tenenbaum, V. de Silva, and J. C. Langford. A global geometric framework for nonlinear dimensionality reduction. Science, 290(5500):2319–2323, 2000.

    Article  Google Scholar 

  33. E. L. Thorndike. Animal intelligence: an experimental study of the associative processes in animals. Psychological Review Monographs, 8, 1898.

    Google Scholar 

  34. B. Tomlinson. From linear to interactive animation: how autonomous characters change the process and product of animating. ACM Computers in Entertainment, 3(1), 2005.

    Google Scholar 

  35. A. Vitzthum, H. B. Amor, G. Heumer, and B. Jung. Action description for animation of virtual characters. In 6. Workshop Virtuelle und Erweiterte Realität. GI-Fachgruppe VR/AR, 2009.

    Google Scholar 

  36. M. Weber, G. Heumer, H. B. Amor, and B. Jung. An animation system for imitation of object grasping in virtual reality. In ICAT, pages 65–76, 2006.

    Google Scholar 

  37. World Wide Web Consortium. Synchronized Multimedia Integration Language (SMIL 3.0), 2008.

    Google Scholar 

  38. K. Yamane, J. J. Kuffner, and J. K. Hodgins. Synthesizing animations of human manipulation tasks. ACM Trans. Graph., 23(3):532–539, 2004.

    Article  Google Scholar 

Download references

Acknowledgements

The research described in this contribution was supported by the DFG (Deutsche Forschungsgemeinschaft) in the Virtual Workers project.

Author information

Authors and Affiliations

  1. VR and Multimedia Group, Institute of Informatics, TU Bergakademie Freiberg, Freiberg, Germany

    Bernhard Jung, Heni Ben Amor, Guido Heumer & Arnd Vitzthum

Authors
  1. Bernhard Jung
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Heni Ben Amor
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guido Heumer
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Arnd Vitzthum
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bernhard Jung .

Editor information

Editors and Affiliations

  1. Fak. Informatik, Professur für Graphische, TU Chemnitz, Straße der Nationen 62, Chemnitz, 09107, Germany

    Guido Brunnett

  2. ZIRST, INRIA Rhône-Alpes, avenue de l'Europe 655, St.Ismier, 38334, France

    Sabine Coquillart

  3. Dept. Computer Science, University of North Carolina, Sitterson Hall, Chapel Hill, 27599-3175, North Carolina, USA

    Greg Welch

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag/Wien

About this chapter

Cite this chapter

Jung, B., Amor, H.B., Heumer, G., Vitzthum, A. (2011). Action Capture: A VR-Based Method for Character Animation. In: Brunnett, G., Coquillart, S., Welch, G. (eds) Virtual Realities. Springer, Vienna. https://doi.org/10.1007/978-3-211-99178-7_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-211-99178-7_6

  • Published:

  • Publisher Name: Springer, Vienna

  • Print ISBN: 978-3-211-99177-0

  • Online ISBN: 978-3-211-99178-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation