Skip to main content
Springer Nature Link
Log in

Challenges for the Animation of Expressive Virtual Characters: The Standpoint of Sign Language and Theatrical Gestures

  • Chapter
  • First Online:
Dance Notations and Robot Motion

Part of the book series: Springer Tracts in Advanced Robotics ((STAR,volume 111))

  • 1630 Accesses

Abstract

Designing and controlling virtual characters endowed with expressive gestures requires the modeling of multiple processes, involving high-level abstract representations to low-level sensorimotor models. An expressive gesture is here defined as a meaningful bodily motion which intrinsically associates sense, style, and expressiveness. The main challenges rely both on the capability to produce a large spectrum of parametrized actions executed with some variability in various situations, and on the biological plausibility of the motion of the virtual characters. The goals of the paper are twofold. First we review the different formalisms used to describe expressive gestures, from notations to computational languages. Secondly we identify and discuss remaining challenges in the generation of expressive virtual characters. The different models and formalisms are illustrated more particularly for theatrical and sign language gestures.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. M. Thiebaux, S. Marsella, A.N. Marshall, M. Kallmann, Smartbody: behavior realization for embodied conversational agents, in Proceedings of AAMAS’08—Volume 1, Richland, SC (2008) (pp. 151–158)

    Google Scholar 

  2. J. Bates et al., The role of emotion in believable agents. Commun. ACM 37(7), 122–125 (1994)

    Article  Google Scholar 

  3. Y. Jung, A. Kuijper, M. Kipp, J. Miksatko, J. Gratch, D, Thalmann, Believable Virtual Characters in Human-Computer Dialogs. In EUROGRAPHICS, Llandudno, United Kingdom (2011)

    Google Scholar 

  4. M. Neff, Lessons from the arts: what the performing arts literature can teach us about creating expressive character movement, in Nonverbal Communication in Virtual Worlds: Understanding and Designing Expressive Characters (ETC Press, 2014)

    Google Scholar 

  5. K. Elam, The Semiotics of Theatre and Drama (Routledge, 2002)

    Google Scholar 

  6. S.C Marsella, S.M. Carnicke, J. Gratch, A. Okhmatovskaia, A. Rizzo, An exploration of delsartes structural acting system, in Intelligent Virtual Agents (Springer, 2006), pp. 80–92

    Google Scholar 

  7. S. Gibet, T. Lebourque, P.F. Marteau, High level specification and animation of communicative gestures. J. Visual Lang. Comput. 12, 657–687 (2001)

    Article  Google Scholar 

  8. A. Bogart, T. Landau, The Viewpoints Book: A Practical Guide To Viewpoints and Composition. Theatre Communications Group (2005)

    Google Scholar 

  9. R.E. Johnson, S.K. Liddell, Toward a phonetic representation of signs: sequentiality and contrast. Sign Lang. Stud. 11(2), 241–274 (2011)

    Article  Google Scholar 

  10. K. Duarte, Motion capture and avatars as portals for analyzing the linguistic structure of signed languages, in PhD thesis, Université de Bretagne Sud (2012)

    Google Scholar 

  11. A. Kendon, Gesticulation and speech two aspects of the process of utterance, in The Relation Between Verbal and Nonverbal Communication (1980), pp. 207–227

    Google Scholar 

  12. D. McNeill, Hand and Mind—What Gestures Reveal about Thought (The University of Chicago Press, Chicago, IL, 1992)

    Google Scholar 

  13. S. Kita, I. van Gijn, H. van der Hulst, Movement phase in signs and co-speech gestures, and their transcriptions by human coders, Proceedings of the International Gesture Workshop on Gesture and Sign Language in Human-Computer Interaction, vol. 1371, Lecture Notes in Computer Science (Springer, London, 1997), pp. 23–35

    Chapter  Google Scholar 

  14. C. Awad, N. Courty, K. Duarte, T. Le Naour, S. Gibet, A combined semantic and motion capture database for real-time sign language synthesis, in IVA (2009), pp. 432–438

    Google Scholar 

  15. R. Laban. The Mastery of Movement (Plays Inc. 1971)

    Google Scholar 

  16. V. Maletik, Body, Space, Expression: The Development of Rudolf Laban’s Movement and Dance Concepts (Mouton de Gruyte, New York, 1987)

    Book  Google Scholar 

  17. D.M. Chi, M. Costa, L. Zhao, N.I. Badler, The EMOTE model for effort and shape, in SIGGRAPH (2000), pp. 173–182

    Google Scholar 

  18. L. Zhao, N.I. Badler, Acquiring and validating motion qualities from live limb gestures. Graph. Models 67(1), 1–16 (2005)

    Article  Google Scholar 

  19. L. Torresani, P. Hackney, C. Bregler, Learning motion style synthesis from perceptual observations, in Advances in Neural Information Processing Systems (2006), pp. 1393–1400

    Google Scholar 

  20. D. Bouchard, N.I. Badler, Semantic segmentation of motion capture using laban movement analysis, in Intelligent Virtual Agents (IVA 2007) (2007), pp. 37–44

    Google Scholar 

  21. M. Karg, A. Samadani, R. Gorbet, K. Kühnlenz, J. Hoey, D. Kulic, Body movements for affective expression: a survey of automatic recognition and generation. T. Affect. Comput. 4(4), 341–359 (2013)

    Article  Google Scholar 

  22. A. Kleinsmith, N. Bianchi-Berthouze, Affective body expression perception and recognition: a survey. T. Affect. Comput. 4(1), 15–33 (2013)

    Article  Google Scholar 

  23. N. Eshkol, J. Harries, Ewmn: Eshkol-wachman movement notation (2011). Accessed 10 Mar 2015 [Online]

    Google Scholar 

  24. J.G. Harries, Symmetry in the movements of T’ai Chi Chuan. Comput. Math Appl. 17(46), 827–835 (1989)

    Article  MathSciNet  Google Scholar 

  25. N. Eshkol, The Hand Book: The Detailed Notation of Hand and Finger Movements and Forms. [Tel Aviv]: Movement Notation Society, 1971

    Google Scholar 

  26. O. Teitelbaum, T. Benton, P.K. Shah, A. Prince, J.L. Kelly, P. Teitelbaum, Eshkol-wachman movement notation in diagnosis: the early detection of Asperger’s syndrome. Proc. Natl. Acad. Sci. USA 101(32), 11909–11914 (2004)

    Article  Google Scholar 

  27. J. Tanenbaum, M. Seif El-Nasr, M. Nixon, Basics of nonverbal communication in the physical world, in Nonverbal Communication in Virtual Worlds: understanding and Designing Expressive Characters (ETC Press, 2014)

    Google Scholar 

  28. M. Nixon, P. Pasquier, M.S. El-Nasr, Delsartmap: applying delsartes aesthetic system to virtual agents, in Intelligent Virtual Agents (Springer, 2010), pp. 139–145

    Google Scholar 

  29. W.C. Stokoe, Semiotics and Human Sign Language. Walter de Gruyter Inc. (1972)

    Google Scholar 

  30. D. Brentari, A Prosodic Model of Sign Language Phonology (MIT Press, Cambridge, MA, 1999)

    Google Scholar 

  31. S. Prillwitz, R. Leven, H. Zienert, T. Hanke, J. Henning, Hamburg Notation System for Sign Languages—An Introductory Guide (University of Hamburg Press, 1989)

    Google Scholar 

  32. T. Johnston, The lexical database of AUSLAN (Australian Sign Language), in Proceedings of the First Intersign Workshop: Lexical Databases, Hamburg (1998)

    Google Scholar 

  33. K. Duarte, S. Gibet, Heterogeneous data sources for signed language analysis and synthesis: the signcom project, in Proceedings of LREC’10 (ELRA, 2010)

    Google Scholar 

  34. D. Vernon, G. Metta, G. Sandini, A survey of artificial cognitive systems: implications for the autonomous development of mental capabilities in computational agents. Trans. Evol. Comp 11(2), 151–180 (2007)

    Article  Google Scholar 

  35. W. Duch, R.J. Oentaryo, M. Pasquier, Cognitive architectures: where do we go from here? in Proceedings of the 2008 Conference on Artificial General Intelligence 2008: Proceedings of the First AGI Conference, Amsterdam, The Netherlands. (IOS Press, The Netherlands, 2008) pp. 122–136

    Google Scholar 

  36. F. Devillers, S. Donikian, F. Lamarche, J.F. Taille, A programming environment for behavioural animation. J. Vis. Comput. Animation 13(5), 263–274 (2002)

    Article  MATH  Google Scholar 

  37. J. Cassell, J. Sullivan, S. Prevost, E.F. Churchill, Embodied Conversational Agents (The MIT Press, 2000)

    Google Scholar 

  38. A. Kranstedt, S. Ko, I. Wachsmuth, MURML: a multimodal utterance representation markup language for conversational agents, in Proceedings of the AAMAS02 Workshop on ECA (2002)

    Google Scholar 

  39. H. Noot, Z. Ruttkay, Variations in gesturing and speech by gestyle. Int. J. Hum.-Comput. Stud. 62(2), 211–229 (2005)

    Article  Google Scholar 

  40. B. Hartmann, M. Mancini, S. Buisine, C. Pelachaud, Implementing expressive gesture synthesis for embodied conversational agents, in Gesture Workshop (Springer, 2005)

    Google Scholar 

  41. C. Pelachaud, Studies on Gesture Expressivity for a Virtual Agent, vol. 63(1) (Springer, 2009)

    Google Scholar 

  42. H. Vilhjálmsson, N. Cantelmo, J. Cassell, N. Ech Chafai, M. Kipp, S. Kopp, M. Mancini, S. Marsella, A.N. Marshall, C. Pelachaud, Z. Ruttkay, K.R. Thórisson, H. van Welbergen, R.J. van der Werf, The behavior markup language: recent developments and challenges, in Intelligent Virtual Agents, 7th International Conference, IVA 2007, Paris, France, September 17–19, 2007, (2007), pp. 99–111

    Google Scholar 

  43. S. Kopp, B. Krenn, S. Marsella, A.N. Marshall, C. Pelachaud, H. Pirker, K.R. Thórisson, H.H. Vilhjálmsson, Towards a common framework for multimodal generation: the behavior markup language, in Intelligent Virtual Agents, 6th International Conference, IVA 2006, Marina Del Rey, CA, USA, August 21–23 (2006), pp. 205–217

    Google Scholar 

  44. D. Tolani, A. Goswami, N.I. Badler, Real-time inverse kinematics techniques for anthropomorphic limbs. Graph. Models 62(5), 353–388 (2000)

    Article  MATH  Google Scholar 

  45. S. Kopp, I. Wachsmuth, Synthesizing multimodal utterances for conversational agents. J. Vis. Comput. Animation 15(1), 39–52 (2004)

    Google Scholar 

  46. A. Heloir, M. Kipp, Real-time animation of interactive agents: specification and realization. Appl. Artif. Intell. 24(6), 510–529 (2010)

    Article  Google Scholar 

  47. M. Gillies, X. Pan, M. Slater, Piavca: a framework for heterogeneous interactions with virtual characters, in Intelligent Virtual Agents, 8th International Conference, IVA 2008, Tokyo, Japan, September 1–3, 2008. Proceedings, (2008) pp. 494–495

    Google Scholar 

  48. L. Kovar, M. Gleicher, F. Pighin, Motion graphs. ACM Trans. Graph. 21(3), 473–482 (2002)

    Article  Google Scholar 

  49. M. Brand, A. Hertzmann, Style machines, in ACM SIGGRAPH 2000 (2000) pp. 183–192

    Google Scholar 

  50. A. Hertzmann, Machine learning for computer graphics: a manifesto and tutorial, in Computer Graphics and Applications, 2003. (IEEE, 2003), pp. 22–36

    Google Scholar 

  51. K. Grochow, S.L. Martin, A. Hertzmann, Z. Popovisć, Style-based inverse kinematics. ACM Trans. Graph. 23(3), 522–531 (2004)

    Article  Google Scholar 

  52. E. Hsu, K. Pulli, J. Popović, Style translation for human motion, in ACM Transactions on Graphics (TOG) (ACM, 2005), pp. 1082–1089. (Volume 24:3)

    Google Scholar 

  53. O. Arikan, D.A. Forsyth, J.F. O’Brien, Motion synthesis from annotations. ACM Trans. Graph. 22(3), 402–408 (2003)

    Article  MATH  Google Scholar 

  54. M. Müller, B. Andreas, H.P. Seidel, Efficient and robust annotation of motion capture data, in Proceedings of the ACM SIGGRAPHEurographics Symposium on Computer Animation (2009), pp. 17–26

    Google Scholar 

  55. C. Rose, B. Bodenheimer, M.F. Cohen, Verbs and adverbs: Multidimensional motion interpolation using radial basis functions. IEEE Comput. Graph. Appl. 18, 32–40 (1998)

    Article  Google Scholar 

  56. M. Stone, D. DeCarlo, O. Insuk, C. Rodriguez, A. Stere, A. Lees, C. Bregler, Speaking with hands: creating animated conversational characters from recordings of human performance. ACM Trans. Graph. 23(3), 506–513 (2004)

    Article  Google Scholar 

  57. S. Jörg, J.K. Hodgins, A. Safonova, Data-driven finger motion synthesis for gesturing characters. ACM Trans. Grap. 31(6), 189:1–189:7 (2012)

    Google Scholar 

  58. A. Safonova, J.K. Hodgins, Construction and optimal search of interpolated motion graphs, in ACM SIGGRAPH 2007 Papers, SIGGRAPH’07 (2007)

    Google Scholar 

  59. J. Starck, A. Hilton, Surface capture for performance-based animation. IEEE Comput. Graph. Appl. 27(3), 21–31 (2007)

    Article  Google Scholar 

  60. S. Gibet, N. Courty, K. Duarte, T. Le Naour, The signcom system for data-driven animation of interactive virtual signers: methodology and evaluation. Trans. Interact. Intel. Syst. 1(1), 6 (2011)

    Google Scholar 

  61. A. Héloir, N. Courty, S. Gibet, F. Multon, Temporal alignment of communicative gesture sequences. Comput. Animation Virtual Worlds 17, 347–357 (2006)

    Article  Google Scholar 

  62. A. Héloir, S. Gibet, A qualitative and quantitative characterisation of style in sign language gestures, in Gesture in Human-Computer Interaction and Simulation, GW 2007, Lisboa, Portugal. Lecture Notes in Artificial Intelligence (Springer, 2009)

    Google Scholar 

  63. Ed Hooks, Acting for Animators (Routledge, 2013)

    Google Scholar 

  64. L. Bishko, Our emphatic experience of believable characters, in Nonverbal Communication in Virtual Worlds: Understanding and Designing Expressive Characters (ETC Press, 2014)

    Google Scholar 

  65. J. Lecoq, J.G. Carasso, J.C. Lallias, D. Bradby, The Moving Body (Le Corps Poétique): Teaching Creative Theatre. Methuen Drama Modern Plays. Bloomsbury Academic (2009)

    Google Scholar 

  66. S. Murray, J. Keefe, Physical Theatres: A Critical Introduction (Taylor & Francis, 2007)

    Google Scholar 

Download references

Acknowledgements

This work is part of the Ingredible project, supported by the French National Research Agency (ANR).

Author information

Authors and Affiliations

  1. IRISA, University of Bretagne Sud, Campus de Tohannic, rue Yves Mainguy, 56017, Vannes, France

    Sylvie Gibet, Pamela Carreno-Medrano & Pierre-Francois Marteau

Authors
  1. Sylvie Gibet
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pamela Carreno-Medrano
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pierre-Francois Marteau
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sylvie Gibet .

Editor information

Editors and Affiliations

  1. LAAS-CNRS, Toulouse Cedex 4, France

    Jean-Paul Laumond

  2. LAAS-CNRS, Toulouse Cedex 4, France

    Naoko Abe

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Gibet, S., Carreno-Medrano, P., Marteau, PF. (2016). Challenges for the Animation of Expressive Virtual Characters: The Standpoint of Sign Language and Theatrical Gestures. In: Laumond, JP., Abe, N. (eds) Dance Notations and Robot Motion. Springer Tracts in Advanced Robotics, vol 111. Springer, Cham. https://doi.org/10.1007/978-3-319-25739-6_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-25739-6_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-25737-2

  • Online ISBN: 978-3-319-25739-6

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics