Abstract
Virtual humans still lack naturalness in their nonverbal behaviour. We present a data-driven solution that moves towards a more natural synthesis of hand and arm gestures by recreating gestural behaviour in the style of a human performer. Our algorithm exploits the concept of gesture units to make the produced gestures a continuous flow of movement. We empirically validated the use of gesture units in the generation and show that it causes the virtual human to be perceived as more natural.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Vinayagamoorthy, V., Gillies, M., Steed, A., Tanguy, E., Pan, X., Loscos, C., Slater, M.: Building Expression into Virtual Characters. In: Eurographics Conference State of the Art Report, Vienna (2006)
Nass, C., Isbister, K., Lee, E.J.: Truth is beauty: Researching embodied conversational agents. In: Cassell, J., Sullivan, J., Prevost, S., Churchill, E. (eds.) Embodied Conversational Agents, pp. 374–402. MIT Press, Cambridge, MA (2000)
Thomas, F., Johnston, O.: The Illusion of Life: Disney Animation. Hyperion Press, New York (1981)
Cassell, J., Pelachaud, C., Badler, N., Steedman, M., Achorn, B., Becket, T., Douville, B., Prevost, S., Stone, M.: Animated Conversation: Rule-Based Generation of Facial Expression, Gesture & Spoken Intonation for Multiple Conversational Agents. In: Proceedings of SIGGRAPH 1994, pp. 413–420 (1994)
Gratch, J., Rickel, J., André, E., Badler, N., Cassell, J., Petajan, E.: Creating interactive virtual humans: Some assembly required. IEEE Intelligent Systems, 54–63 (July/ August, 2002)
Kita, S., van Gijn, I., van der Hulst, H.: Movement phases in signs and co-speech gestures, and their transcription by human coders. In: Wachsmuth, I., Fröhlich, M. (eds.) Gesture and Sign Language in Human-Computer Interaction, pp. 23–35. Springer, Berlin (1998)
Ruttkay, Z., Pelachaud, C. (eds.): From Brows to Trust: Evaluating Embodied Conversational Agents. Kluwer Academic Publishers, Dordrecht (2004)
Neff, M., Kipp, M., Albrecht, I., Seidel, H.-P.: Gesture Modeling and Animation Based on a Probabilistic Recreation of Speaker Style. Transactions on Graphics (accepted, 2007)
Kipp, M., Neff, M., Albrecht, I.: An Annotation Scheme for Conversational Gestures: How to economically capture timing and form. Journal on Language Resources and Evaluation - Special Issue on Multimodal Corpora (2007)
Kipp, M.: Gesture Generation by Imitation: From Human Behavior to Computer Character Animation. Dissertation.com, Boca Raton, Florida (2004)
Kendon, A.: Gesture – Visible Action as Utterance. Cambridge University Press, Cambridge (2004)
McNeill, D.: Hand and Mind: What Gestures Reveal about Thought. University of Chicago Press, Chicago (1992)
Stone, M., DeCarlo, D., Oh, I., Rodriguez, C., Stere, A., Lees, A., Bregler, C.: Speaking with hands: Creating animated conversational characters from recordings of human performance. In: Proc. SIGGRAPH 2004, pp. 506–513 (2004)
Kopp, S., Wachsmuth, I.: Synthesizing multimodal utterances for conversational agents. Computer Animation and Virtual Worlds 15(1), 39–52 (2004)
de Ruiter, J.P.: The production of gesture and speech. In: McNeill, D. (ed.) Language and Gesture: Window into Thought and Action, pp. 284–311. Cambridge University Press, Cambridge (2000)
Cassell, J., Vilhjálmsson, H., Bickmore, T.: BEAT: the Behavior Expression Animation Toolkit. In: Proceedings of SIGGRAPH 2001, pp. 477–486 (2001)
Lee, J., Marsella, S.: Nonverbal behavior generator for embodied conversational agents. In: Gratch, J., Young, M., Aylett, R., Ballin, D., Olivier, P. (eds.) IVA 2006. LNCS (LNAI), vol. 4133, pp. 243–255. Springer, Heidelberg (2006)
Hartmann, B., Mancini, M., Buisine, S., Pelachaud, C.: Design and evaluation of expressive gesture synthesis for ecas. In: Proc. AAMAS (2005)
Chi, D.M., Costa, M., Zhao, L., Badler, N.I.: The EMOTE model for effort and shape. In: Proc. SIGGRAPH 2000, pp. 173–182 (2000)
Hartmann, B., Mancini, M., Pelachaud, C.: Implementing Expressive Gesture Synthesis for Embodied Conversational Agents. In: Gibet, S., Courty, N., Kamp, J.-F. (eds.) GW 2005. LNCS (LNAI), vol. 3881, pp. 188–199. Springer, Heidelberg (2006)
Neff, M., Fiume, E.: AER: Aesthetic Exploration and Refinement for expressive character animation. In: Proc. ACM SIGGRAPH / Eurographics Symposium on Computer Animation 2005, pp. 161–170. ACM Press, New York (2005)
Hodgins, J.K., Wooten, W.L., Brogan, D.C., O’Brien, J.F.: Animating human athletics. In: Proc. SIGGRAPH 1995, pp. 71–78 (1995)
Faloutsos, P., van de Panne, M., Terzopoulos, D.: The virtual stuntman: Dynamic characters with a repertoire of autonomous motor skills. Computers & Graphics 25(6), 933–953 (2001)
Zordan, V.B., Hodgins, J.K.: Motion capture-driven simulations that hit and react. In: Proc. ACM SIGGRAPH Symposium on Computer Animation, pp. 89–96. ACM Press, New York (2002)
Noot, H., Ruttkay, Z.: Gesture in style. In: Camurri, A., Volpe, G. (eds.) GW 2003. LNCS (LNAI), vol. 2915, pp. 324–337. Springer, Heidelberg (2004)
Webb, R.: Linguistic Properties of Metaphoric Gestures. UMI, New York (1997)
Boersma, P., Weenink, D.: Praat: doing phonetics by computer (version 4.3.14) [computer program] (2005), Retrieved from http://www.praat.org/
Kipp, M.: Anvil – a Generic Annotation Tool for Multimodal Dialogue. In: Proceedings of Eurospeech, pp. 1367–1370 (2001)
McNeill, D.: Gesture and Thought. University of Chicago Press, Chicago (2005)
Schegloff, E.A.: On some gestures’ relation to talk. In: Atkinson, J.M., Heritage, J. (eds.) Structures of Social Action, pp. 266–296. Cambridge University Press, Cambridge (1984)
Steedman, M.: Information structure and the syntax-phonology interface. Linguistic Inquiry 34, 649–689 (2000)
Kendon, A.: Gesticulation and speech: Two aspects of the process of utterance. In: Key, M.R. (ed.) Nonverbal Communication and Language, pp. 207–227. Mouton, The Hague (1980)
Neff, M., Fiume, E.: Modeling tension and relaxation for computer animation. In: Proc. ACM SIGGRAPH Symposium on Computer Animation 2002, pp. 81–88. ACM Press, New York (2002)
Press, W.H., Tukolsky, S.A., Vetterling, W.T., Flannery, B.P.: Numerical Recipes in C: The Art of Scientific Computing, 2nd edn. Cambridge University Press, Cambridge (1992)
Hollars, M.G., Rosenthal, D.E., Sherman, M.A.: SD/FAST User’s Manual. Symbolic Dynamics Inc. (1994)
McCrae, R.R., John, O.P.: An Introduction to the Five-Factor Model and Its Applications. Journal of Personality 60, 175–215 (1992)
Martin, J.C., Niewiadomski, R., Devillers, L., Buisine, S., Pelachaud, C.: Multimodal Complex Emotions: Gesture Expressivity And Blended Facial Expressions. Special issue of the Journal of Humanoid Robotics 3, 269–291 (2006)
Krämer, N.C., Tietz, B., Bente, G.: Effects of embodied interface agents and their gestural activity. In: Rist, T., Aylett, R., Ballin, D., Rickel, J. (eds.) IVA 2003. LNCS (LNAI), vol. 2792, pp. 292–300. Springer, Heidelberg (2003)
Frey, S.: Die Macht des Bildes: der Einfluß der nonverbalen Kommunikation auf Kultur und Politik. Verlag Hans Huber, Bern (1999)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kipp, M., Neff, M., Kipp, K.H., Albrecht, I. (2007). Towards Natural Gesture Synthesis: Evaluating Gesture Units in a Data-Driven Approach to Gesture Synthesis. In: Pelachaud, C., Martin, JC., André, E., Chollet, G., Karpouzis, K., Pelé, D. (eds) Intelligent Virtual Agents. IVA 2007. Lecture Notes in Computer Science(), vol 4722. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74997-4_2
Download citation
DOI: https://doi.org/10.1007/978-3-540-74997-4_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74996-7
Online ISBN: 978-3-540-74997-4
eBook Packages: Computer ScienceComputer Science (R0)