Skip to main content

Advertisement

SpringerLink
Log in

Generating distinctive behavior for Embodied Conversational Agents

  • Original Paper
  • Published:
Journal on Multimodal User Interfaces Aims and scope Submit manuscript

Abstract

In this paper we describe an algorithm for generating distinctive behavior for Embodied Conversational Agents. To this aim, we introduce the concepts of agent’s general behavior tendency, named Baseline, and local behavior tendency, called in turn Dynamicline. Depending on the communicative intentions of the agent, the Baseline is modulated. The obtained behavior tendency corresponds to the Dynamicline which is then used to determine the nonverbal signals and their expressivity the agent will produce to communicate its intentions. We also propose a system to extract the movement expressivity of a human user standing in front of a camera. The extracted characteristics are then used to characterize the agent’s Baseline. We end the paper by presenting an evaluation study of our model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Allbeck J, Badler N (2003) Representing and parameterizing agent behaviors. In: Prendinger H, Ishizuka M (eds) Life-like characters: tools, affective functions and applications. Springer, Berlin

    Google Scholar 

  2. Allwood J (2002) Bodily communication - dimensions of expression and content. In: Granstrom B, House D, Karlsson I (eds) Multimodality in language and speech systems. Kluwer, Dordrecht, pp 7–26

    Google Scholar 

  3. Argyle M (1988) Bodily communication, 2nd edn. Methuen & Co, London

    Google Scholar 

  4. Badler N, Phillips C, Webber B (1993) Simulating humans: computer graphics animation and control. Oxford University Press, Oxford

    MATH  Google Scholar 

  5. Ball G, Breese J (2000) Emotion and personality in a conversational agent. In: Cassell J, Sullivan J, Prevost S, Churchill E (eds) Embodied conversational characters. MIT Press, Cambridge

    Google Scholar 

  6. Byun M, Badler N (2002) Facemote: qualitative parametric modifiers for facial animations. In: Proceedings of the 2002 ACM SIGGRAPH/Eurographics symposium on computer animation. ACM, New York, p 71

    Google Scholar 

  7. Caldognetto EM, Poggi I (2000) Dall’analisi della multimodalità quotidiana alla costruzione di facce parlanti. In: IV Conference of GFS (Gruppo di Fonetica Sperimentale). Padova, Italy

  8. Camurri A, Mazzarino B, Volpe G (2004) Analysis of expressive gesture: The eyesweb expressive gesture processing library. In: Camurri A, Volpe G (eds) Gesture-based communication in human-computer interaction. LNAI, vol 2915. Springer, Berlin

    Chapter  Google Scholar 

  9. Cassell J, Nakano YI, Bickmore TW, Sidner CL, Rich C (2001) Annotating and generating posture from discourse structure in embodied conversational agents. In: Workshop on representating, annotating, and evaluating non-verbal and verbal communicative acts to achieve contextual embodied agents. Montreal, Quebec

  10. Cassell J, Sullivan J, Prevost S, Churchill E (2000) Embodied conversational agents. MIT Press, Cambridge

    Google Scholar 

  11. DeCarolis B, Pelachaud C, Poggi I (2000) Verbal and nonverbal discourse planning. In: Workshop on achieving human-like behaviors. Autonomous Agents

  12. DeCarolis B, Pelachaud C, Poggi I, Steedman M (2004) APML, a mark-up language for believable behavior generation. In: Prendinger H, Ishizuka M (eds) Life-like characters, cognitive technologies. Springer, Berlin, pp 65–86

    Google Scholar 

  13. DiPaola S, Arya A (2004) Affective communication remapping in musicface system. In: Electronic imaging & visual arts

  14. Egges A, Kshirsagar S, Magnenat-Thalmann N (2003) A model for personality and emotion simulation. In: Knowledge-based intelligent information and engineering systems. Springer, Berlin, pp 453–461

    Google Scholar 

  15. Ekman P, Friesen W, O’Sullivan M, Scherer K (1980) Relative importance of face, body, and speech in judgments of personality and affect. J Pers Soc Psychol 38:270–277

    Article  Google Scholar 

  16. Gallaher PE (1992) Individual differences in nonverbal behavior: Dimensions of style. J Pers Soc Psychol 63(1):133–145

    Article  Google Scholar 

  17. Hartmann B, Mancini M, Buisine S, Pelachaud C (2005) Design and evaluation of expressive gesture synthesis for embodied conversational agents. In: Third international joint conference on autonomous agents & multi-agent systems. Utretch

  18. Kallmann M, Marsella S (2005) Hierarchical motion controllers for real-time autonomous virtual humans. In: Intelligent virtual agents. Springer, Berlin, pp 253–265

    Chapter  Google Scholar 

  19. Kendon A (2004) Gesture: visible action as utterance. Cambridge University Press, Cambridge

    Google Scholar 

  20. Kipp M (2006) Creativity meets automation: Combining nonverbal action authoring with rules and machine learning. In: Intelligent virtual agents, pp 230–242

  21. Kipp M, Neff M, Kipp KH, Albrecht I (2007) Towards natural gesture synthesis: Evaluating gesture units in a data-driven approach to gesture synthesis. In: Proceedings of the 7th international conference on intelligent virtual agents. Lecture notes in computer science, vol 4722. Springer, Berlin, pp 15–28

    Google Scholar 

  22. Kopp S, Wachsmuth I (2002) Model-based animation of coverbal gesture. In: Proceedings of computer animation, pp 252–257

  23. Laban R (1971) The mastery of movement Plays. Inc, Boston

  24. Lebourque T, Gibet S (1999) High level specification and control of communication gestures: the GESSYCA system. In: Proceedings of computer animation, vol 99

  25. Lester J, Voerman J, Towns S, Callaway C (1997) Cosmo: A life-like animated pedagogical agent with deictic believability, pp 61–69

  26. Mancini M, Pelachaud C (2007) Dynamic behavior qualifiers for conversational agents. In: Lecture notes in computer science, vol 4722. Springer, Berlin, p 112

    Google Scholar 

  27. Mancini M, Pelachaud C (2008) Distinctiveness in multimodal behaviors. In: Proceedings of the 7th international joint conference on autonomous agents and multiagent systems, pp 159–166

  28. McNeill D (1992) Hand and mind: what gestures reveal about thought. University of Chicago Press, Chicago

    Google Scholar 

  29. Neff M, Fiume E (2004) Artistically based computer generation of expressive motion. In: Proceedings of the AISB symposium on language, speech and gesture for expressive characters, pp 29–39

  30. Neff M, Fiume E (2005) AER: Aesthetic exploration and refinement for expressive character animation. In: Proceedings of the 2005 ACM SIGGRAPH/Eurographics symposium on computer animation. ACM Press, New York, pp 161–170

    Chapter  Google Scholar 

  31. Neff M, Kim Y (2009) Interactive editing of motion style using drives and correlations. In: SCA ’09: Proceedings of the 2009 ACM SIGGRAPH/Eurographics symposium on computer animation. ACM, New York, pp 103–112

    Chapter  Google Scholar 

  32. Neff M, Kipp M, Albrecht I, Seidel H (2008) Gesture modeling and animation based on a probabilistic re-creation of speaker style. ACM Trans Graph 27(1)

  33. Niewiadomski R, Pelachaud C (2007) Intelligent expressions of emotions. In: Affective computing and intelligent interaction. Lecture Notes in Computer Science, vol 4738. Springer, Berlin, pp 12–23

    Chapter  Google Scholar 

  34. Pelachaud C (2005) Multimodal expressive embodied conversational agents. In: MULTIMEDIA ’05: Proceedings of the 13th annual ACM international conference on multimedia. ACM Press, New York, pp 683–689

    Chapter  Google Scholar 

  35. Poggi I (2007) Mind, hands, face and body. A goal and belief view of multimodal communication. Weidler, Berlin

    Google Scholar 

  36. Poggi I, Pelachaud C (2000) Performative facial expressions in animated faces. In: Embodied conversational agents. MIT Press, Cambridge, pp 155–188

    Google Scholar 

  37. Poggi I, Pelachaud C, Caldognetto EM (2003) Gestural mind markers in ECAs. In: Proceedings of the second international joint conference on Autonomous agents and multiagent systems. ACM Press, New York, pp 1098–1099

    Chapter  Google Scholar 

  38. Reeves B, Nass C (1996) The media equation: How people treat computers, television and new media like real people and places. CSLI Publications, Stanford

    Google Scholar 

  39. Ruttkay Z, Pelachaud C, Poggi I, Noot H (2008) Exercises of style for virtual humans. In: Canamero L, Aylett R (eds) Animating expressive characters for social interactions. Benjamins, Amsterdam

    Google Scholar 

  40. Scherer KR, Wallbott HG (1985) Analysis of nonverbal behavior. Handbook of discourse analysis, vol 2, pp 199–230

  41. Schröder M, Pirker H, Lamolle M (2006) First suggestions for an emotion annotation and representation language. In: Devillers, L, Martin, JC, Cowie, R, Douglas-Cowie, E, Batliner, A (eds) Proceedings of the international conference on language resources and evaluation: workshop on corpora for research on emotion and affect, pp 88–92. Genova, Italy

  42. Vilhjaĺmsson H, Cantelmo N, Cassell J, Chafai NE, Kipp M, Kopp S, Mancini M, Marsella S, Marshall AN, Pelachaud C, Ruttkay Z, Thoŕisson KR, van Welbergen H, van der Werf R (2007) The behavior markup language: Recent developments and challenges. In: 7th international conference on intelligent virtual agents

  43. Wallbott HG (1998) Bodily expression of emotion. Eur J Soc Psychol 28:879–896

    Article  Google Scholar 

  44. Wallbott HG, Scherer KR (1986) Cues and channels in emotion recognition. J Pers Soc Psychol 51(4):690–699

    Article  Google Scholar 

  45. Zhao L, Badler N (2005) Acquiring and validating motion qualities from live limb gestures. Graph Models 67(1):1–16

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. InfoMus Lab, DIST, Università di Genova, Viale Causa 13, 16145, Genova, Italy

    Maurizio Mancini

  2. CNRS-LTCI, TELECOM ParisTech, 37 rue Dareau, 75014, Paris, France

    Catherine Pelachaud

Authors
  1. Maurizio Mancini
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Catherine Pelachaud
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Maurizio Mancini.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Mancini, M., Pelachaud, C. Generating distinctive behavior for Embodied Conversational Agents. J Multimodal User Interfaces 3, 249–261 (2009). https://doi.org/10.1007/s12193-010-0048-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12193-010-0048-y

Keywords

Search

Navigation