Abstract
In this paper we present a two-steps approach towards the creation of affective Embodied Conversational Agents (ECAs): annotation of a real-life non-acted emotional corpus and animation by copy-synthesis. The basis of our approach is to study how coders perceive and annotate at several levels the emotions observed in a corpus of emotionally rich TV video interviews. We use their annotations to specify the expressive behavior of an agent at several levels. We explain how such an approach can be useful for providing knowledge as input for the specification of non-basic patterns of emotional behaviors to be displayed by the ECA (e.g. which perceptual cues and levels of annotation are required for enabling the proper recognition of the emotions).
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Abrilian, S., Devillers, L., Buisine, S., Martin, J.-C.: EmoTV1: Annotation of Real-life Emotions for the Specification of Multimodal Affective Interfaces. In: HCI International, Las Vegas, USA (2005a)
Abrilian, S., Martin, J.-C., Devillers, L.: A Corpus-Based Approach for the Modeling of Multimodal Emotional Behaviors for the Specification of Embodied Agents. In: HCI International, Las Vegas, USA (2005b)
Batliner, A., Fisher, K., Huber, R., Spilker, J., Noth, E.: Desperately seeking emotions or: Actors, wizards, and human beings. Speech Emotion, pp. 195–200 (2000)
Boone, R.T., Cunningham, J.G.: Children’s decoding of emotion in expressive body movement: The development of cue attunement. Developmental Psychology 34(5) (1998)
Calbris, G.: The semiotics of French gestures. University Press Bloomington Indiana (1990)
Cao, Y., Faloutsos, P., Kohler, E., Pighin, F.: Real-time Speech Motion Synthesis from Recorded Motions. In: ACM SIGGRAPH/Eurographics Symposium on Computer Animation (2004)
Cassell, J., Stone, M., Hao, Y.: Coordination and context-dependence in the generation of embodied conversation. In: INLG, pp. 171–178 (2000)
Cassell, J., Sullivan, J., Prevost, S., Churchill, E.: Embodied Conversational Agents. MIT Press, Cambridge (2000)
Cassell, J., Vilhjàlmsson, H., Bickmore, T.: BEAT: the Behavior Expression Animation Toolkit. In: 28th annual conference on Computer graphics and interactive techniques, pp. 477–486 (2001)
Chi, D., Costa, M., Zhao, L., Badler, N.: The EMOTE model for effort and shape. In: 27th annual conference on Computer graphics and interactive techniques, pp. 173–182 (2000)
Cowie, R.: Emotion recognition in human-computer interaction. IEEE Signal processing Magazine 18 (2001)
Cowie, R., Douglas-Cowie, E., Savvidou, S., McMahon, E., Sawey, M., Schröder, M.: ’FEELTRACE’: An Instrument for Recording Perceived Emotion in Real Time. In: ISCA Workshop on Speech & Emotion (2000)
Craggs, R., Wood, M.M.: A categorical annotation scheme for emotion in the linguistic content. In: André, E., Dybkjær, L., Minker, W., Heisterkamp, P. (eds.) ADS 2004. LNCS (LNAI), vol. 3068, pp. 89–100. Springer, Heidelberg (2004)
De Carolis, B., Pelachaud, C., Poggi, I., Steedman, M.: APML, a Markup Language for Believable Behavior Generation. In: Life-like characters. Tools, affective functions and applications. Springer, Heidelberg (2004)
DeMeijer, M.: The contribution of general features of body movement to the attribution of emotions. Journal of Nonverbal Behavior 13 (1989)
Douglas-Cowie, E., Campbell, N., Cowie, R., Roach, P.: Emotional speech; Towards a new generation of databases. Speech Communication 40 (2003)
Egges, A., Kshirsagar, S., Magnenat-Thalmann, N.: Imparting Individuality to Virtual Humans. In: First International Workshop on Virtual Reality Rehabilitation, Lausanne, Switzerland (2002)
Ekman, P.: Basic emotions. Handbook of Cognition & Emotion. John Wiley, Chichester (1999)
Ekman, P., Friesen, W.V.: Manual for the facial action coding system. Consulting Psychology Press Palo Alto, CA (1978)
Gallaher, P.: Individual differences in nonverbal behavior: Dimensions of style. Journal of Personality and Social Psychology 63 (1992)
Hartmann, B., Mancini, M., Pelachaud, C.: Formational Parameters and Adaptive Prototype Instantiation for MPEG-4 Compliant Gesture Synthesis. Computer Animation (2002)
Hartmann, B., Mancini, M., Pelachaud, C.: Implementing Expressive Gesture Synthesis for Embodied Conversational Agents. In: Gesture Workshop, Vannes (2005)
Kendon, A.: Human gesture. Tools, Language and Intelligence. Cambridge University Press, Cambridge (1993)
Kipp, M.: Anvil - A Generic Annotation Tool for Multimodal Dialogue. Eurospeech 2001 (2001)
Kipp, M.: Gesture Generation by Imitation. From Human Behavior to Computer Character Animation. Boca Raton, Dissertation.com Florida (2004)
Kopp, S., Wachsmuth, I.: A knowledge-based approach for lifelike gesture animation. In: 14th European Conference on Artificial Intelligence, ECAI (2000)
Kshirsagar, S., Molet, T., Magnant-Thalmann, N.: Principal components of expressive speech animation. Computer Graphics International, 38–44 (2001)
Maya, V., Lamolle, M., Pelachaud, C.: Influences on Embodied Conversational Agent’s Expressivity: Toward an Individualization of the ECAs. In: AISB 2004 convention. Symposium on Language, Speech and Gesture for Expressive Characters, pp. 75–85 (2004)
McNeill, D.: Hand and mind - what gestures reveal about thoughts. University of Chicago Press (1992)
Newlove, J.: Laban for actors and dancers. Routledge New York (1993)
Noot, H., Ruttkay, Z.: Gesture in Style. In: Camurri, A., Volpe, G. (eds.) GW 2003. LNCS (LNAI), vol. 2915, pp. 324–337. Springer, Heidelberg (2004)
Ochs, M., Niewiadomski, R., Pelachaud, C., Sadek, D.: Intelligent expressions of emotions. In: Tao, J., Tan, T., Picard, R.W. (eds.) ACII 2005. LNCS, vol. 3784, pp. 707–714. Springer, Heidelberg (2005)
Pandzic, I.S.: Facial Motion Cloning. Elsevier Graphical Models Journal 65(6) (2003)
Pelachaud, C., Carofiglio, V., De Carolis, B., De Rosis, F., Poggi, I.: Embodied Contextual Agent in Information Delivering Application. In: First International Joint Conference on Autonomous Agent and Multiagent Systems(AAMAS), Bologna, Italy, pp. 758–765 (2002)
Plutchik, R.: The psychology and Biology of Emotion. Harper Collins College, New York (1994)
Poggi, I.: Mind Markers. In: 5th International Pragmatics Conference, Mexico City (1996)
Poggi, I., Pelachaud, C., de Rosis, F.: Eye communication in a conversational 3D synthetic agent. AI Communications. Special Issue on Behavior Planning for Life-Like Characters and Avatars 13(3) (2000)
Prendinger, H., Ishizuka, M.: Life-like characters. Tools, affective functions and applications. Springer, Heidelberg (2004)
Scherer, K.R.: Emotion. Introduction to Social Psychology: A European perspective. Blackwell, Oxford (2000)
Tepper, P., Kopp, S., Cassell, J.: Content in Context: Generating Language and Iconic Gesture without a Gestionary. In: Workshop on Balanced Perception and Action in ECAs at Automous Agents and Multiagent Systems (AAMAS), New York, NY (2004)
Theune, M., Heylen, D., Nijholt, A.: Generating Embodied Information Presentations. In: Multimodal Intelligent Information Presentation. Kluwer Academic Publishers, Dordrecht (2004)
Tsapatsoulis, N., Raouzaiou, A., Kollias, S., Cowie, R., Douglas-Cowie, E.: Emotion Recognition and Synthesis based on MPEG-4 FAPs. In: MPEG-4 Facial Animation. John Wiley & Sons, Chichester (2002)
Wallbott, H.G.: Bodily expression of emotion. European Journal of Social Psychology 28 (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Martin, JC., Abrilian, S., Devillers, L., Lamolle, M., Mancini, M., Pelachaud, C. (2005). Levels of Representation in the Annotation of Emotion for the Specification of Expressivity in ECAs. In: Panayiotopoulos, T., Gratch, J., Aylett, R., Ballin, D., Olivier, P., Rist, T. (eds) Intelligent Virtual Agents. IVA 2005. Lecture Notes in Computer Science(), vol 3661. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11550617_34
Download citation
DOI: https://doi.org/10.1007/11550617_34
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-28738-4
Online ISBN: 978-3-540-28739-1
eBook Packages: Computer ScienceComputer Science (R0)