Abstract
The modeling of realistic emotional behavior is needed for various applications in multimodal human-machine interaction such as emotion detection in a surveillance system or the design of natural Embodied Conversational Agents. Yet, building such models requires appropriate definition of various levels for representing: the emotional context, the emotion itself and observed multimodal behaviors. This paper presents the multi-level emotion and context coding scheme that has been defined following the annotation of fifty one videos of TV interviews. Results of annotation analysis show the complexity and the richness of the real-life data: around 50% of the clips feature mixed emotions with multi-modal conflictual cues. A typology of mixed emotional patterns is proposed showing that cause-effect conflict and masked acted emotions are perceptually difficult to annotate regarding the valence dimension.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Douglas-Cowie, E., Campbell, N., Cowie, R., Roach, R.: Emotional speech, Towards a new generation of databases. Speech Communication (2003)
Maya, V., Lamolle, M., Pelachaud, C.: Influences on Embodied Conversational Agent’s Expressivity: Toward an Individualization of the ECAs. In: AISB convention. Symposium on Language, Speech and Gesture for Expressive Characters (2004)
Ekman, P.: Basic emotions. In: Dalgleish, T., Power, M.J. (eds.) Handbook of Cognition & Emotion, pp. 301–320. John Wiley, New York (1999)
Plutchik, R.: The psychology and Biology of Emotion. Harper Collins College, New York (1994)
Batliner, A., Fisher, K., Huber, R., Spilker, J., Noth, E.: Desperately seeking emotions or: Actors, wizards, and human beings. In: SpeechEmotion (2000)
Schröder, M.: Dimensional Emotion Representation as a Basis for Speech Synthesis with Non-extreme Emotions. Affective Dialogue Systems, 209–220 (2004)
Scherer, K.R.: Appraisal theories. In: Dalgleish, T., Power, M. (eds.) Handbook of cognition and emotion, pp. 637–639. Wiley, Chichester (1999)
Gratch, J., Marsella, S.: Domain-independent Framework for Modeling Emotion. Journal of Cognitive Systems Research 5(4), 269–306 (2004)
Abrilian, S., Devillers, L., Buisine, S., Martin, J.-C.: EmoTV1: Annotation of Real-life Emotions for the Specification of Multimodal Affective Interfaces. In: HCI International, Las Vegas, USA (2005a)
Devillers, L., Vidrascu, L., Lamel, L.: Challenges in real-life emotion annotation and machine learning based detection. Journal of Neural Networks (2005) (to appear)
Abrilian, A., Martin, J.-C., Devillers, L.: A Corpus-Based Approach for the Modeling of Multimodal Emotional Behaviors for the Specification of Embodied Agents. In: HCI International, Las Vegas, USA (2005b)
Martin, J.-C., Abrilian, S., Devillers, L.: Annotating Multimodal Behaviors Occuring during Non Basic Emotions. In: Tao, J., Tan, T., Picard, R.W. (eds.) ACII 2005. LNCS, vol. 3784, pp. 550–557. Springer, Heidelberg (2005)
Lamolle, M., Mancini, M., Pelachaud, C., Abrilian, S., Martin, J.-C., Devillers, L.: Contextual Factors and Adaptative Multimodal Human-Computer Interaction: Multi-Level Specification of Emotion and Expressivity in ECAs. Context (2005)
Kipp, M.: Anvil - A Generic Annotation Tool for Multimodal Dialogue. In: Eurospeech (2001)
Cowie, R.: Emotion recognition in human-computer interaction. IEEE Signal processing Magazine (2001)
Cowie, R., Douglas-Cowie, E., Savvidou, S., McMahon, E., Sawey, M., Schröder, M.: FEELTRACE: An Instrument for Recording Perceived Emotion in Real Time. In: ISCA Workshop on Speech & Emotion, Northern Ireland, pp. 19–24 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Devillers, L., Abrilian, S., Martin, JC. (2005). Representing Real-Life Emotions in Audiovisual Data with Non Basic Emotional Patterns and Context Features. In: Tao, J., Tan, T., Picard, R.W. (eds) Affective Computing and Intelligent Interaction. ACII 2005. Lecture Notes in Computer Science, vol 3784. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11573548_67
Download citation
DOI: https://doi.org/10.1007/11573548_67
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29621-8
Online ISBN: 978-3-540-32273-3
eBook Packages: Computer ScienceComputer Science (R0)