Abstract
The paper presents our initial attempts in building an audio video emotion recognition system. Both, audio and video sub-systems are discussed, and description of the database of spontaneous emotions is given. The task of labelling the recordings from the database according to different emotions is discussed and the measured agreement between multiple annotators is presented. Instead of focusing on the prosody in audio emotion recognition, we evaluate the possibility of using linear transformations (CMLLR) as features. The classification results from audio and video sub-systems are combined using sum rule fusion and the increase in recognition results, when using both modalities, is presented.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Song, M., Chen, C., You, M.: Audio-visual based emotion recognition using tripled hidden Markov model. In: Proceedings of Acoustics, Speech, and Signal Processing (ICASSP 2004), vol. 5, pp. 877–880 (2004)
Gajšek, R., et al.: Multi-Modal Emotional Database: AvID. Informatica 33, 101–106 (2009)
Busso, C., et al.: Analysis of emotion recognition using facial expressions, speech and multimodal information. In: ICMI 2004: Proceedings of the 6th international conference on Multimodal interfaces, pp. 205–211. ACM, New York (2004)
Eckman, P.: Strong Evidence for universals in facial expressions. Psychol. Bull. 115(2), 268–287 (1994)
Pantic, M., Rothkrantz, L.J.M.: Automatic analysis of facial expressions: the state of the art. IEEE TPAMI 22(12), 1424–1445 (2000)
Viola, P., Jones, M.: Robust real-time object detection. In: Proc. of the Second Intenrnational Workshop on Statistical and Computational Theories of Vision - Modeling, Learning, Computing and Sampling, Vancouver, Canada (2001)
Ang, J., et al.: Prosody-based automatic detection of annoyance and frustration in human-computer dialog. In: Proc. ICSLP 2002, vol. 3, pp. 2037–2040 (2002)
Gales, M.J.F.: Maximum likelihood linear transformations for HMM-based speech recognition. Computer Speech and Language 12(2), 75–98 (1998)
Mihelič, F., et al.: Spoken language resources at LUKS of the University of Ljubljana. Int. J. of Speech Technology 6(3), 221–232 (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gajšek, R., Štruc, V., Dobrišek, S., Žibert, J., Mihelič, F., Pavešić, N. (2009). Combining Audio and Video for Detection of Spontaneous Emotions. In: Fierrez, J., Ortega-Garcia, J., Esposito, A., Drygajlo, A., Faundez-Zanuy, M. (eds) Biometric ID Management and Multimodal Communication. BioID 2009. Lecture Notes in Computer Science, vol 5707. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-04391-8_15
Download citation
DOI: https://doi.org/10.1007/978-3-642-04391-8_15
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-04390-1
Online ISBN: 978-3-642-04391-8
eBook Packages: Computer ScienceComputer Science (R0)