Abstract
Communication between humans is complex and is not limited to verbal signals; emotions are conveyed with gesture, pose and facial expression. Facial Emotion Recognition and Analysis (FERA), the techniques by which non-verbal communication is quantified, is an exemplar case where humans consistently outperform computer methods. While the field of FERA has seen many advances, no system has been proposed which scales well to very large data sets. The challenge for computer vision is how to automatically and non-heuristically downsample the data while maintaining the maximum representational power that does not sacrifice accuracy. In this paper, we propose a method inspired by human vision and attention theory [2]. Video is segmented into temporal partitions with a dynamic sampling rate based on the frequency of visual information. Regions are homogenized by a match-score fusion technique. The approach is shown to provide classification rates higher than the baseline on the AVEC 2011 video-subchallenge dataset [15].
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Chang, C., Lin, C.: LibSVM: A Library for Support Vector Machines, http://www.csie.ntu.edu.tw/~cjlin/libsvm
Findlay, J., Gilchrist, I.: Active Vision: The Psychology of Looking and Seeing. Oxford University Press, Oxford (2003)
Fontaine, J., Scherer, K., Roesch, B., Ellsworth, P.E.: The World of Emotions is Not Two-dimensional. Psychological Science 18(2), 1050–1057 (2007)
Gautama, T., Van Hulle, M.: A Phase-Based Approach to the Estimation of the Optical Flow Field Using Spatial Filtering. IEEE Trans. on. Neural Nets 13(5), 1127–1136 (2002)
Haber, R., Hershenson, M.: The Psychology of Visual Perception. Holt, Rinehart & Winston, Oxford (1973)
Jain, A.K., Nandakumar, K., Ross, A.: Score Normalization in Multimodal Biometric Systems. Pattern Recognition 38(12), 2270–2285 (2005)
Jiang, B., Valstar, M., Pantic, M.: Action Unit Detection Using Sparse Appearance Descriptors in Space-time Video Volumes. In: IEEE Intl. Conf. on Automatic Face and Gesture Recognition (2011)
Liu, C., Yuen, J., Torralba, A.: SIFT Flow: Dense Correspondence across Scenes and Its Applications. IEEE Trans. on Pattern Analysis and Machine Intelligence 33(5), 978–994 (2011)
Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The Extended Cohn-Kande Dataset (CK+): A complete facial expression dataset for action unit and emotion-specified expression. In: Human Communicative Behavior Analysis, Workshop of CVPR (2010)
Lyons, M., Akamatsu, S., Kamachi, M., Gyoba, J.: Coding Facial Expressions with Gabor Wavelets. In: Proc. IEEE Intl. Conf. on Automatic Face and Gesture Recognition (1998)
Mckeown, G., Valstar, M.F., Cowie, R., Pantic, M.: The Semaine Corpus of Emotionally Coloured Character Interactions. In: IEEE Intl. Conf. on Multimedia and Expo. (2010)
Ojansivu, V., Heikkila, J.: Blur Insensitive Texture Classification Using Local Phase Quantization. In: IEEE Intl. Conf. on Image and Signal Processing (2008)
Pantic, M., Rothkrantz, L.: Automatic analysis of facial expressions: the state of the art. IEEE Trans. on Pattern Analysis and Machine Intelligence 22(12), 1424–1445 (2000)
Samal, A., Iyengar, P.A.: Automatic recognition and analysis of human faces and facial expressions: a survey. Pattern Recognition 22(1) (1992)
Schuller, B., Valstar, M., Eyben, F., McKeown, G., Cowie, R., Pantic, M.: AVEC 2011 – The First Int’l. Audio/Visual Emotion Challenge. In: D´Mello, S., et al. (eds.) ACII 2011, Part II. LNCS, vol. 6975, pp. 415–424. Springer, Heidelberg (2011)
Snelick, R., Uludag, U., Mink, A., Indovina, M., Jain, A.K.: Large Scale Evaluation of Multimodal Biometric Authentication Using State-of-the-Art Systems. IEEE Trans. on Pattern Analysis and Machine Intelligence 27(3), 450–455 (2005)
Szeliski, R., Zabih, R., Scharstein, D., Veksler, O., Kolmogorov, V., Agarwala, A., et al.: A comparative study of energy minimization methods for Markov random fields with smoothness-based priors. IEEE Trans. on Pattern Analysis and Machine Intelligence 30(6), 1068–1080 (2008)
Valstar, M.F., Jiang, B., Mehu, M., Pantic, M., Scherer, K.: The First Facial Expression Recognition and Analysis Challenge. In: Proc. of IEEE Intl. Conf. on Face and Gesture Recognition (2011)
Valstar, M., Pantic, M.: Induced disgust, happiness and surprise: an addition to the MMI facial expression database. In: Proc. 3rd Intern. Workshop on EMOTION (satellite of LREC): Corpora for Research on Emotion and Affect (2010)
Viola, P., Jones, M.: Robust Real-Time Face Detection. Intl. J. on Computer Vision (2002)
Yang, S., Bhanu, B.: Facial expression recognition using emotion avatar image. The First Facial Expression Recognition and Analysis Challenge. In: IEEE Intl. Conf. on Face and Gesture Recognition (2011)
Yu, J., Bhanu, B.: Evolutionary feature synthesis for facial expression recognition. Pattern Recognition Letters 27 (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Cruz, A., Bhanu, B., Yang, S. (2011). A Psychologically-Inspired Match-Score Fusion Model for Video-Based Facial Expression Recognition. In: D’Mello, S., Graesser, A., Schuller, B., Martin, JC. (eds) Affective Computing and Intelligent Interaction. ACII 2011. Lecture Notes in Computer Science, vol 6975. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24571-8_45
Download citation
DOI: https://doi.org/10.1007/978-3-642-24571-8_45
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-24570-1
Online ISBN: 978-3-642-24571-8
eBook Packages: Computer ScienceComputer Science (R0)