Abstract
Automated facial expression classification is very important in the design of new human-computer interaction modes and multimedia interactive services and arises as a difficult, yet crucial, pattern recognition problem. Recently, we have been building such a system, called NEU-FACES, which processes multiple camera images of computer user faces with the ultimate goal of determining their affective state. In here, we present results from an empirical study we conducted on how humans classify facial expressions, corresponding error rates, and to which degree a face image can provide emotion recognition from the perspective of a human observer. This study lays related system design requirements, quantifies statistical expression recognition performance of humans, and identifies quantitative facial features of high expression discrimination and classification power.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Csikszentmihalyi, M.: Flow: The Psychology of Optimal Experience, Harper and Row, New York (1994)
De Silva, L.C., Miyasato, T., Nakatsu, R.: Facial Emotion Recognition Using Multimodal Information. In: Proc. IEEE Int. Conf. on Information, Communications and Signal Processing - ICICS, Singapore, pp. 397–401 ( September 1997)
Ekman, P.: The Handbook of Cognition and Emotion. In: Dalgleish, T., Power, T. (eds.), Sussex, U.K, pp. 45–60. John Wiley & Sons, Ltd, Chichester (1999)
Ekman, P., Friesen, W.: Unmasking the Face. Prentice-Hall, Englewood Cliffs (1975)
Ekman, P.: Emotion In the Human Face. Cambridge University Press, Cambridge (1982)
Ekman, P., Campos, J., Davidson, R.J., De Waals, F.: Darwin, Deception, and Facial Expression. In: Emotions Inside Out, vol. 1000. Annals of the New York Academy of Sciences, New York (2003)
Ekman, P., Rosenberg, E.L.: What the Face Reveals: Basic and applied studies of spontaneous expression using the Facial Action Coding System (FACS). Oxford University Press, New York
Ortony, A., Clore, G.L., Collins, A.: The Cognitive Structure of Emotions. Cambridge University Press, Cambridge (1988)
Pantic, M., Rothkrantz, L.J.M.: Automatic Analysis of Facial Expressions: The State of the Art. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(12), 1424–1445 (2000)
Pantic, M., Rothkrantz, L.J.M.: Toward an affect-sensitive multimodal HCI. Proceedings of the IEEE 91(9), 1370–1390 (2003)
Pantic, M., Valstar, M.F., Rademaker, R., Maat, L.: Web-based Database for Facial Expression Analysis. In: Proc. IEEE Int’l Conf. Multmedia and Expo. (ICME 2005), Amsterdam, The Netherlands (July 2005)
Picard, R.W.: Affective Computing. MIT Press, Cambridge (1997)
Picard, R.W.: Affective Computing: Challenges. International Journal of Human-Computer Studies 59(1-2), 55–64 (2003)
Reeves, B., Nass, C.: Social and Natural Interfaces: Theory and Design. CHI Extended Abstracts, pp. 192–193 (1997)
Reeves, B., Nass, C.: The Media Equation: How People Treat Computers, Television, and New Media Like Real People and Places. Cambridge University Press and CSLI, New York
Rosenberg, M.: Conceiving the Self. Basic Books, New York (1979)
Russell, J.A.: Core affect and the psychological construction of emotion. Psychological Review 110, 145–172 (2003)
Russell, J.A.: Is there universal recognition of emotion from facial expression?: A review of the cross-cultural studies. Psychological Bulletin 115, 102–114 (1994)
Stathopoulou, I.-O., Tsihrintzis, G.A.: A neural network-based facial analysis system. In: 5th International Workshop on Image Analysis for Multimedia Interactive Services, Lisboa, Portugal, April 21-23 (2004)
Stathopoulou, I.-O., Tsihrintzis, G.A.: An Improved Neural Network-Based Face Detection and Facial Expression Classification System. In: IEEE International Conference on Systems, Man, and Cybernetics 2004, The Hague, Netherlands, October 10-13 (2004)
Stathopoulou, I.-O., Tsihrintzis, G.A.: Pre-processing and expression classification in low quality face images. In: 5th EURASIP Conference on Speech and Image Processing, Multimedia Communications and Services, Smolenice, Slovak Republic, June 29 – July 2 (2005)
Stathopoulou, I.-O., Tsihrintzis, G.A.: Evaluation of the Discrimination Power of Features Extracted from 2-D and 3-D Facial Images for Facial Expression Analysis. In: 13th European Signal Processing Conference, Antalya, Turkey, September 4-8 (2005)
Stathopoulou, I.-O., Tsihrintzis, G.A.: Detection and Expression Classification Systems for Face Images (FADECS). In: 2005 IEEE Workshop on Signal Processing Systems (SiPS 2005), Athens, Greece, November 2 – 4 (2005)
Stathopoulou, I.-O., Tsihrintzis, G.A.: An Accurate Method for eye detection and feature extraction in face color images. In: 13th International Conference on Signals, Systems, and Image Processing, Budapest, Hungary, September 21-23 (2006)
Stathopoulou, I.-O., Tsihrintzis, G.A.: Facial Expression Classification: Specifying Requirements for an Automated System. In: 10th International Conference on Knowledge-Based & Intelligent Information & Engineering Systems, Bournemouth, United Kingdom, October 9-11 (2006)
Stathopoulou, I.-O., Tsihrintzis, G.A.: NEU-FACES: A Neural Network-based Face Image Analysis System. In: 8th International Conference on Adaptive and Natural Computing Systems, Warsaw, Poland, April 11-14 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Stathopoulou, IO., Tsihrintzis, G.A. (2008). Recent Developments in Automated Inferencing of Emotional State from Face Images. In: Filipe, J., Shishkov, B., Helfert, M., Maciaszek, L.A. (eds) Software and Data Technologies. ICSOFT ENASE 2007 2007. Communications in Computer and Information Science, vol 22. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-88655-6_23
Download citation
DOI: https://doi.org/10.1007/978-3-540-88655-6_23
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-88654-9
Online ISBN: 978-3-540-88655-6
eBook Packages: Computer ScienceComputer Science (R0)