Skip to main content
Springer Nature Link
Log in

Combining Features for Recognizing Emotional Facial Expressions in Static Images

  • Conference paper
Verbal and Nonverbal Features of Human-Human and Human-Machine Interaction

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 5042))

Abstract

This work approaches the problem of recognizing emotional facial expressions in static images focusing on three preprocessing techniques for feature extraction such as Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), and Gabor filters. These methods are commonly used for face recognition and the novelty consists in combining features provided by them in order to improve the performance of an automatic procedure for recognizing emotional facial expressions. Testing and recognition accuracy were performed on the Japanese Female Facial Expression (JAFFE) database using a Multi-Layer Perceptron (MLP) Neural Network as classifier. The best classification accuracy on variations of facial expressions included in the training set was obtained combining PCA and LDA features (93% of correct recognition rate), whereas, combining PCA, LDA and Gabor filter features the net gave 94% of correct classification on facial expressions of subjects not included in the training set.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Abbound, B., Davoine, F.: Facial Expression Recognition and Synthesis Based on an Appereance Model. Signal Processing: Image Communication 19, 723–740 (2004)

    Google Scholar 

  2. La Barre, W.: The Cultural basis of Emotions Gnd gestures. Journal of Personality 16, 49–68 (1947)

    Article  Google Scholar 

  3. Birdwhistell, R.: Kinesies and Context. University of Pennsylvania Press, Philadelphia (1970)

    Google Scholar 

  4. Cohen, I., Cozman, F.G., Sebe, N., Cirelo, M.C., Huang, T.S.: Semisupervised Learning of Classifiers: Theory, Algorithms and their Application to Human-Computer Interaction. IEEE Transactions on PAMI 26, 1553–1567 (2004)

    Article  Google Scholar 

  5. Darwin, C.: The Expression of the Emotions in Man and Animals. J. Murray, London (1872)

    Book  Google Scholar 

  6. Ekman, P., Friesen, W.: Constants across Cultures in the Face and Emotion. Journal of Personality and Social Psychology (1971)

    Google Scholar 

  7. Ekman, P., Friesen, W.: Emotional Facial Action Coding System. Unpublished manual (1978)

    Google Scholar 

  8. Ekman, P., Friesen, W.V.: Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press, Palo Alto (1978)

    Google Scholar 

  9. Ekman, P.: The Argument and Evidence about Universals in Facial Expressions of Emotion. In: Wagner, H., Manstead, A. (eds.) Handbook of Social Psychophysiology, pp. 143–164. Wiley, Chichester (1989)

    Google Scholar 

  10. Ekman, P.: Facial Expression of Emotion: New Findings. New Questions. Psychological Science 3, 34–38 (1992)

    Article  Google Scholar 

  11. Fasel, B., Luettin, J.: Automatic Facial Expression Analysis: A Survey. Pattern Recognition 36(1), 259–275 (2003)

    Article  MATH  Google Scholar 

  12. Fisher, R.A.: The Statistical Utilization of Multiple Measurements. Annali of Eugenics 8, 376–386 (1938)

    Article  MATH  Google Scholar 

  13. Fridlund, A.J.: The New Ethnology of Human Facial Expressions. In: Russell, J.A., Fernandez-Dols, J. (eds.) The Psychology of Facial Expressions, pp. 103–129. Cambridge University Press, Cambridge (1997)

    Chapter  Google Scholar 

  14. Hong, H., Neven, H., von der Malsburg, C.: Online Facial Expression Recognition Based on Personalized Galleries. In: Proceedings of the International Conference on Automatic Face and Gesture Recognition, pp. 354–359 (1998)

    Google Scholar 

  15. Huang, C.L., Huang, Y.M.: Facial Expression Recognition Using Model-Based Feature Extraction and Action Parameters Classification. Journal of Visual Communication and Image Representation 8(3), 278–290 (1997)

    Article  Google Scholar 

  16. Huang, S.H., Wu, Q.J., Lai, S.H.: Improved AdaBoost-based Image Retrieval with Relevance Feedback Via Paired Feature Learning. Multimedia Systems 12, 14–26 (2006)

    Article  Google Scholar 

  17. Izard, C.E., Dougherty, L.M., Hembree, E.A.: A System for Identifying Affect Expressions by Holistic Judgments. Unpublished manuscript. Available from Instructional Resource Center, University of Delaware (1983)

    Google Scholar 

  18. Izard, C.E.: Innate and Universal Facial Expressions: Evidence from Developmental and Cross-Cultural Research. Psychological Bulletin 115, 288–299 (1994)

    Article  Google Scholar 

  19. Jollife, I.T.: Principal Component Analysis, 2nd edn. Springer, New York (2002)

    Google Scholar 

  20. Kamachi, M., Lyons, M., Gyoba, J.: Japanese Female Facial Expression Database, Psychology Department in Kyushu University, http://www.kasrl.org/jaffe.html

  21. Klinerberg, O.: Emotional Expression in Chinese Literature. Journal of Abnormal and Social Psychology 33, 517–520 (1938)

    Article  Google Scholar 

  22. Lee, Y., Kim, I., Shim, J., Marshall, D.: 3D Facial Image Recognition Using a Nose Volume and Curvature Based Eigenface. In: Kim, M.-S., Shimada, K. (eds.) GMP 2006. LNCS, vol. 4077, pp. 616–622. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  23. Lyons, M.J., Budynek, J., Akamatsu, S.: Automatic Classification of Single Facial Images. IEEE Transactions on Pattern Analysis and Machine Intelligence 21, 1357–1362 (1999)

    Article  Google Scholar 

  24. Martýnez, A.M.: Recognition of Partially Occluded and/or Imprecisely Localized Faces Using a Probabilistic Approach. In: Proceeding of the International Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 712–717 (2000)

    Google Scholar 

  25. Martýnez, A.M.: PCA versus LDA. IEEE Transactions on Pattern Analysis and Machine Intelligence 23(2), 228–233 (2001)

    Article  Google Scholar 

  26. Moghaddam, B., Pentland, A.: Probabilistic Visual Learning for Object Representation. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(7), 696–710 (1997)

    Article  Google Scholar 

  27. Moon, H., Phillips, P.J.: Analysis of PCA-based Face Recognition Algorithms. In: Bowyer, K.J., Phillips, P.J. (eds.) Empirical Evaluation Techniques in Computer Vision. IEEE Computer Soceity, Los Alamitos (1998)

    Google Scholar 

  28. Pantic, M., Rothkrantz, J.M.: Automatic Analysis of Facial Expression: The State of the Art. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(12), 1424–1445 (2000)

    Article  Google Scholar 

  29. Pantic, M., Rothkrantz, J.M.: Expert System for Automatic Analysis of Facial Expression. Image and Vision Computing Journal 18(11), 881–905 (2000)

    Article  Google Scholar 

  30. Papageorgiou, C., Oren, M., Poggio, T.: A General Framework for Object Detection. In: International Conference on Computer Vision, pp. 992–998 (1998)

    Google Scholar 

  31. Petkov, N., Wieling, M.B.: Gabor Filtering Augmented with Surround Inhibition for Improved Contour Detection by Texture Suppression. Perception 33, 68c (2004)

    Google Scholar 

  32. Phillips, P.J., Moon, H., Rauss, P., Rizvi, S.A.: The FERET Evaluation Methodology for Face-Recognition Algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence 22, 1090–1104 (2000)

    Article  Google Scholar 

  33. Phillips, P.J., Flynn, P.J., Scruggs, T., Bowyer, K.W.: Overview of the Face Recognition Grand Challenge. In: Proc. IEEE Conf. Computer Vision and Pattern Recognition (2005)

    Google Scholar 

  34. Roth, D., Yang, M., Ahuja, N.: A SNoW-Based Face Detector. Advances in Neural Information Processing Systems, 855–861 (2000)

    Google Scholar 

  35. Russell, J.A.: A Circumplex Model of Affect. Journal of Personality and Social Psychology 39, 1161–1171 (1980)

    Article  Google Scholar 

  36. Ryu, H., Chun, S.S., Sull, S.: Multiple Classifiers Approach for Computational Efficiency in Multi-scale Search Based Face Detection. In: Jiao, L., Wang, L., Gao, X.-b., Liu, J., Wu, F. (eds.) ICNC 2006. LNCS, vol. 4221. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  37. Samaria, F., Harter, A.: The ORL Database of Faces, AT&T Laboratories Cambridge University, http://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html

  38. Schneiderman, H., Kanade, T.: A Statistical Method for 3D Object Detection Applied to Faces and Cars. In: International Conference on Computer and Pattern Recognition, vol. 1, pp. 746–751 (2000)

    Google Scholar 

  39. Schlosberg, H.: Three Dimensions of Emotion. The Psychological Review 61(2), 81–88 (1953)

    Article  Google Scholar 

  40. Simoncelli, E.P., Olshausen, B.A.: Natural Image Statistics and Neural Representation. Annual Review of Neuroscience 24, 1193–1216 (2001)

    Article  Google Scholar 

  41. Sung, K., Poggio, T.: Example-Based Learning for View-Based Face Detection. IEEE Transaction on Pattern Analyses and Machine Intelligence 20, 39–51 (1998)

    Article  Google Scholar 

  42. Tomkins, S.S.: Affect Theory. In: Scherer, K.R., Ekman, P. (eds.) Approaches to Emotion, pp. 163–196. Erlbaum, Hillsdale (1984)

    Google Scholar 

  43. Turk, M., Pentland, A.: Face Recognition Using Eigenfaces. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 586–591 (1991)

    Google Scholar 

  44. Viola, A.P., Jones, M.J.: Robust Real-Time Face Detection. International Journal of Computer Vision 57(2), 137–154 (2004)

    Article  Google Scholar 

  45. White, G.M.: Emotion Inside Out the Anthropology of Affect. In: Haviland, M., Lewis, J.M. (eds.) Handbook of Emotion, pp. 29–40. Guilford Press, New York (1993)

    Google Scholar 

  46. Zhao, J., Kearney, G.: Classifying Facial Emotions by Backpropagation Neural Networks with Fuzzy Inputs. In: Proceedings of the International Conference on Neural Information Processing, vol. 1, pp. 454–457 (1996)

    Google Scholar 

  47. Zhao, W., Chellappa, R., Phillips, P.J., Rosenfeld, A.: Face Recognition: A Literature Survey. ACM, Computing Surveys 35(4), 399–458 (2003)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Brno University of Technology, Czech Republic

    Jiří Přinosil & Zdeněk Smékal

  2. Department of Psychology and IIASS, Second University of Naples, Italy

    Anna Esposito

Authors
  1. Jiří Přinosil
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zdeněk Smékal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anna Esposito
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Psychology, Second University of Naples, and IIASS, Via Pellegrino 19, 84019, Vietri sul Mare (SA), Italy

    Anna Esposito

  2. ATRC Center, Wright State University, Dayton, OH, USA

    Nikolaos G. Bourbakis

  3. Human Computer Interaction Group, University of Patras, Rio Patras, Greece

    Nikolaos Avouris

  4. Department of Computer Engineering, University of Patras, Patras, Greece

    Ioannis Hatzilygeroudis

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Přinosil, J., Smékal, Z., Esposito, A. (2008). Combining Features for Recognizing Emotional Facial Expressions in Static Images. In: Esposito, A., Bourbakis, N.G., Avouris, N., Hatzilygeroudis, I. (eds) Verbal and Nonverbal Features of Human-Human and Human-Machine Interaction. Lecture Notes in Computer Science(), vol 5042. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-70872-8_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-70872-8_5

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-70871-1

  • Online ISBN: 978-3-540-70872-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics