Skip to main content

Advertisement

SpringerLink
Log in

An exposition of facial expression recognition techniques

  • Review
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Automated facial expression recognition schemes have been a subject of interest ever since the inception of its idea. The initial efforts required supervised and controlled environments to get realistic and convincing experimental results. Various improved methods have been suggested to detect the facial expressions of a human. Some of the popularly used methods are automatic expression recognition system (AERS), graph-preserving sparse nonnegative matrix factorization (GSNMF) algorithm, two-phase test sample representation (TPTSR) technique and temporal template method. The current state-of-the-art techniques are able to detect facial expressions in difficult and obscured environments. Advancements in this technique have helped researchers to use video sequences as images to detect expressions. A single image is extracted from a video sequence, and elaborate techniques are applied to detect the expression. This article endeavors to discuss these intricate techniques and critically analyzes them. It helps the reader to understand the contemporary problems in facial recognition systems and how various researchers have employed different models to overcome these challenges. In this paper, the performance of various techniques such as AERS, GSNMF algorithm, TPTSR, performance-based character animation, temporal template method, feature extractions using Gabor filter and image sequencing-based methods has been scrutinized in terms of their efficiency, accuracy and effectiveness. The efficiency and accuracy of the techniques have been compared using various benchmarks such as leave one out, cross-validation and receiver operating characteristics. Each technique bears its own advantages and disadvantages in terms of accuracy and efficiency. The highest accuracy rate is exhibited by the technique using canny edge detection algorithm and chamfer image method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26

Similar content being viewed by others

References

  1. Ahad MAR, Tan JK, Kim H, Ishikawa S (2012) Motion history image: its variants and applications. Mach Vis Appl 23(2):255–281. doi:10.1007/s00138-010-0298-4

  2. Alirezaee S et al (2006) An efficient algorithm for face localization. Int J Inf Technol 12(7):30–36

  3. Bartlett MS, Whitehill J (2010) Automated facial expression measurement: recent applications to basic research in human behavior, learning, and education. In: Calder A, Rhodes G, Haxby JV (eds) Handbook of face perception, vol 7. Oxford University Press, San Diego

    Google Scholar 

  4. Brunelli R, Poggio T (1993) Face recognition: features vs templates. IEEE Trans Pattern Anal Mach Intell 15(10):1042–1052

  5. Carlson NR (1986) Physiology of behavior. Allyn & Bacon, Boston

  6. Chakrabarti D, Dutta D (2013) Facial expression recognition using eigenspaces. In: International conference on computational intelligence: modeling techniques and applications, India

  7. Dailey MN, Joyce C, Lyons MJ, Kamachi M, Ishi H, Gyoba J, Cottrell GW (2010) Evidence and a computational explanation of cultural differences in facial expression recognition. Emotion 10(6):874–893

    Article  Google Scholar 

  8. Deng HB, Jin LW, Zhen LX, Huang JC (2005) A new facial expression recognition method based on local Gabor filter bank and PCA plus LDA. Int J Inf Technol 86–96

  9. Dornaika F, Raducanu B (2009) Facial expression recognition for HCI applications. Encycl Artif Intell 2(2009):625–631

  10. Ekman EL (1993) Facial expression and emotion. Encycl Neurosci (Suppl. 3):51–52

  11. Ekman P (2006) Darwin and facial expression: a century of research in review. Ishk, San Jose, CA

  12. Ekman P, Rolls ET, Perrett DI, Ellis HD (1992) Facial expressions of emotion: an old controversy and new findings [and discussion]. Philos Trans R Soc B Biol Sci 335(1273):63–69

    Article  Google Scholar 

  13. Fasel B, Luettin J (2003) Automatic facial expression analysis: a survey. Pattern Recognit 259–275

  14. Fogel A, Thelen E (1987) Development of early expressive and communicative action: reinterpreting the evidence from a dynamic systems perspective. Dev Psychol 747–761

  15. Fridlund AJ (1994) Human facial expression: an evolutionary view. Academic Press, San Deigo

  16. Frith C (2009) Role of facial expressions in social interactions. Comput Emot Man Mach 3453–3458

  17. Jack RE, Garrod OGB, Yu H, Caldara R, Schyns PG (2012) Facial expressions of emotion are not culturally universal. Proc Natl Acad Sci 109(19):7241–7244

    Article  Google Scholar 

  18. Kanade T, Yamada A (2003) Multi-subregion based probabilistic approach toward poseinvariant face recognition. In: Proceedings of the 2003 IEEE international symposium on computational intelligence in robotics and automation, 2003, vol 2. IEEE, pp 954–959

  19. Kanade T, Cohn JF, Tian Y (2000) Comprehensive database for facial expression analysis. In: Proceedings of the fourth IEEE international conference on automatic face and gesture recognition. IEEE

  20. Kimura MY (2010) Facial expression recognition and its degree estimation. The Kansai Electric Power, Osaka

    Google Scholar 

  21. Kumbhar M, Jadhav A, Patil M (2012) Facial expression recognition based on image feature. Int J Comput Commun Eng 1(2):117

  22. Lucey PC (2010) The extended Cohn–Kanade dataset (ck+): a complete dataset for action unit and emotion-specified expression. In: IEEE Computer Society conference on computer vision. IEEE

  23. Lyons MJ, Akamatsu S, Kamachi M, Gyoba J (1998) Coding facial expressions with Gabor wavelets. In: Proceedings of the third IEEE international conference on automatic face and gesture recognition, IEEE Computer Society, Nara, Japan, 14–16 April 1998, pp 200–205

  24. Malatesta CZ, Culver C, Tesman JR, Shepard B, Fogel A, Reimers M, Zivin G (1989) The development of emotion expression during the first two years of life. Monogr Soc Res Child Dev 1–136

  25. Matsumoto D, Ekman P (2008) Facial expression analysis. Scholarpedia 3(5):4237

  26. Mehrabian A (1981) Silent messages: implicit communication of emotions and attitudes. Wadsworth, Belmont, CA

    Google Scholar 

  27. Meshgini S, Aghagolzadeh A, Seyedarabi H (2012) Face recognition using Gabor filter bank, kernel principle component analysis and support vector machine. Int J Comput Theory Eng 4(5):767

  28. Moore S, Ong EJ, Bowden R (2010) Facial expression recognition using spatiotemporal boosted discriminatory classifiers. Int Conf Image Anal Recognit 1(1):405–414

    Google Scholar 

  29. Pantic M, Rothkrantz LJM (2000) Automatic analysis of facial expressions: the state art. IEEE Trans Pattern Analysis and Mach Intell 22(12):1424–1445

    Article  Google Scholar 

  30. Pitas KS (1996) Face localization and facial feature extraction based on shape and color information. In: International conference on image processing, Lausanne, pp 483–486

  31. Samaria FS (1994) Parameterisation of a stochastic model for human face identification. In: Proceedings of the second IEEE workshop on applications of computer vision. IEEE, pp 138–142

  32. Shergill GS et al (2008) Computerized sales assistants: the application of computer technology to measure consumer interest—a conceptual framework. J Electron Commer Res 9(2):176

  33. Tian D (2013) A review on image feature extraction and representation techniques. Int J Multimed Ubiquitous Eng 385–396

  34. Trevarthen C, Aitken CT (2001) Infant intersubjectivity: research, theory, and clinical applications. J Child Psychol Psychiatry 3–48

  35. Trujillo L, Olague G, Hammoud R, Hernandez B (2005) Automatic feature localization in thermal images for facial expression recognition. In: IEEE Computer Society conference on computer vision and pattern recognition (CVPR ‘05)-workshops, pp 14–25. IEEE

  36. Valstar MF, Jiang B, Mehu M, Pantic M, Scherer K (2011) The first facial expression recognition and analysis challenge. In: 9th IEEE international conference on face and gesture recognition, Santa Barbara, CA, USA

  37. Weise T, Bouaziz S, Li H, Pauly M (2011) Realtime performance-based facial animation. ACM Trans Graph 30(4):9

    Article  Google Scholar 

  38. Xu Y et al (2011) A two-phase test sample sparse representation method for use with face recognition. IEEE Trans Circuits Syst Video Technol 21(9):1255–1262

  39. Zhi R et al (2011) Graph-preserving sparse nonnegative matrix factorization with application to facial expression recognition. IEEE Trans Syst Man Cybern B 41(1):38–52

Download references

Author information

Authors and Affiliations

  1. School of Systems and Technology, University of Management and Technology, Lahore, Pakistan

    Somia Saeed & Yaser Daanial Khan

  2. Department of Mathematics, University of Punjab, Lahore, Pakistan

    M. Khalid Mahmood

Authors
  1. Somia Saeed
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. Khalid Mahmood
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yaser Daanial Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yaser Daanial Khan.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interest regarding this publication.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Saeed, S., Mahmood, M.K. & Khan, Y.D. An exposition of facial expression recognition techniques. Neural Comput & Applic 29, 425–443 (2018). https://doi.org/10.1007/s00521-016-2522-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-016-2522-2

Keywords

Search

Navigation