Skip to main content
Springer Nature Link
Log in

A Simple Yet Effective Convolutional Neural Network Model to Classify Facial Expressions

  • Conference paper
  • First Online:
Modelling and Implementation of Complex Systems (MISC 2020)

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 156))

Abstract

Facial Expression Recognition (FER) is a form of nonverbal communication; it translates the internal and emotional state of a human being by changing one or several facial muscles. Automated classification of facial expressions has known a great progress over the last decade; we observed the appearance of new methods based on Deep Learning (DL) instead of traditional classification methods. In this paper we propose an improved method based on Convolutional Neural Networks (CNN) that responds to the problem of classification of the six basic emotions(anger, disgust, fear, happy, sad and surprise) plus the neutral case. We validated our model on three public databases and we achieved better results than the state-of-the-art: CK+ 88,23%, JAFFE 86.24%, KDEF 82.38%. Our accuracies out perform results from recently proposed traditional methods as well as DL based methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Ahonen, T., Hadid, A., Pietikainen, M.: Face description with local binary patterns: application to face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 12, 2037–2041 (2006)

    Article  Google Scholar 

  2. Chakraborty, A., Konar, A., Chakraborty, U.K., Chatterjee, A.: Emotion recognition from facial expressions and its control using fuzzy logic. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 39(4), 726–743 (2009)

    Article  Google Scholar 

  3. Cohen, I., Sebe, N., Gozman, F., Cirelo, M.C., Huang, T.S.: Learning Bayesian network classifiers for facial expression recognition both labeled and unlabeled data. In: Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, p. I. IEEE (2003)

    Google Scholar 

  4. Ekman, P., Friesen, W.V.: Constants across cultures in the face and emotion. J. Pers. Soc. Psychol. 17(2), 124 (1971)

    Article  Google Scholar 

  5. Eng, S., Ali, H., Cheah, A., Chong, Y.: Facial expression recognition in JAFFE and KDEF datasets using histogram of oriented gradients and support vector machine. In: IOP Conference Series: Materials Science and Engineering, vol. 705, p. 012031. IOP Publishing (2019)

    Google Scholar 

  6. Ghaffar, F.: Facial emotions recognition using convolutional neural net. arXiv preprint arXiv:2001.01456 (2020)

  7. Ghimire, D., Lee, J.: Geometric feature-based facial expression recognition in image sequences using multi-class adaboost and support vector machines. Sensors 13(6), 7714–7734 (2013)

    Article  Google Scholar 

  8. Goeleven, E., De Raedt, R., Leyman, L., Verschuere, B.: The Karolinska directed emotional faces: a validation study. Cogn. Emot. 22(6), 1094–1118 (2008)

    Article  Google Scholar 

  9. Happy, S., Routray, A.: Automatic facial expression recognition using features of salient facial patches. IEEE Trans. Affect. Comput. 6(1), 1–12 (2014)

    Article  Google Scholar 

  10. Harms, M.B., Martin, A., Wallace, G.L.: Facial emotion recognition in autism spectrum disorders: a review of behavioral and neuroimaging studies. Neuropsychol. Rev. 20(3), 290–322 (2010)

    Article  Google Scholar 

  11. Jafri, R., Arabnia, H.R.: A survey of face recognition techniques. J. Inf. Process. Syst. 5(2), 41–68 (2009)

    Article  Google Scholar 

  12. Kobayashi, H., Hara, F.: Recognition of six basic facial expression and their strength by neural network. In: 1992 Proceedings of the IEEE International Workshop on Robot and Human Communication, pp. 381–386. IEEE (1992)

    Google Scholar 

  13. Koelstra, S., Pantic, M., Patras, I.: A dynamic texture-based approach to recognition of facial actions and their temporal models. IEEE Trans. Pattern Anal. Mach. Intell. 32(11), 1940–1954 (2010)

    Article  Google Scholar 

  14. Kristensen, R.L., Tan, Z.H., Ma, Z., Guo, J.: Binary pattern flavored feature extractors for facial expression recognition: an overview. In: 2015 38th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), pp. 1131–1137. IEEE (2015)

    Google Scholar 

  15. Kumari, J., Rajesh, R., Pooja, K.: Facial expression recognition: a survey. Procedia Comput. Sci. 58, 486–491 (2015)

    Article  Google Scholar 

  16. Lawrence, S., Giles, C.L., Tsoi, A.C., Back, A.D.: Face recognition: a convolutional neural-network approach. IEEE Trans. Neural Netw. 8(1), 98–113 (1997)

    Article  Google Scholar 

  17. Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended Cohn-Kanade dataset (CK+): a complete dataset for action unit and emotion-specified expression. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops, pp. 94–101. IEEE (2010)

    Google Scholar 

  18. Lucey, P., et al.: Automatically detecting pain in video through facial action units. IEEE Trans. Syst. Man Cybern. Part B 41(3), 664–674 (2010)

    Article  Google Scholar 

  19. Melaugh, R., Siddique, N., Coleman, S., Yogarajah, P.: Facial expression recognition on partial facial sections. In: 2019 11th International Symposium on Image and Signal Processing and Analysis (ISPA), pp. 193–197. IEEE (2019)

    Google Scholar 

  20. Michael, J., Lyons, M.K., Gyoba, J.: Japanese female facial expressions (JAFFE). Database of digital images (1997)

    Google Scholar 

  21. Sajjad, M., Nasir, M., Ullah, F.U.M., Muhammad, K., Sangaiah, A.K., Baik, S.W.: Raspberry pi assisted facial expression recognition framework for smart security in law-enforcement services. Inf. Sci. 479, 416–431 (2019)

    Article  Google Scholar 

  22. Sandbach, G., Zafeiriou, S., Pantic, M., Yin, L.: Static and dynamic 3d facial expression recognition: a comprehensive survey. Image Vis. Comput. 30(10), 683–697 (2012)

    Article  Google Scholar 

  23. Shan, K., Guo, J., You, W., Lu, D., Bie, R.: Automatic facial expression recognition based on a deep convolutional-neural-network structure. In: 2017 IEEE 15th International Conference on Software Engineering Research, Management and Applications (SERA), pp. 123–128. IEEE (2017)

    Google Scholar 

  24. Sprengelmeyer, R.: Facial expression recognition in people with medicated and unmedicated Parkinson’s disease. Neuropsychologia 41(8), 1047–1057 (2003)

    Article  Google Scholar 

  25. Tian, Y., Kanade, T., Cohn, J.F.: Facial expression recognition. In: Handbook of Face Recognition, pp. 487–519. Springer (2011)

    Google Scholar 

  26. Valstar, M.F., et al.: Fera 2015-second facial expression recognition and analysis challenge. In: 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), vol. 6, pp. 1–8. IEEE (2015)

    Google Scholar 

  27. Valstar, M.F., Jiang, B., Mehu, M., Pantic, M., Scherer, K.: The first facial expression recognition and analysis challenge. In: Face and Gesture 2011, pp. 921–926. IEEE (2011)

    Google Scholar 

  28. Yang, J., Zhang, D., Frangi, A.F., Yang, J.: Two-dimensional PCA a new approach to appearance-based face representation and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 26(1), 131–137 (2004)

    Article  Google Scholar 

  29. Zhang, L., Tjondronegoro, D.: Facial expression recognition using facial movement features. IEEE Trans. Affect. Comput. 2(4), 219–229 (2011)

    Article  Google Scholar 

  30. Zhang, Y.D.: Facial emotion recognition based on biorthogonal wavelet entropy, fuzzy support vector machine, and stratified cross validation. IEEE Access 4, 8375–8385 (2016)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Ferhat Abbas Setif1, Setif, Algeria

    Meriem Sari & Abdelouahab Moussaoui

  2. Center for Machine Learning and Signal Analysis, University of Oulu, Oulu, Finland

    Abdenour Hadid

Authors
  1. Meriem Sari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abdelouahab Moussaoui
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Abdenour Hadid
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Meriem Sari .

Editor information

Editors and Affiliations

  1. Faculty of New Information and Communication Technologies, University of Constantine 2, Constantine, Algeria

    Salim Chikhi

  2. Faculty of Technology, University of Saida, Saida, Algeria

    Abdelmalek Amine

  3. Faculty of New Information and Communication Technologies, University of Constantine 2, Constantine, Algeria

    Allaoua Chaoui

  4. Faculty of New Information and Communication Technologies, University of Constantine 2, Constantine, Algeria

    Djamel Eddine Saidouni

  5. Faculty of Science and Technology, Department of Mathematics and Computer Science, University of El Oued, El-Oued, Algeria

    Mohamed Khireddine Kholladi

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sari, M., Moussaoui, A., Hadid, A. (2021). A Simple Yet Effective Convolutional Neural Network Model to Classify Facial Expressions. In: Chikhi, S., Amine, A., Chaoui, A., Saidouni, D., Kholladi, M. (eds) Modelling and Implementation of Complex Systems. MISC 2020. Lecture Notes in Networks and Systems, vol 156. Springer, Cham. https://doi.org/10.1007/978-3-030-58861-8_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-58861-8_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-58860-1

  • Online ISBN: 978-3-030-58861-8

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics