Skip to main content
SpringerLink
Log in

AutoFER: PCA and PSO based automatic facial emotion recognition

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Automatic emotion recognition is a critical part of human-machine interactions. Reflection of emotions and to develop its understanding is crucial to provide dealings across human beings and machine frameworks. This work determines an automatic system that distinguishes different emotions connoted on the face. The framework is deliberated to apply the hybridization of feature extraction and optimization using PCA and PSO, respectively, to accomplish a high precision rate. PCA is used to get high-quality feature vectors for each category of emotion. Swarm intelligence, optimization is applied to get an optimized feature vector which is essential for classifying the features in the testing phase. For exploratory work, the authors have considered the Japanese Female Facial Expression (JAFFE) dataset. A maximum classification rate of 94.97% is achieved with the proposed technique. The proposed framework execution is assessed in terms of the false rejection rate, false acceptance rate, and accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Agrawal D, Dubey SR, Jalal AS (2014) Emotion recognition from facial expressions based on multi-level classification. Int J Comput Vis Robot 4(4):365–389

    Article  Google Scholar 

  2. Bastanfard A, Bastanfard O, Takahashi H, Nakajima M (2004) Toward anthropometrics simulation of face rejuvenation and skin cosmetic. Comput Anim Virtual Worlds 15(3–4):347–352

    Article  Google Scholar 

  3. Bastanfard A, Takahashi H, Nakajima M (2004) Toward E-appearance of human face and hair by age, expression and rejuvenation. In: 2004 International conference on Cyberworlds, pp 306–311. https://doi.org/10.1109/CW.2004.65

  4. Belhumeur PN, Hespanha JP, Kriegman DJ (1997) Eigenfaces vs. Fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720

    Article  Google Scholar 

  5. Black MJ, Yacoob Y (1997) Recognizing facial expressions in image sequences using local parameterized models of image motion. Int J Comput Vis 25(1):23–48

    Article  Google Scholar 

  6. Chernykh V, Prikhodko P (2017) Emotion recognition from speech with recurrent neural networks arXiv preprint arXiv:170108071

  7. Chiranjeevi P, Gopalakrishnan V, Moogi P (2015) Neutral face classification using personalized appearance models for fast and robust emotion detection. IEEE Trans Image Process 24:2701–2711

    Article  MathSciNet  Google Scholar 

  8. Draper BA, Baek K, Bartlett MS, Beveridge JR (2003) Recognizing faces with PCA and ICA. Comput Vis Image Underst 91(1–2):115–137

    Article  Google Scholar 

  9. Jain N, Kumar S, Kumar A, Shamsolmoali P, Zareapoor M (2018) Hybrid deep neural networks for face emotion recognition. Pattern Recogn Lett 115:101–106

    Article  Google Scholar 

  10. Khorrami P, Le Paine T, Brady K, Dagli C, Huang TS (2016) How deep neural networks can improve emotion recognition on video data. In: 2016 IEEE international conference on image processing (ICIP), pp 619–623. https://doi.org/10.1109/ICIP.2016.7532431

  11. Krestinskaya O, James AP (2017) Facial emotion recognition using min-max similarity classifier. In: International Conference on Advances in Computing, Communications and Informatics (ICACCI), pp 752–758

    Google Scholar 

  12. Kumar S, Singh S, Kumar J (2018) Automatic live facial expression detection using genetic algorithm with haar wavelet features and SVM. Wirel Pers Commun 103(3):2435–2453

    Article  Google Scholar 

  13. Lopes AT, de Aguiar E, De Souza AF, Oliveira-Santos T (2017) Facial expression recognition with convolutional neural networks: coping with few data and the training sample order. Pattern Recogn 61:610–628

    Article  Google Scholar 

  14. Lyons MJ, Akamatsu S, Kamachi M, Gyoba J, Budynek J (1998) The Japanese female facial expression (JAFFE) database. In: Proceedings of third international conference on automatic face and gesture recognition, pp 14–16. https://doi.org/10.5281/zenodo.3451524

  15. Moore S, Bowden R (2011) Local binary patterns for multi-view facial expression recognition. Comput Vis Image Underst 115(4):541–558

    Article  Google Scholar 

  16. Murtaza M, Sharif M, AbdullahYasmin M, Ahmad T (2019) Facial expression detection using six facial expressions hexagon (SFEH) model. In: 2019 IEEE 9th annual computing and communication workshop and conference (CCWC), pp 0190–0195

    Chapter  Google Scholar 

  17. Saeed A, Niese R, Elzobi M (2014) Frame-based facial expression recognition using geometric features. Adv Hum-Comput Interact 1–13. https://doi.org/10.1155/2014/408953

  18. Sánchez A, Ruiz JV, Moreno AB, Montemayor AS, Hernández J, Pantrigo JJ (2011) Differential optical flow applied to automatic facial expression recognition. Neurocomputing 74(8):1272–1282

    Article  Google Scholar 

  19. Shang YC, Yan-Feng S, Bao-Cai Y (2009) A novel hybrid approach based on sub pattern technique and extended 2DPCA for color face recognition. In: Proceedings of the 11th IEEE international symposium on multimedia, pp 630–634. https://doi.org/10.1109/ISM.2009.24

  20. Torres L, Reutter J, Lorente L (1999) The importance of the color information in face recognition in proceedings of International Conference on image Processing, 3:627–631. https://doi.org/10.1109/ICIP.1999.817191

  21. Virrey RA, Liyanage CDS, Petra MIBPH, Abas PE (2019) Visual data of facial expressions for automatic pain detection. J Vis Commun Image Represent 61:209–217

    Article  Google Scholar 

  22. Yang J, Zhang D, Frangi AF, Yang JY (2004) Two-dimensional PCA: a new approach to appearance-based face representation and recognition. IEEE Trans Pattern Anal Mach Intell 26(1):131–137

    Article  Google Scholar 

  23. Zhang T, Zheng W, Cui Z, Zong Y, Li Y (2018) Spatial–temporal recurrent neural network for emotion recognition. IEEE Trans Cybern 49(3):839–847

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computational Sciences, Maharaja Ranjit Singh Punjab Technical University, Bathinda, Punjab, India

    Malika Arora & Munish Kumar

Authors
  1. Malika Arora
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Munish Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Munish Kumar.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Significance of the Work

A framework proposed in this article will be helpful for the automatic facial emotion recognition system based on facial expression. A hybridization of feature extraction and optimization using PCA and PSO has been employed in this work to achieve acceptable recognition results.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Arora, M., Kumar, M. AutoFER: PCA and PSO based automatic facial emotion recognition. Multimed Tools Appl 80, 3039–3049 (2021). https://doi.org/10.1007/s11042-020-09726-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09726-4

Keywords

Search

Navigation