Skip to main content
SpringerLink
Log in

Automatic facial expression recognition: feature extraction and selection

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

In this paper, we investigate feature extraction and feature selection methods as well as classification methods for automatic facial expression recognition (FER) system. The FER system is fully automatic and consists of the following modules: face detection, facial detection, feature extraction, selection of optimal features, and classification. Face detection is based on AdaBoost algorithm and is followed by the extraction of frame with the maximum intensity of emotion using the inter-frame mutual information criterion. The selected frames are then processed to generate characteristic features using different methods including: Gabor filters, log Gabor filter, local binary pattern (LBP) operator, higher-order local autocorrelation (HLAC) and a recent proposed method called HLAC-like features (HLACLF). The most informative features are selected based on both wrapper and filter feature selection methods. Experiments on several facial expression databases show comparisons of different methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Battiti R.: Using mutual information for selecting features in supervised neural net learning. IEEE Trans. Neural Netw. 5, 537–550 (1994)

    Article  Google Scholar 

  2. Chibelushi, C.C., Bourel, F.: Facial expression recognition: A brief tutorial overview. CVonline: On-Line Compendium of Computer Vision, vol. 9, (2003)

  3. Duda R.O., Hart P.E., Stork D.G.: Pattern Classification. Wiley, New York (2001)

    MATH  Google Scholar 

  4. Fasel B., Luettin J.: Automatic facial expression analysis: a survey. Pattern Recognit. 36, 259–275 (2003)

    Article  MATH  Google Scholar 

  5. Field D.J.: Relations between the statistics of natural images and the response properties of cortical cells. J. Opt. Soc. Am. 4, 2379–2394 (1987)

    Article  Google Scholar 

  6. Guyon I., Gunn S., Nikravesh M., Zadeh L.: Feature extraction: foundations and applications. Springer, Verlag (2006)

    MATH  Google Scholar 

  7. Hanchuan P., Fuhui L., Ding C.: Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans. Pattern Anal. Mach. Intell. 27, 1226–1238 (2005)

    Article  Google Scholar 

  8. Kanade, T., Cohn, J.F., Tian, Y.: Comprehensive database for facial expression analysis. In: Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 46–53, Grenoble, France, March (2000)

  9. Kohavi R., John G.H.: Wrapper for feature subset selection. Artif. Intell. 97, 273–324 (1997)

    Article  MATH  Google Scholar 

  10. Lajevardi, S.M., Lech, M.: Averaged Gabor Filter Features for Facial Expression Recognition. In: Computing: Techniques and Applications, 2008. DICTA’08. Digital Image Canberra Canberra, Australia, Dec, pp. 71–76 (2008)

  11. Lajevardi, S.M., Lech, M.: Facial Expression Recognition from Image Sequences Using Optimised Feature Selection. 23rd International Conference on Image and Vision Computing (IVCNZ’08) Christchurch, New Zealand, Nov, pp. 1–6 (2008)

  12. Lajevardi S.M., Hussain Z.M.: Novel higher-order local autocorrelation-like feature extraction methodology for facial expression recognition. IET Image Process. 4, 114–119 (2010)

    Article  Google Scholar 

  13. Lee T.S.: Image representation using 2D Gabor wavelets. IEEE Trans. Pattern Anal. Mach. Intell. 18, 959–971 (1996)

    Article  Google Scholar 

  14. Liu, F. et al.: Facial expression recognition using HLAC features and WPCA. In: Affective Computing and Intelligent Interaction, First International Conference (ACII 2005) Beijing, China, Oct 2005, pp. 88–94 (2005)

  15. Lyons, M., Akamatsu, S., Kamachi, M., Gyoba, J.: Coding facial expressions with Gabor wavelets, in automatic face and gesture recognition, 1998. In: Proceedings. Third IEEE International Conference on Japan, pp. 200–205 (1998)

  16. Mehrabian A.: Communication without words. Psychol. Today 2(4), 53–56 (1968)

    Google Scholar 

  17. Ojala T., Pietikäinen M., Harwood D.: A comparative study of texture measures with classification based on featured distributions. Pattern Recognit. 29, 51–59 (1996)

    Article  Google Scholar 

  18. Otsu, N., Kurita, T.: A new scheme for practical flexible and intelligent vision systems. In: Proceedings of the IAPR Workshop on Computer Vision, pp. 431–435 (1988)

  19. Rish, I.: An empirical study of the naive Bayes classifier. In: IJCAI Workshop on Empirical Methods in Artificial Intelligence, pp. 41–46 (2001)

  20. Srinivas M., Patnik L.M.: Genetic algorithms: a Survey. IEEE Comput. Soc. Press 27, 17–26 (1994)

    Google Scholar 

  21. Tian, Y.L., Kanade, T., Cohn, J.F.: Facial Expression Analysis. Hand Book of Face Recognition, pp. 247–275 (2005)

  22. Toyoda, T., Hasegawa, O.: Texture classification using extended higher order local autocorrelation features. In: Proceedings of the 4th International Workshop on Texture Analysis and Synthesis, pp. 131–136 (2005)

  23. Viola P., Jones M.J.: Robust real-time face detection. Int. J. Comput. Vis. 57, 137–154 (2004)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electrical and Computer Engineering, RMIT University, Melbourne, VIC, Australia

    Seyed Mehdi Lajevardi & Zahir M. Hussain

Authors
  1. Seyed Mehdi Lajevardi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zahir M. Hussain
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Seyed Mehdi Lajevardi.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lajevardi, S.M., Hussain, Z.M. Automatic facial expression recognition: feature extraction and selection. SIViP 6, 159–169 (2012). https://doi.org/10.1007/s11760-010-0177-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-010-0177-5

Keywords

Search

Navigation