Abstract
We present an automated new approach for facial expression recognition of seven emotions. Three types of texture features (HOG, D-SURF and LBP) from static images are combined, and the resulting features are classified using random forests. We achieve better than state-of-the-art accuracies using multiple texture feature descriptors. The use of random forests allows identification of the most important feature types and locations for emotion classification. Regions around the eyes, forehead, sides of the nose and mouth are found to be most significant.
We introduce the “Emotional Faces in the Wild” dataset (eLFW), a citizen-labelling of 1310 faces from the Labelled Faces in the Wild data. Like people, machine classification of these and the Karolinska Directed Emotional Faces data obtained from actors; poorest results are obtained in distinguishing the sad, angry and fearful emotions. We describe a new weighted voting algorithm, in which the weighted predictions of classifiers trained on pairs of classes are combined with the weights learned using an evolutionary algorithm. This method yields superior results, particularly for the hard-to-distinguish emotions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
SOWN, M.: A preliminary note on pattern recognition of facial emotional expression. In: The 4th International Joint Conferences on Pattern Recognition (1978)
Oztel, I., Yolcu, G., Oz, C., Kazan, S., Bunyak, F.: iFER: facial expression recognition using automatically selected geometric eye and eyebrow features. J. Electron. Imaging 27(2), 023003 (2018)
Faria, D.R., Vieira, M., Faria, F.C., Premebida, C.: Affective facial expressions recognition for human-robot interaction. In: IEEE RO-MAN17: IEEE International Symposium on Robot and Human Interactive Communication, Lisbon, Portugal (2017)
Yuqian, Z., Bertram, E.: Action unit selective feature maps in deep networks for facial expression recognition. In: The 2017 International Joint Conference on Neural Networks (IJCNN 2017) (2017)
Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), vol. 1, pp. 886–893. IEEE (2005)
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)
Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (SURF). Comput. Vis. Image Underst. 110(3), 346–359 (2008)
Ojala, T., Pietikäinen, M., Harwood, D.: A comparative study of texture measures with classification based on featured distributions. Pattern Recogn. 29(1), 51–59 (1996)
Ali, G., Iqbal, M.A., Choi, T.S.: Boosted NNE collections for multicultural facial expression recognition. Pattern Recogn. 55, 14–27 (2016)
Rao, Q., Qu, X., Mao, Q., Zhan, Y.: Multi-pose facial expression recognition based on surf boosting. In: 2015 International Conference on Affective Computing and Intelligent Interaction (ACII), pp. 630–635. IEEE (2015)
Lundqvist, D., Flykt, A., Öhman, A.: The Karolinska directed emotional faces (KDEF) (1998)
Huang, G., Mattar, M., Lee, H., Learned-Miller, E.G.: Learning to align from scratch. In: Advances in Neural Information Processing Systems, pp. 764–772 (2012)
Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004)
Kendall, D.G.: A survey of the statistical theory of shape. Stat. Sci. 4, 87–99 (1989)
Breiman, L.: Bagging predictors. Mach. Learn. 24(2), 123–140 (1996)
Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)
Otsu, N.: A threshold selection method from gray-level histograms. Automatica 11(285–296), 23–27 (1975)
Santra, B., Mukherjee, D.P.: Local dominant binary patterns for recognition of multi-view facial expressions. In: Proceedings of the Tenth Indian Conference on Computer Vision, Graphics and Image Processing, p. 25. ACM (2016)
Santra, B., Mukherjee, D.P.: Local saliency-inspired binary patterns for automatic recognition of multi-view facial expression. In: IEEE International Conference on Image Processing (ICIP), pp. 624–628. IEEE (2016)
Hansen, N.: The CMA evolution strategy: a comparing review. In: Lozano, J., Larranaga, P., Inza, I., Bengoetxea, E. (eds.) Towards a New Evolutionary Computation. STUDFUZZ, vol. 192, pp. 75–102. Springer, Heidelberg (2006). https://doi.org/10.1007/3-540-32494-1_4
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Abuhammad, H., Everson, R. (2018). Emotional Faces in the Wild: Feature Descriptors for Emotion Classification. In: Campilho, A., Karray, F., ter Haar Romeny, B. (eds) Image Analysis and Recognition. ICIAR 2018. Lecture Notes in Computer Science(), vol 10882. Springer, Cham. https://doi.org/10.1007/978-3-319-93000-8_19
Download citation
DOI: https://doi.org/10.1007/978-3-319-93000-8_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-92999-6
Online ISBN: 978-3-319-93000-8
eBook Packages: Computer ScienceComputer Science (R0)