Abstract
Automatic recognition of facial expression is an important task in many applications such as face recognition and animation, human-computer interface and online/remote education. It is still challenging due to variations of expression, background and position. In this paper, we propose a method for facial expression recognition based on ensemble of multiple Convolutional Neural Networks (CNNs). First, the face region is extracted by a face detector from the pre-processed image. Second, five key points are detected for each image and the face images are aligned by two eye center points. Third, the face image is cropped into local eye and mouth regions, and three CNNs are trained for the whole face, eye and mouth regions, individually. Finally, the classification is made by ensemble of the outputs of three CNNs. Experiments were carried for recognition of six facial expressions on the Extended Cohn-Kanade database (CK+). The results and comparison show the proposed algorithm yields performance improvements for facial expression recognition.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Wu, Y., Liu, H., Zha, H.: Modeling facial expression space for recognition. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1968–1973 (2005)
Caleanu, C.-D.: Face expression recognition: a brief overview of the last decade. In: 8th IEEE International Symposium on Applied Computational Intelligence and Informatics, pp. 157–161 (2013)
Bettadapura, V.: Face expression recognition and analysis: the state of the art. arXiv preprint arXiv:1203.6722 (2012)
Liu, P., Han, S., Meng, Z., Tong, Y.: Facial expression recognition via a boosted deep belief network. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1805–1812 (2014)
Andre, T.L., Edilson, A., Thiago, O.-S.: A facial expression recognition system using convolutional networks. In: 28th SIBGRAPI Conference on Graphics, Patterns and Images, pp. 273–280 (2015)
Byeon, Y.-H., Kwak, K.-C.: Facial expression recognition using 3d convolutional neural network. Int. J. Adv. Comput. Sci. Appl. 5(12) (2014)
Shan, C., Gong, S., McOwan, P.W.: Facial expression recognition based on local binary patterns: a comprehensive study. Image Vis. Comput. 27(6), 803–816 (2009)
Jain, S., Hu, C., Aggarwal, J.K.: Facial expression recognition with temporal modeling of shapes. In: ICCV Workshops, pp. 1642–1649 (2011)
Whitehill, J., Bartlett, M.S., Littlewort, G., Fasel, I., Movellan, J.R.: Towards practical smile detection. IEEE T-PAMI 31(11), 2106–2111 (2009)
Foster, I., Kesselman, C., Nick, J., Tuecke, S.: Image based static facial expression recognition with multiple deep network learning. In: Proceedings of the ACM on International Conference on Multimodal Interaction, Seattle, WA, USA, 09–13 November 2015
Burkert P., Trier, F., Afzal, M.Z., Dengel, A., Liwicki, M.: DeXpression deep convolutional neural network for expression recognition. arXiv:1509.05371 [cs.CV]
Sun, Y., Wang, X., Tang, X.: Deep convolutional network cascade for facial point detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2013)
Cohn, J.F., Zlochower, A.: A Computerized Analysis of Facial Expression: Feasibility of Automated Discrimination. American Psychological Society, New York (1995)
Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended cohn-kanade dataset (CK+): a complete dataset for action unit and emotion-specified expression. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 94–101 (2010)
Palm, R.B.: Prediction as a candidate for learning deep hierarchical models of data (2012). http://www2.imm.dtu.dk/pubdb/views/publication_details.php?id=6284
Zhen, Q., Huang, D., Wang, Y., Chen, L.: Muscular movement model based automatic 3d facial expression recognition. In: He, X., Luo, S., Tao, D., Xu, C., Yang, J., Hasan, M.A. (eds.) MMM 2015, Part I. LNCS, vol. 8935, pp. 522–533. Springer, Heidelberg (2015)
Acknowledgement
This work was supported by National Natural Science Foundation of China under the grants No. 61375112 and 61005024.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Cui, R., Liu, M., Liu, M. (2016). Facial Expression Recognition Based on Ensemble of Mulitple CNNs. In: You, Z., et al. Biometric Recognition. CCBR 2016. Lecture Notes in Computer Science(), vol 9967. Springer, Cham. https://doi.org/10.1007/978-3-319-46654-5_56
Download citation
DOI: https://doi.org/10.1007/978-3-319-46654-5_56
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-46653-8
Online ISBN: 978-3-319-46654-5
eBook Packages: Computer ScienceComputer Science (R0)