Abstract
We present a deep convolutional neural network (CNN) architecture for facial expression recognition. Inspired by the fact that regions located around certain facial parts (e.g. mouth, nose, eyes, and brows) contain the most representative information of expressions, an architecture extracts features at different scale from intermediate layers is designed to combine both local and global information. In addition, noticing that in specific to facial expression recognition, traditional face alignment would distort the images and lose expression information. To avoid this side effect, we apply batch normalization to the architecture instead of face alignment and feed the network with original images. Moreover, considering the tiny differences between classes caused by the same facial movements, a triplet-loss learning method is used to train the architecture, which improves the discrimination of deep features. Experiments show that the proposed architecture achieves superior performance to other state-of-the-art methods on the FER2013 dataset.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Parkhi, O.M., Vedaldi, A., Zisserman, A.: Deep face recognition. In: Proceedings of the British Machine Vision, vol. 1(3), p. 6 (2015)
Yi, D., Lei, Z., Liao, S., et al.: Learning face representation from scratch. Eprint Arxiv (2014)
Sun, Y., Wang, X., Tang, X.: Deep learning face representation from predicting 10,000 classes. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1891–1898. IEEE (2014)
Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 815–823. IEEE (2015)
Hoffer, E., Ailon, N.: Deep metric learning using triplet network. In: Feragen, A., Pelillo, M., Loog, M. (eds.) SIMBAD 2015. LNCS, vol. 9370, pp. 84–92. Springer, Heidelberg (2015). doi:10.1007/978-3-319-24261-3_7
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 (2015)
Yu, Z., Zhang, C.: Image based static facial expression recognition with multiple deep network learning. In: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, pp. 435–442. ACM (2015)
Mollahosseini, A., Chan, D., Mahoor, M.H.: Going deeper in facial expression recognition using deep neural networks. arXiv preprint arXiv:1511.04110 (2015)
Khatri, N.N., Shah, Z.H., Patel, S.A.: Facial expression recognition: a survey. IJCSIT Int. J. Comput. Sci. Inf. Technol. 5(1), 149–152 (2014)
Burkert, P., Trier, F., Afzal, M.Z., et al.: DeXpression: deep convolutional neural network for expression recognition. arXiv preprint arXiv:1509.05371 (2015)
Ekman, P., Friesen, W.V.: Constants across cultures in the face and emotion. J. Pers. Soc. Psychol. 17(2), 124 (1971)
Szegedy, C., Liu, W., Jia, Y., et al.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)
Liu, Z., Luo, P., Wang, X., et al.: Deep learning face attributes in the wild. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3730–3738 (2015)
Liu, M., Li, S., Shan, S., et al.: Au-aware deep networks for facial expression recognition. In: 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), pp. 1–6. IEEE (2013)
Ranjan, R., Patel, V.M., Chellappa, R.: HyperFace: a deep multi-task learning framework for face detection, landmark localization, pose estimation, and gender recognition. arXiv preprint arXiv:1603.01249 (2016)
Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. CoRR, abs/1311.2901 (2013)
Tang, Y.: Deep learning using linear support vector machines. arXiv preprint arXiv:1306.0239 (2013)
Shimodaira, H.: Improving predictive inference under covariate shift by weighting the log-likelihood function. J. Stat. Planning Infer. 90(2), 227–244 (2000)
Acknowledgements
This work is supported by the High Technology Development Program of China (863 Program), under Grant No. 2011AA01A205, National Significant Science and Technology Projects of China, under Grant No. 2013ZX01039001-002-003; by the NSFC project under Grant Nos. U1433112 and 61170253.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Wang, J., Yuan, C. (2016). Facial Expression Recognition with Multi-scale Convolution Neural Network. In: Chen, E., Gong, Y., Tie, Y. (eds) Advances in Multimedia Information Processing - PCM 2016. PCM 2016. Lecture Notes in Computer Science(), vol 9916. Springer, Cham. https://doi.org/10.1007/978-3-319-48890-5_37
Download citation
DOI: https://doi.org/10.1007/978-3-319-48890-5_37
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-48889-9
Online ISBN: 978-3-319-48890-5
eBook Packages: Computer ScienceComputer Science (R0)