Abstract
The accuracy of facial expression recognition is greatly impacted by individual attributes. To address this problem, we propose a Disentangle Expressions based on Self-Attention Conditional Generative Adversarial Nets method, where facial expression recognition takes by two steps. The first step constructed a generative model to generate the corresponding neutral face image and disentangle expression features. The second step trained the classifier with preserved disentangled expression features. A self-attention layer is used to learn correlations among different facial motion units. Inspired by the relativistic GAN [1], we use the discriminator to predict the relative realness of the generated images and provide strong supervision for more details recovery. The results from extensive experiments on three public facial expression datasets (CK+ , MMI, Oulu-CASIA) proved that our method is more effective than the known state-of-the-art methods in recognition accuracy.
Haohao Li is a student at Beijing Information Science and Technology University.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Jolicoeur-Martineau, A.: The relativistic discriminator: a key element missing from standard GAN. arXiv preprint arXiv:1807.00734 (2018)
Zhao, X., Lin, Y., Heikkila, J.: Dynamic texture recognition using volume local binary count patterns with an application to 2D face spoofing detection. IEEE Trans. Multimedia 20(3), 552–566 (2018)
Klaser, A., Marszałek, M., Schmid, C.: A spatio-temporal descriptor based on 3D-gradients. In: British Machine Vision Conference, pp. 1–10 (2008)
Liu, M., Li, S., Shan, S.: Deeply learning deformable facial action parts model for dynamic expression analysis. In: Asian Conference on Computer Vision, pp. 143–157 (2014)
Liu, M., Shan, S., Wang, R.: Learning expression lets on spatio-temporal manifold for dynamic facial expression recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1749–1756 (2014)
Zhang, F., Zhang, T., Mao, Q.: Joint pose and expression modeling for facial expression recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3359–3368 (2018)
Meng, Z., Liu, P., Cai, J.: Identity-aware convolutional neural network for facial expression recognition. In: IEEE International Conference on Automatic Face & Gesture Recognition, pp. 558–565 (2017)
Kim, Y., Yoo, B.I., Kwak, Y.: Deep generative-contrastive networks for facial expression recognition. arXiv preprint arXiv:1703.07140 (2017)
Lee, S.H., Plataniotis, K.N.K., Ro, Y.M.: Intra-class variation reduction using training expression images for sparse representation based facial expression recognition. IEEE Trans. Affect. Comput. 5(3), 340–351 (2014)
Zafeiriou, S., Petrou, M.: Sparse representations for facial expressions recognition via l1 optimization. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 32–39 (2010)
Zeng, Z., Pantic, M., Roisman, G.I.: A survey of affect recognition methods: audio, visual, and spon-taneous expressions. IEEE Trans. Pattern Anal. Mach. Intell. 31(1), 39–58 (2009)
Sariyanidi, E., Gunes, H., Cavallaro, A.: Automatic analysis of facial affect: a survey of registration, representation, and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 37(6), 1113–1133 (2015)
Pantic, M., Pentland, A., Nijholt, A., Huang, T.S.: Human computing and machine understanding of human behavior: a survey. In: Huang, T.S., Nijholt, A., Pantic, M., Pentland, A. (eds.) Artifical Intelligence for Human Computing. LNCS (LNAI), vol. 4451, pp. 47–71. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-72348-6_3
Valstar, M.F., Mehu, M., Jiang, B.: Meta-analysis of the first facial expression recognition challenge. IEEE Trans. Syst. Man, Cybern. 42(4), 966–979 (2012)
Lu, Y., Tai, Y.W., Tang, C.K.: Attribute-guided face generation using conditional cycleGAN. In: European Conference on Computer Vision, pp. 282–297 (2018)
Isola, P., Zhu, J.Y., Zhou, T.: Image-to-image translation with conditional adversarial networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125–1134 (2017)
Zhang, H., Sindagi, V., Patel, V.M.: Image de-raining using a conditional generative adversarial network. IEEE Trans. Circ. Syst. Video Technol. (2019)
Zhao, G., Pietikainen, M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 6, 915–928 (2007)
Klaser, A., Marszalek, M., Schmid, C.: A spatio-temporal descriptor based on 3D-gradients. In: British Machine Vision Conference, pp. 1–10 (2008)
Yuce, A., Gao, H., Thiran, J.P.: Discriminant multi-label manifold embedding for facial action unit detection. In: IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, pp. 1–6 (2015)
Jain, S., Hu, C., Aggarwal, J.K.: Facial expression recognition with temporal modeling of shapes. In: IEEE International Conference on Computer Vision Workshops, pp. 1642–1649 (2011)
Yang, H., Ciftci, U., Yin, L.: Facial expression recognition by de-expression residue learning. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2168–2177 (2018)
Zhang, H., Goodfellow, I., Metaxas, D.: Self-attention generative adversarial networks. arXiv preprint arXiv:1805.08318 (2018)
Lucey, P., Cohn, J.F., Kanade, T.: The extended Cohn-Kanade dataset (CK+): a complete dataset for action unit and emotion-specified expression. In: IEEE Conference on Computer Vision and Pattern Recognition-Workshops, pp. 94–101 (2010)
Zhao, G., Huang, X., Taini, M.: Facial expression recognition from near-infrared videos. Image Vis. Comput. 29(9), 607–619 (2011)
Pantic, M., Valstar, M., Rademaker, R.: Web-based database for facial expression analysis. In: IEEE International Conference on Multimedia and Expo (2005)
Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004)
Acknowledgement
This work is supported by the Program for the Outstanding Young Talents of Municipal Colleges and Universities of Beijing under contract No.CIT&TCD201804054.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Li, H., Liu, Q., Wei, X., Chai, Z., Chen, W. (2019). Facial Expression Recognition: Disentangling Expression Based on Self-attention Conditional Generative Adversarial Nets. In: Lin, Z., et al. Pattern Recognition and Computer Vision. PRCV 2019. Lecture Notes in Computer Science(), vol 11858. Springer, Cham. https://doi.org/10.1007/978-3-030-31723-2_62
Download citation
DOI: https://doi.org/10.1007/978-3-030-31723-2_62
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-31722-5
Online ISBN: 978-3-030-31723-2
eBook Packages: Computer ScienceComputer Science (R0)