skip to main content
10.1145/3453800.3453819acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicmlscConference Proceedingsconference-collections
research-article

Differential Residual Learning for Facial Expression Recognition

Published: 18 June 2021 Publication History

Abstract

The facial expression recognition algorithm based on convolution neural network (Convolutional Neural Network) still has the problem of imperfect feature point extraction. Therefore, a facial expression recognition algorithm based on the combination of the detail feature extraction network and pre-training model is proposed. According to the emotional face and neutral face of the residual of the images, learn the residual to extract features, and make the feature information accurate. Then, the face feature markers generated by the feature extraction network are loaded into the pre-training model respectively for classification and recognition. By grouping and crossing experiments on datasets CK+ and FER2013, the average accuracy of face recognition is 95.74% and 73.11%, respectively. Compared with the state-of-the-art recognition model, this method is effective in facial expression recognition to some extent.

References

[1]
BEHZAD, M., VO, N., LI, X., and ZHAO, G., 2020. Sparsity-Aware Deep Learning for Automatic 4D Facial Expression Recognition. arXiv preprint arXiv:2002.03157.
[2]
CHEN, Y., WANG, J., CHEN, S., SHI, Z., and CAI, J., 2019. Facial motion prior networks for facial expression recognition. In 2019 IEEE Visual Communications and Image Processing (VCIP) IEEE, 1-4.
[3]
GEORGESCU, M.-I., IONESCU, R.T., and POPESCU, M., 2019. Local learning with deep and handcrafted features for facial expression recognition. IEEE Access 7, 64827-64836.
[4]
GLOROT, X. and BENGIO, Y., 2010. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, 249-256.
[5]
HE, K., ZHANG, X., REN, S., and SUN, J., 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 770-778.
[6]
JAIN, S., HU, C., and AGGARWAL, J.K., 2011. Facial expression recognition with temporal modeling of shapes. In 2011 IEEE international conference on computer vision workshops (ICCV workshops) IEEE, 1642-1649.
[7]
KINGMA, D.P. and BA, J., 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
[8]
KLASER, A., MARSZAŁEK, M., and SCHMID, C., 2008. A spatio-temporal descriptor based on 3d-gradients.
[9]
KOTSIA, I. and PITAS, I., 2006. Facial expression recognition in image sequences using geometric deformation features and support vector machines. IEEE transactions on image processing 16, 1, 172-187.
[10]
LI, S. and DENG, W., 2020. Deep facial expression recognition: A survey. IEEE Transactions on Affective Computing.
[11]
LINLIN, X. and ZHANGSHUMEI, Z., 2019. Expression recognition algorithm for parallel convolutional neural networks. Journalof ImageandGraphics 24, 2, 227-236.
[12]
LITTLEWORT, G., BARTLETT, M.S., FASEL, I., SUSSKIND, J., and MOVELLAN, J., 2004. Dynamics of facial expression extracted automatically from video. In 2004 Conference on Computer Vision and Pattern Recognition Workshop IEEE, 80-80.
[13]
LIU, M., LI, S., SHAN, S., WANG, R., and CHEN, X., 2014. Deeply learning deformable facial action parts model for dynamic expression analysis. In Asian conference on computer vision Springer, 143-157.
[14]
LUCEY, P., COHN, J.F., KANADE, T., SARAGIH, J., AMBADAR, Z., and MATTHEWS, I., 2010. The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression. In 2010 ieee computer society conference on computer vision and pattern recognition-workshops IEEE, 94-101.
[15]
MARRERO FERNANDEZ, P.D., GUERRERO PENA, F.A., REN, T., and CUNHA, A., 2019. Feratt: Facial expression recognition with attention net. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 0-0.
[16]
MOLLAHOSSEINI, A., CHAN, D., and MAHOOR, M.H., 2016. Going deeper in facial expression recognition using deep neural networks. In 2016 IEEE Winter conference on applications of computer vision (WACV) IEEE, 1-10.
[17]
OJANSIVU, V. and HEIKKILä, J., 2008. Blur insensitive texture classification using local phase quantization. In International conference on image and signal processing Springer, 236-243.
[18]
PANTIC, M. and PATRAS, I., 2006. Dynamics of facial expression: recognition of facial actions and their temporal segments from face profile image sequences. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 36, 2, 433-449.
[19]
PRAMERDORFER, C. and KAMPEL, M., 2016. Facial expression recognition using convolutional neural networks: state of the art. arXiv preprint arXiv:1612.02903.
[20]
PTUCHA, R., TSAGKATAKIS, G., and SAVAKIS, A., 2011. Manifold based sparse representation for robust expression recognition without neutral subtraction. In 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops) IEEE, 2136-2143.
[21]
SAGONAS, C., PANAGAKIS, Y., ZAFEIRIOU, S., and PANTIC, M., 2015. Robust statistical face frontalization. In Proceedings of the IEEE international conference on computer vision, 3871-3879.
[22]
SANIN, A., SANDERSON, C., HARANDI, M.T., and LOVELL, B.C., 2013. Spatio-temporal covariance descriptors for action and gesture recognition. In 2013 IEEE Workshop on applications of Computer Vision (WACV) IEEE, 103-110.
[23]
SHAN, C., GONG, S., and MCOWAN, P.W., 2009. Facial expression recognition based on local binary patterns: A comprehensive study. Image and vision Computing 27, 6, 803-816.
[24]
SZEGEDY, C., VANHOUCKE, V., IOFFE, S., SHLENS, J., and WOJNA, Z., 2016. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2818-2826.
[25]
TIAN, Y.-L., 2004. Evaluation of face resolution for expression analysis. In 2004 Conference on Computer Vision and Pattern Recognition Workshop IEEE, 82-82.
[26]
TIAN, Y.-L., KANADE, T., and COHN, J.F., 2005. Facial expression analysis. In Handbook of face recognition Springer, 247-275.
[27]
WANG, K., PENG, X., YANG, J., LU, S., and QIAO, Y., 2020. Suppressing uncertainties for large-scale facial expression recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 6897-6906.
[28]
WANG, S.-J., YAN, W.-J., LI, X., ZHAO, G., and FU, X., 2014. Micro-expression recognition using dynamic textures on tensor independent color space. In 2014 22nd international conference on pattern recognition IEEE, 4678-4683.
[29]
YANG, P., LIU, Q., and METAXAS, D.N., 2010. Exploring facial expressions with compositional features. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition IEEE, 2638-2644.
[30]
ZENG, N., ZHANG, H., SONG, B., LIU, W., LI, Y., and DOBAIE, A.M., 2018. Facial expression recognition via learning deep sparse autoencoders. Neurocomputing 273, 643-649.
[31]
ZHANG, C., PLATT, J.C., and VIOLA, P.A., 2006. Multiple instance boosting for object detection. In Advances in neural information processing systems, 1417-1424.
[32]
ZHANG, C., XU, X., and TU, D., 2018. Face detection using improved faster rcnn. arXiv preprint arXiv:1802.02142.
[33]
ZHAOfigure, G. and PIETIKAINEN, M., 2007. Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE transactions on pattern analysis and machine intelligence 29, 6, 915-928.

Cited By

View all
  • (2023)Facial Expression Recognition with Machine Learning2023 11th International Conference on Information and Communication Technology (ICoICT)10.1109/ICoICT58202.2023.10262748(125-130)Online publication date: 23-Aug-2023

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
ICMLSC '21: Proceedings of the 2021 5th International Conference on Machine Learning and Soft Computing
January 2021
178 pages
ISBN:9781450387613
DOI:10.1145/3453800
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 June 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. CNN
  2. Differential
  3. Facial expression recognition
  4. Feature extraction
  5. Residual learning

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

ICMLSC '21

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)2
  • Downloads (Last 6 weeks)0
Reflects downloads up to 25 Dec 2024

Other Metrics

Citations

Cited By

View all
  • (2023)Facial Expression Recognition with Machine Learning2023 11th International Conference on Information and Communication Technology (ICoICT)10.1109/ICoICT58202.2023.10262748(125-130)Online publication date: 23-Aug-2023

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media