Skip to main content

Advertisement

SpringerLink
Log in

Expression recognition based on residual rectification convolution neural network

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In order to solve the problem of low face recognition rate in controlled scene, an expression recognition algorithm based on residual rectification intensive convolutional neural network is proposed. This method takes convolutional neural network as the prototype. In the process of training model, the idea of residual network is introduced to correct the difference between the effect of test set and the effect of training set. The linear rectification operation of the residual block by the excitation function embedded in the convolution layer helps to express complex features. At the same time, the data intensive method is used to suppress the fast fitting of the deep neural network model during the training process, to improve its generalization performance on a given recognition task, and then to improve the robustness of the model learning effect. In the experiment, the method is applied to simulate the online teaching environment, and get effective facial expression recognition result in controlled scene. According to the experimental data, this method can effectively classify the facial image input under controlled conditions, and the highest accuracy is up to 91.7%. This research is helpful to the development of facial expression recognition and human-computer interaction.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Barnouti NH, Aldabbagh SSM, Matti WE et al (2016) Face detection and recognition using viola-jones with PCA-LDA and square euclidean distance[J]. Int J Advanc Comp Sci Appl 7(5):371–377

    Google Scholar 

  2. Ekman P, Friesen W. V. Pictures of facial affect//Proceedings of the Consulting Psychologists Press. Palo Alto, USA, 1976: 1–7.

  3. Feifei Z, Tianzhu Z, Qirong M, Changsheng X (2019) Multi-pose facial expression recognition via generative adversarial network[J]. Chin J Comp 42(120):1–16

    Google Scholar 

  4. Goodfellow IJ, Erhan D, Carrier PL et al (2013) Int Conf Neural Inform Process:117–124

  5. Goodfellow, I., Bengio, Y., Courville, A.. Deep learning (Vol. 1). Cambridge:MIT press, 2016:326–366.

  6. Goyal SJ, Upadhyay AK, Jadon RS et al (2018) Real-life facical expression recognition systems: a review[J]. Smart Comput Inform 77(1):311–331

    Article  Google Scholar 

  7. Gu WF, Xiang C, Venkatesh YV et al (2012) Facial expression recognition using radial encoding of local Gabor features and classifier synthesis[J]. Pattern Recogn 45(1):80–91

    Article  Google Scholar 

  8. Gu, J., Wang, Z., Kuen, J., Ma, L., Shahroudy, A., Shuai, B., Liu, T., Wang, X., Wang, L., Wang, G. and Cai, J., 2015. Recent advances in convolutional neural networks. arXiv preprint arXiv:1512.07108.

  9. Gurovich Y, Hanani Y, Bar O, Nadav G, Fleischer N, Gelbman D, Basel-Salmon L, Krawitz PM, Kamphausen SB, Zenker M, Bird LM, Gripp KW (2019) Identifying facial phenotypes of genetic disorders using deep learning. Nat Med 25(1):60–64

    Article  Google Scholar 

  10. Hongfei L, Qing L, Li Z (2019) Dynamic facial expression recognition based on multi-visual and audio descriptors[J]. Acta Electron Sin 47(8):1643–1653

    Google Scholar 

  11. Jia MX, Du JQ, Song PF et al (2017) Face recognition based on multi-feature optimization fusion of LBP,LPQ and Gabor with multi-scale blocks [J]. J N Univ (Natural Science) 38(3):310–314

    MathSciNet  Google Scholar 

  12. Juan L, Hu M, Zhong H (2016) Expression recognition method based on evidence fusion of optimal supportability[J]. J Elec Measur Instrum 30(5):714–721

    Google Scholar 

  13. Kaya H (2017) Gurp?Nar F, Salah a. A. Video-based emotion recognition in the wild using deep transfer learning and score fusion[J]. Image Vis Comput 65:66–75

    Article  Google Scholar 

  14. Lekdioui K, Messoussi (2017) Facial decomposition for expression recognition using texture/shape descriptors and SVM classifier[J]. Signal Process Image Commun 58:300–312

    Article  Google Scholar 

  15. Li Y, Li X, Jiang M (2018) Facial expression recognition based on cross-connect LeNet-5 network[J]. Acta Automat Sin 44(1):176–182

    Google Scholar 

  16. Yao Lisha,Xu Guoming,Zhao Feng. Facial expression recognition based on CNN local feature fusion[J]. Laser Optoelectron Progress,2020,57(03):031501.

  17. Liu YP, Li YB, Ma X et al (2017) Facial expression recognition with fusion features extracted from salient facial areas[J]. Sensors 17(4):712

    Article  Google Scholar 

  18. Lucey P, Cohn J F, Kanade T, et al. The extended cohn-kanade dataset (CK+): A complete dataset for action unit and emotion-specified expression [C]. Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops, San Francisco, USA, 2010:94–101.

  19. Mao Q, Zhang F, Wang L, … Dong M (2018) Cascaded multi-level transformed Dirichlet process for multi-pose facial expression recognition[J]. Comput J 61(11):1605–1619

    Article  Google Scholar 

  20. Qianlu H, Qiang W (2019) Facial expression recognition based on cross-connect feature fusion network[J]. Computer Eng Design 31(4):552–559

    Google Scholar 

  21. Soyel H, Demirel H (2012) Localized discriminative scale invariant feature transform based facial expression ecognition[J]. Comp Elec Eng 38(5):1299–1309

    Article  Google Scholar 

  22. Su H, Maji S, Kalogerakis E et al (2015) Multi-view convolutional neural networks for 3d shape recognition//proceedings of the IEEE international conference on computer vision. Santiago, Chile:945–953

  23. Sun X, Pan T (2017) Static facial expression recognition system using ROI deep neural networks[J]. Tien Tzu Hsueh Pao/Acta Electronica Sinica 45(5):1189–1197

    Google Scholar 

  24. Tan X, Li Z, Fan Y (2019) Facial expression recognition method based on multi-scale detail enhancement[J]. J Electron Inform Technol 41(11):2752–2759

    Google Scholar 

  25. Wang XY, Fan JZ, Liu HM et al (2019) Multi-expression sequence fusion recognition based on probabilistic cooperative representation[J]. Laser Optoelectron Progress 56(13):131004

    Article  Google Scholar 

  26. Xia T, Zhang Y, Yuan L (2019) Landmark-based facial expression recognition by joint training of multiple networks[J]. J Comp-Aided Design Comp Graphics 31(4):552–559

    Article  Google Scholar 

  27. Xu Y, Zhenhong S (2020) Facial expression recognition based on improved AlexNet[J]. Laser Optoelectron Progress 57(14):141026

    Article  Google Scholar 

  28. Yao L, Xu G, Zhao F (2020) Facial expression recognition based on CNN local feature fusion[J]. Laser Optoelectron Progress 57(03):031501

    Google Scholar 

  29. Yu HM, Sheng nan, Xiaohua W (2019) Facial expression recognition method based on constraint cycle-consistent generative adversarial networks[J]. J Electron Meas Instrum 33(4):169–177

    Google Scholar 

  30. Zheng W, Zhou X, Zou C, … Zhao L (2006) Facial expression recognition using kernel canonical correlation analysis (KCCA)[J]. IEEE Trans Neural Netw 17(1):233–238

    Article  Google Scholar 

  31. Zheng W, Zhou X, Zou C et al (2006) Facial expression recognition using kernel canonical correlation analysis (KCCA)[J]. IEEE Trans Neural Netw 17(1):233–238

    Article  Google Scholar 

  32. Zhou HL, Kinman L, He XJ (2016) Shape-appearance-correlated active appearance model[J]. Pattern Recogn 56(C):88–99

    Article  Google Scholar 

Download references

Funding

Supported by Research on Intelligent Campus of Modern Educational Technology in Jiangsu Province (2021-R-96609).

Author information

Authors and Affiliations

  1. Informatization Office of Nanjing Normal University, Nanjing, 210023, China

    Bin Chen, Jin-ning Zhu & Yi-zhou Dong

Authors
  1. Bin Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jin-ning Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yi-zhou Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bin Chen.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, B., Zhu, Jn. & Dong, Yz. Expression recognition based on residual rectification convolution neural network. Multimed Tools Appl 81, 9671–9683 (2022). https://doi.org/10.1007/s11042-022-12159-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12159-w

Keywords

Search

Navigation