Skip to main content
SpringerLink
Log in

Relation-Aware Facial Expression Recognition Using Contextual Residual Network with Attention Mechanism

  • Conference paper
  • First Online:
Computer Science and Education (ICCSE 2022)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1811))

Included in the following conference series:

  • 408 Accesses

Abstract

With the existence of occlusion or posture changes, facial expression recognition (FER) under uncontrolled conditions is difficult. To obtain a discriminative representation for challenging expression recognition, we present a relation-aware FER model, referred to attention-coordinated contextual residual network (ACRN), which contains a contextual residual network (CRNet) and attention module (AM). Firstly, multi-level features of facial expression are extracted by CRNet. There are some contextual convolution (CoConv) blocks in CRNet to integrate expression information in facial space; then, AM is embedded into each stage of CRNet to coordinate contextual information from CoConv blocks across the channel and spatial location, weighting multi-level expression features to differentiate their importance; finally, to highlight expression-related facial areas, the output of CRNet will be fed into AM, which can focus on salient features. The experiments on Affectnet-7 and RAF_DB datasets have shown that ACRN can both explore the interaction of facial information and capture subtle features related to expression, thus obtaining a better recognition performance.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ramdani, C., Ogier, M., Coutrot, A.: Communicating and reading emotion with masked faces in the Covid era: a short review of the literature. Psychiat. Res. 316, 114755 (2022)

    Article  Google Scholar 

  2. Deng, S., Sun, Y., Galvan, E.: Neural architecture search using genetic algorithm for facial expression recognition. In: Genetic and Evolutionary Computation Conference Companion, pp. 423–426 (2022)

    Google Scholar 

  3. Seal, A., Saha, R., Kumar, R., et al.: facial emotion recognition in static and live streaming image dataset using CNN. In: International Conference on Computational Intelligence in Communications and Business Analytics, pp. 288–297 (2022)

    Google Scholar 

  4. Tuama, B.A., Shawkat, S.A., Askar, N.A.: Recognition and classification of facial expressions using artificial neural networks. In: 2022 International Congress on Human-Computer Interaction, Optimization and Robotic Applications, pp. 1–8 (2022)

    Google Scholar 

  5. Yan, R., Yang, M., Zheng, Q., et al.: Facial expression recognition based on hybrid geometry-appearance and dynamic-still feature fusion. Multimedia Tools. Appl. 82, 1–26 (2022)

    Google Scholar 

  6. Lei, J., Liu, Z., Zou, Z., et al.: Mid-level representation enhancement and graph embedded uncertainty suppressing for facial expression recognition. arXiv preprint arXiv: 2207.13235 (2022)

    Google Scholar 

  7. Ramadhan, A.D., Usman, K., Pratiwi, N.K.C.: Comparative analysis of various optimizers on residual network architecture for facial expression identification. In: 2nd International Conference on Electronics, Biomedical Engineering, and Health Informatics, pp. 279–288 (2022)

    Google Scholar 

  8. Ahadit, A.B., Jatoth, R.K.: A novel multi-feature fusion deep neural network using HOG and VGG-Face for facial expression classification. Mach. Vision Appl. 33(4), 1–23 (2022)

    Article  Google Scholar 

  9. Kartheek, M.N., Prasad, M.V.N.K., Bhukya R.: Texture based feature extraction using symbol patterns for facial expression recognition. Cogn. Neurodyn., 1–19 (2022)

    Google Scholar 

  10. Du, Y., Wang, Q., Xiong, Y.: Adaptive graph-based feature normalization for facial expression recognition. arXiv preprint arXiv: 2207.11123 (2022)

    Google Scholar 

  11. Li, Y., Zeng, J., Shan, S., et al.: Patch-gated CNN for occlusion-aware facial expression recognition. In: 24th International Conference on Pattern Recognition, pp. 2209–2214 (2018)

    Google Scholar 

  12. Li, S., Deng, W., Du, J, P.: Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2852–2861 (2017)

    Google Scholar 

  13. Song, L., Gong, D., Li, Z., et al.: Occlusion robust face recognition based on mask learning with pairwise differential siamese network. In: IEEE/CVF International Conference on Computer Vision, pp. 773–782 (2019)

    Google Scholar 

  14. Xie, S., Hu, H.: Facial expression recognition using hierarchical features with deep comprehensive multipatches aggregation convolutional neural networks. IEEE Trans. Multimedia. 21(1), 211–220 (2018)

    Article  MathSciNet  Google Scholar 

  15. Ruan, D., Yan, Y., Lai, S., et al.: Feature decomposition and reconstruction learning for effective facial expression recognition. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7660–7669 (2021)

    Google Scholar 

  16. Zhao, S., Cai, H., Liu, H., et al.: Feature selection mechanism in CNNs for facial expression recognition. In: British Machine Vision Conference, p. 317 (2018)

    Google Scholar 

  17. Huang, Q., Huang, C., Wang, X., et al.: Facial expression recognition with grid-wise attention and visual transformer. Inf. Sci. 580, 35–54 (2021)

    Article  MathSciNet  Google Scholar 

  18. Liu, D., Ouyang, X., Xu, S., et al.: SAANet: siamese action-units attention network for improving dynamic facial expression recognition. Neurocomputing 413, 145–157 (2020)

    Article  Google Scholar 

  19. Duta, I.C., Georgescu, M.I., Ionescu, R.T.: Contextual convolutional neural networks. In: IEEE/CVF International Conference on Computer Vision, pp. 403–412 (2021)

    Google Scholar 

  20. Hou, Q., Zhou, D., Feng, J.: Coordinate attention for efficient mobile network design. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13713–13722 (2021)

    Google Scholar 

  21. Mollahosseini, A., Hasani, B., Mahoor, M.H.: Affectnet: a database for facial expression, valence, and arousal computing in the wild. IEEE Trans. Affect. Comput. 10(1), 18–31 (2017)

    Article  Google Scholar 

  22. Li, S., Deng, W.: Reliable crowdsourcing and deep locality-preserving learning for unconstrained facial expression recognition. IEEE Trans. Image. Process. 28(1), 356–370 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  23. Chen, Y., Wang, J., Chen, S., et al.: Facial motion prior networks for facial expression recognition. In: IEEE Visual Communications and Image Processing (VCIP), pp. 1–4 (2019)

    Google Scholar 

  24. Ding, H., Zhou, P., Chellappa, R.: Occlusion-adaptive deep network for robust facial expression recognition. In: IEEE International Joint Conference on Biometrics (IJCB), pp. 1–9 (2020)

    Google Scholar 

  25. Hua, W., Dai, F., Huang, L., et al.: HERO: human emotions recognition for realizing intelligent Internet of Things. IEEE Access 7, 24321–24332 (2019)

    Article  Google Scholar 

  26. Farzaneh, A.H., Qi, X.: Discriminant distribution-agnostic loss for facial expression recognition in the wild. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 406–407 (2020)

    Google Scholar 

  27. Lian, Z., Li, Y., Tao, J.H., et al.: Expression analysis based on face regions in real-world conditions. Int. J. Autom. Comput. 17(1), 96–107 (2020)

    Article  Google Scholar 

  28. Skublov, S.G., Gavrilchik, A.K., Berezin, A.V.: Geochemistry of beryl varieties: comparative analysis and visualization of analytical data by principal component analysis (PCA) and t-distributed stochastic neighbor embedding (t-SNE). J. Min. Inst. 255, 455–469 (2022)

    Article  Google Scholar 

Download references

Acknowledgment

This work was supported in part by the National Science Foundation of China (Grant No. 61871176), applied research plan of key scientific research projects in Henan colleges and Universities (Grant No. 22A510013), the Scientific Research Foundation Natural Science Project in Henan University of Technology (Grant No. 2018RCJH18), and the Innovative Funds Plan of Henan University of Technology Plan (Grant No. 2020ZKCJ02).

Author information

Authors and Affiliations

  1. Key Laboratory of Grain Information Processing and Control, Ministry of Education, Henan University of Technology, Zhengzhou, Henan, China

    Xue Li, Chunhua Zhu & Fei Zhou

  2. College of Information Science and Engineering, Henan University of Technology, Zhengzhou, Henan, China

    Xue Li, Chunhua Zhu & Fei Zhou

  3. Henan Key Laboratory of Grain Photoelectric Detection and Control, Henan University of Technology, Zhengzhou, Henan, China

    Xue Li, Chunhua Zhu & Fei Zhou

Authors
  1. Xue Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chunhua Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fei Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chunhua Zhu .

Editor information

Editors and Affiliations

  1. Xiamen University, Xiamen, China

    Wenxing Hong

  2. Sichuan University, Chengdu, China

    Yang Weng

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Li, X., Zhu, C., Zhou, F. (2023). Relation-Aware Facial Expression Recognition Using Contextual Residual Network with Attention Mechanism. In: Hong, W., Weng, Y. (eds) Computer Science and Education. ICCSE 2022. Communications in Computer and Information Science, vol 1811. Springer, Singapore. https://doi.org/10.1007/978-981-99-2443-1_55

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-2443-1_55

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-2442-4

  • Online ISBN: 978-981-99-2443-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation