Skip to main content

Improved ConvNeXt Facial Expression Recognition Embedded with Attention Mechanism

  • Conference paper
  • First Online:
Applied Intelligence (ICAI 2023)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 2014))

Included in the following conference series:

  • 392 Accesses

Abstract

Facial expression recognition (FER) is an emerging and important research field in the field of pattern recognition, with wide applications in safe driving, intelligent monitoring, and human-computer interaction. This article addresses the problems of insufficient key information extraction, low recognition accuracy, and easy overfitting in facial expression recognition, and proposes an ECA-ConvNeXt network based on transfer learning strategy and channel attention mechanism. Firstly, the weights of the pre-trained model are initialized using transfer learning on the FER 2013 dataset. Secondly, a series of data augmentation operations are performed on the facial images, allowing them to pass through the ECA-Net attention module of the network, enhancing the key information of the feature regions with high relevance to expressions and suppressing the interference of irrelevant regions in the feature maps. Finally, the inverse bottleneck layer, maximum pooling layer, global average pooling layer, and classification layer are sequentially passed into the network to accelerate the convergence speed and improve the expression recognition rate. Compared to the baseline network, the improved network achieved an accuracy of 72.86%, a recall rate of 72.04%, and a specificity of 64.15% on the FER 2013 dataset. Compared to the commonly used ResNet network and its improvement methods, the proposed ECA-ConvNeXt in this article achieved a 0.19% improvement in recognition accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Michael Revina, I., Sam Emmanuel, W.R.: A survey on human face expression recognition techniques. J. King Saud Univ. Comput. Inform. Sci. 33(6), 619–628 (2021). https://doi.org/10.1016/j.jksuci.2018.09.002

    Article  Google Scholar 

  2. Xi, Z., et al.: Facial expression recognition of industrial internet of things by parallel neural networks combining texture features. IEEE Trans. Indust. Inform. 17(4), 2784–2793 (2020)

    Google Scholar 

  3. Shen, L., Bai, L.: A review on gabor wavelets for face recognition. Pattern Anal. Appl. 9, 273–292 (2006)

    Article  MathSciNet  Google Scholar 

  4. Zhao, G., Pietikainen, M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 915–928 (2007)

    Article  Google Scholar 

  5. Wang, X., et al.: Feature fusion of HOG and WLD for facial expression recognition. In: Proceedings of the 2013 IEEE/SICE International Symposium on System Integration, pp. 227–332. IEEE (2013)

    Google Scholar 

  6. Luo, Y., et al.: Facial expression recognition based on fusion feature of PCA and LBP with SVM. Optik-Int. J. Light Electron Opt. 124(17), 2767–2770 (2013)

    Google Scholar 

  7. Dino, H.I., Maiwan, B.A.: Facial expression classification based on SVM, KNN and MLP classifiers. In: 2019 International Conference on Advanced Science and Engineering (ICOASE), pp. 70–75. IEEE (2019)

    Google Scholar 

  8. Krizhevsky, A., et al.: ImageNet classification with deep convolutional neural networks. Commun. ACM 60(6), 84–90 (2017)

    Google Scholar 

  9. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: 3rd International Conference on Learning Representations (ICLR 2015), Computational and Biological Learning Society (2015)

    Google Scholar 

  10. Szegedy, C., et al.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)

    Google Scholar 

  11. He, K., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  12. Tureke, Y., Xu, W., Zhao, J.: AlexNet based facial expression classification. In: International Conference on Cloud Computing, Performance Computing, and Deep Learning (CCPCDL 2022). SPIE, vol. 12287, pp. 521–527 (2022)

    Google Scholar 

  13. Kusuma, G.P., et al.: Emotion Recognition on FER-2013 Face Images Using Fine-Tuned VGG-16. Adv. Sci. Technol. Eng. Syst. J. 5(6), 315–322 (2020). DOI.org (Crossref), https://doi.org/10.25046/aj050638

  14. Gu, S., et al.: Facial expression recognition based on global and local feature fusion with CNNs. In: 2019 IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC), pp. 1–5. IEEE (2019). DOI.org (Crossref). https://doi.org/10.1109/ICSPCC46631.2019.8960765

  15. Vaswani, A., et al.: Attention Is All You Need. Adv. Neural Inform. Process. Syst. 30 (2017)

    Google Scholar 

  16. Tian, C., et al.: Attention-Guided CNN for Image Denoising. Neural Networks 124, 117–129 (2020)

    Google Scholar 

  17. Wang, X., et al.: ECA-ConvNeXt: a rice leaf disease identification model based on ConvNeXt. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6234–6242 (2023)

    Google Scholar 

  18. Woo, S., et al.: Cbam: convolutional block attention module. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–19 (2018)

    Google Scholar 

  19. Hu, J., et al.: Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2018)

    Google Scholar 

  20. Liu, Z., et al.: A Convnet for the 2020s. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11976–11986 (2022)

    Google Scholar 

  21. Liu, Z., et al.: Swin transformer: hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10012–10022 (2021)

    Google Scholar 

  22. Niu, S., et al.: A decade survey of transfer learning (2010–2020). IEEE Trans. Artific. Intell. 1(2), 151–66 (2020)

    Google Scholar 

  23. Yuan, L., et al.: Revisiting knowledge distillation via label smoothing regularization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3903–11 (2020)

    Google Scholar 

  24. Loshchilov, I., Frank H.. Fixing Weight Decay Regularization in Adam (2018)

    Google Scholar 

  25. Han, B., et al.: Masked FER-2013: augmented dataset for facial expression recognition. In: 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), pp. 747–48. IEEE (2023)

    Google Scholar 

  26. Moutan, M., et al.: A deep-learning-based facial expression recognition method using textural features. Neural Comput. Appl. 35(9), 6499–6514 (2023). Springer Link. https://doi.org/10.1007/s00521-022-08005-7

  27. Shervin, M., et al.: Deep-emotion: facial expression recognition using attentional convolutional network. Sensors 21(9), 3046 (2021). DOI.org (Crossref). https://doi.org/10.3390/s21093046

  28. Chen, J., et al.: Facial expression recognition based on the ensemble learning of CNNs. In: 2020 IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC), pp. 1–5. IEEE (2020). Google Scholar. https://ieeexplore.ieee.org/abstract/document/9259543/

  29. Xie, W., et al.: Adaptive weighting of handcrafted feature losses for facial expression recognition. IEEE Trans. Cybern. 51(5), 2787–2800 (2019)

    Google Scholar 

  30. Chen, Y., Hu, H.: Facial expression recognition by inter-class relational learning. IEEE Access 7, 94106–94117 (2019). DOI.org (Crossref). https://doi.org/10.1109/ACCESS.2019.2928983

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Lina Ge .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhao, Y., Ge, L., Cui, G., Fang, T. (2024). Improved ConvNeXt Facial Expression Recognition Embedded with Attention Mechanism. In: Huang, DS., Premaratne, P., Yuan, C. (eds) Applied Intelligence. ICAI 2023. Communications in Computer and Information Science, vol 2014. Springer, Singapore. https://doi.org/10.1007/978-981-97-0903-8_10

Download citation

  • DOI: https://doi.org/10.1007/978-981-97-0903-8_10

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-97-0902-1

  • Online ISBN: 978-981-97-0903-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics