Abstract
Facial expression recognition (FER) is an emerging and important research field in the field of pattern recognition, with wide applications in safe driving, intelligent monitoring, and human-computer interaction. This article addresses the problems of insufficient key information extraction, low recognition accuracy, and easy overfitting in facial expression recognition, and proposes an ECA-ConvNeXt network based on transfer learning strategy and channel attention mechanism. Firstly, the weights of the pre-trained model are initialized using transfer learning on the FER 2013 dataset. Secondly, a series of data augmentation operations are performed on the facial images, allowing them to pass through the ECA-Net attention module of the network, enhancing the key information of the feature regions with high relevance to expressions and suppressing the interference of irrelevant regions in the feature maps. Finally, the inverse bottleneck layer, maximum pooling layer, global average pooling layer, and classification layer are sequentially passed into the network to accelerate the convergence speed and improve the expression recognition rate. Compared to the baseline network, the improved network achieved an accuracy of 72.86%, a recall rate of 72.04%, and a specificity of 64.15% on the FER 2013 dataset. Compared to the commonly used ResNet network and its improvement methods, the proposed ECA-ConvNeXt in this article achieved a 0.19% improvement in recognition accuracy.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Michael Revina, I., Sam Emmanuel, W.R.: A survey on human face expression recognition techniques. J. King Saud Univ. Comput. Inform. Sci. 33(6), 619–628 (2021). https://doi.org/10.1016/j.jksuci.2018.09.002
Xi, Z., et al.: Facial expression recognition of industrial internet of things by parallel neural networks combining texture features. IEEE Trans. Indust. Inform. 17(4), 2784–2793 (2020)
Shen, L., Bai, L.: A review on gabor wavelets for face recognition. Pattern Anal. Appl. 9, 273–292 (2006)
Zhao, G., Pietikainen, M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 915–928 (2007)
Wang, X., et al.: Feature fusion of HOG and WLD for facial expression recognition. In: Proceedings of the 2013 IEEE/SICE International Symposium on System Integration, pp. 227–332. IEEE (2013)
Luo, Y., et al.: Facial expression recognition based on fusion feature of PCA and LBP with SVM. Optik-Int. J. Light Electron Opt. 124(17), 2767–2770 (2013)
Dino, H.I., Maiwan, B.A.: Facial expression classification based on SVM, KNN and MLP classifiers. In: 2019 International Conference on Advanced Science and Engineering (ICOASE), pp. 70–75. IEEE (2019)
Krizhevsky, A., et al.: ImageNet classification with deep convolutional neural networks. Commun. ACM 60(6), 84–90 (2017)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: 3rd International Conference on Learning Representations (ICLR 2015), Computational and Biological Learning Society (2015)
Szegedy, C., et al.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)
He, K., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Tureke, Y., Xu, W., Zhao, J.: AlexNet based facial expression classification. In: International Conference on Cloud Computing, Performance Computing, and Deep Learning (CCPCDL 2022). SPIE, vol. 12287, pp. 521–527 (2022)
Kusuma, G.P., et al.: Emotion Recognition on FER-2013 Face Images Using Fine-Tuned VGG-16. Adv. Sci. Technol. Eng. Syst. J. 5(6), 315–322 (2020). DOI.org (Crossref), https://doi.org/10.25046/aj050638
Gu, S., et al.: Facial expression recognition based on global and local feature fusion with CNNs. In: 2019 IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC), pp. 1–5. IEEE (2019). DOI.org (Crossref). https://doi.org/10.1109/ICSPCC46631.2019.8960765
Vaswani, A., et al.: Attention Is All You Need. Adv. Neural Inform. Process. Syst. 30 (2017)
Tian, C., et al.: Attention-Guided CNN for Image Denoising. Neural Networks 124, 117–129 (2020)
Wang, X., et al.: ECA-ConvNeXt: a rice leaf disease identification model based on ConvNeXt. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6234–6242 (2023)
Woo, S., et al.: Cbam: convolutional block attention module. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–19 (2018)
Hu, J., et al.: Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2018)
Liu, Z., et al.: A Convnet for the 2020s. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11976–11986 (2022)
Liu, Z., et al.: Swin transformer: hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10012–10022 (2021)
Niu, S., et al.: A decade survey of transfer learning (2010–2020). IEEE Trans. Artific. Intell. 1(2), 151–66 (2020)
Yuan, L., et al.: Revisiting knowledge distillation via label smoothing regularization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3903–11 (2020)
Loshchilov, I., Frank H.. Fixing Weight Decay Regularization in Adam (2018)
Han, B., et al.: Masked FER-2013: augmented dataset for facial expression recognition. In: 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), pp. 747–48. IEEE (2023)
Moutan, M., et al.: A deep-learning-based facial expression recognition method using textural features. Neural Comput. Appl. 35(9), 6499–6514 (2023). Springer Link. https://doi.org/10.1007/s00521-022-08005-7
Shervin, M., et al.: Deep-emotion: facial expression recognition using attentional convolutional network. Sensors 21(9), 3046 (2021). DOI.org (Crossref). https://doi.org/10.3390/s21093046
Chen, J., et al.: Facial expression recognition based on the ensemble learning of CNNs. In: 2020 IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC), pp. 1–5. IEEE (2020). Google Scholar. https://ieeexplore.ieee.org/abstract/document/9259543/
Xie, W., et al.: Adaptive weighting of handcrafted feature losses for facial expression recognition. IEEE Trans. Cybern. 51(5), 2787–2800 (2019)
Chen, Y., Hu, H.: Facial expression recognition by inter-class relational learning. IEEE Access 7, 94106–94117 (2019). DOI.org (Crossref). https://doi.org/10.1109/ACCESS.2019.2928983
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Zhao, Y., Ge, L., Cui, G., Fang, T. (2024). Improved ConvNeXt Facial Expression Recognition Embedded with Attention Mechanism. In: Huang, DS., Premaratne, P., Yuan, C. (eds) Applied Intelligence. ICAI 2023. Communications in Computer and Information Science, vol 2014. Springer, Singapore. https://doi.org/10.1007/978-981-97-0903-8_10
Download citation
DOI: https://doi.org/10.1007/978-981-97-0903-8_10
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-0902-1
Online ISBN: 978-981-97-0903-8
eBook Packages: Computer ScienceComputer Science (R0)