Abstract
In order to solve the problems of insufficient facial expression feature extraction and large parameter amount in some convolutional neural networks, a facial expression recognition algorithm based on multi-feature fusion is proposed. This method first modifies the residual block in the ResNet network, reduces the amount of network parameters and uses pre-activation to reduce the error rate. After that, the features extracted by the improved ResNet network are fused with the features extracted by the VGG network after the cut layer, and the network model P-ResNet-VGG is obtained. The loss function uses the cross entropy loss function. This model has been extensively tested on the FER2013 and JAFFE datasets. The experimental results show that this model has improved accuracy on the expression data set than other models, and it has a significant effect on the FER2013 and JAFFE data sets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Logie, R.H., Baddeley, A.D., Woodhead, M.M.: Face recognition, pose and ecological validity. Appl. Cogn. Psychol. 1(1), 53–69 (2015)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Proceedings of Advances in Neural Information Processing Systems, New York, pp. 1097–1105 (2012)
Russakovsky, O., Deng, J., Su, H., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015)
Goodfellow, I.J., Erhan, D., Carrier, P.L., et al.: Challenges in representation learning: a report on three machine learning contests. Neural Netw. 64, 59–63 (2015)
Xu, M.Y., Tang, Z.M., Yao, Y.Z., et al.: Deep learning for person reidentification using support vector machines. Adv. Multimed. 2017, 11–18 (2017)
Wang, Y., Su, W.J., Liu, H.L.: Facial expression recognition based on linear discriminant locality preserving analysis algorithm. J. Inf. Comput. Sci. 9(11), 4281–4289 (2013)
Tang, C., Zheng, W., Yan, J., et al.: View-independent facial action unit detection. In: Proceedings of the 12th IEEE International Conference on Automatic Face and Gesture Recognition, Los Alamitos, CA, USA, pp. 878–882 (2017)
Chu, J., Tang, W., Zhang, S.: Facial expression recognition algorithm based on attention model. Laser Optoelectron. Prog. 57(12), 121015 (2020)
Lu, G., He, J., Yan, J.: A convolutional neural network for facial expression recognition. J. Nanjing Univ. 36(1), 16–22 (2016)
Li, X., Niu, H.: Facial expression recognition based on feature fusion based on VGG-NET. Comput. Eng. Sci. 42(03), 500–509 (2020)
Li, M., Li, X., Wang, X., et al.: Real-time face expression recognition based on multi-scale kernel feature convolutional neural network. J. Comput. Appl. 39(09), 2568–2574 (2019)
Mishra, G., Vishwakarma, V.P., Aggarwal, A.: Face recognition using linear sparse approximation with multi-modal feature fusion. J. Discrete Math. Sci. Crypt. 22(2), 161–175 (2019)
Wang, H.: Enhanced forest microexpression recognition based on optical flow direction histogram and deep multiview network. Math. Probl. Eng. 2020(8), 1–11 (2020)
Li, D., Zhao, X., Yuan, G., et al.: Robustness comparison between the capsule network and the convolutional network for facial expression recognition. Appl. Intell. 51(4), 2269–2278 (2020)
Zhang, T., Zheng, W., Cui, Z., et al.: A deep neural network-driven feature learning method for multi-view facial expression recognition. IEEE Trans. Multimed. 18(12), 2528–2536 (2016)
Guo, Y.N., Tao, D.P., Yu, J.: Deep neural networks with relativity learning for facial expression recognition. In: Proceedings of the 2016 IEEE International Conference on Multimedia and Expo Workshop, Washington, pp. 166–170 (2016)
Zhou, S., Liang, Y., Wan, J., Li, S.Z.: Facial expression recognition based on multi-scale CNNs. In: You, Z., et al. (eds.) CCBR 2016. LNCS, vol. 9967, pp. 503–510. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46654-5_55
Hua, W., Dai, F., Huang, L., et al.: HERO: human emotions recognition for realizing intelligent Internet of Things. IEEE Access 7, 1 (2019)
Zhang, Z.Y., Wang, R.Q., Wei, M.M.: Stack hybrid self-encoder facial expression recognition method. Comput. Eng. Appl. 55(13), 140–144 (2019)
Kommineni, J., Mandala, S., Sunar, M.S., et al.: Accurate computing of facial expression recognition using a hybrid feature extraction technique. J. Supercomput. 77(11), 1–26 (2020)
Kola, D.G.R., Samayamantula, S.K.: A novel approach for facial expression recognition using local binary pattern with adaptive window. Multimed. Tools Appl. 80(12), 1–20 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Miao, Z., Li, J., Lin, K. (2022). Facial Expression Recognition Based on Multi-feature Fusion. In: Shi, S., Ma, R., Lu, W. (eds) 6GN for Future Wireless Networks. 6GN 2021. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 439. Springer, Cham. https://doi.org/10.1007/978-3-031-04245-4_23
Download citation
DOI: https://doi.org/10.1007/978-3-031-04245-4_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-04244-7
Online ISBN: 978-3-031-04245-4
eBook Packages: Computer ScienceComputer Science (R0)