Abstract
In the current paper, we present a solution to recognise emotions from facial expressions, based on two publicly available datasets: AffectNet and Fer2013. Our model was trained according to convolutional neural networks (CNN) that are widely used. Towards we reviewed and analysed recent research in the field of deep learning and emotion recognition, and we propose a CNN architecture that includes several layers of convolutions and pooling, followed by fully connected layers. We evaluate the performance of our model on the two datasets and compare it with the state-of-the-art methods. Although our model did not outperform the existing methods in terms of accuracy, our results show that it achieved competitive performance and provides an alternative approach to the problem. The impact of different parameters, such as batch size, learning rate, and dropout, on the model's accuracy was evaluated. This paper contributes to the field of emotion recognition and deep learning by providing a comprehensive analysis of the effectiveness of CNNs for facial emotion recognition and proposing an efficient and accurate model, as well as identifying areas for further improvement in this research field.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Calvo, R.A., D’Mello, S., Gratch, J., Kappas, A. (eds.) The Oxford handbook of affective computing. Oxford University Press, Oxford, New York (2015). ISBN: 978-0-19-994223-7
Cai, Y., Li, X., Li, J.: Emotion recognition using different sensors emotion models, methods and datasets: a comprehensive review. Sensors 23(5), 2455 (2023). https://doi.org/10.3390/s23052455. Feb.
Li, S., Deng, W.: Deep facial expression recognition: a survey. IEEE Trans. Affective Comput. 13(3), 1195–1215 (2022). https://doi.org/10.1109/TAFFC.2020.2981446. Jul.
Pal, S., Mukhopadhyay, S., Suryadevara, N.: Development and progress in sensors and technologies for human emotion recognition. Sensors 21(16), 5554 (2021). https://doi.org/10.3390/s21165554. Aug.
Heo, S., Kwon, S., Lee, J.: Stress detection with single PPG sensor by orchestrating multiple denoising and peak-detecting methods. IEEE Access 9, 47777–47785 (2021). https://doi.org/10.1109/ACCESS.2021.3060441
Spezialetti, M., Placidi, G., Rossi, S.: Emotion recognition for human-robot interaction: recent advances and future perspectives. Front. Robot. AI 7, 532279 (2020). https://doi.org/10.3389/frobt.2020.532279. Dec.
Mollahosseini, A., Hasani, B., Mahoor, M.H.: AffectNet: a facial expression database for valence and arousal recognition. IEEE Transactions on Affective Computing 10(1), 18–31 (2016)
FER2013 Dataset: available (June 23, 2023) in https://datasets.activeloop.ai/docs/ml/datasets/fer2013-dataset
AffectNet Benchmark: (Facial Expression Recognition (FER))’. https://paperswithcode.com/sota/facial-expression-recognition-on-affectnet, accessed 20 Apr. 2023
Goodfellow, I., Bengio, Y., Courville, A.: Deep learning. In: Adaptive computation and machine learning. The MIT Press, Cambridge, Massachusetts (2016)
Ekman, P., Friesen, W.V.: Facial Action Coding System (Jan. 14, 2019). https://doi.org/10.1037/t27734-000
Camras, L.A., Allison, K.: Children’s understanding of emotional facial expressions and verbal labels. J Nonverbal Behav 9(2), 84–94 (1985). https://doi.org/10.1007/BF00987140
Kanade, T., Cohn, J.F., Tian, Y.: Comprehensive database for facial expression analysis. In: Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580), pp. 46–53. Grenoble, France: IEEE Comput. Soc (2000). https://doi.org/10.1109/AFGR.2000.840611
Nojavanasghari, B., Baltrušaitis, T., Hughes, C.E., Morency, L.-P.: EmoReact: a multimodal approach and dataset for recognizing emotional responses in children. In: Proceedings of the 18th ACM International Conference on Multimodal Interaction, pp. 137–144. ACM, Tokyo Japan (Oct. 2016). https://doi.org/10.1145/2993148.2993168
Livingstone, S.R., Russo, F.A.: The ryerson audio-visual database of emotional speech and song (RAVDESS): a dynamic, multimodal set of facial and vocal expressions in North American English. PLoS ONE 13(5), e0196391 (2018). https://doi.org/10.1371/journal.pone.0196391. May
Huang, Y., Chen, F., Lv, S., Wang, X.: Facial Expression Recognition: A Survey. Symmetry 11(10), 1189 (2019). https://doi.org/10.3390/sym11101189. Sep.
Koujan, M.R., Alharbawee, L., Giannakakis, G., Pugeault, N., Roussos, A.: Real-time facial expression recognition. “In The Wild’’ by Disentangling 3D Expression from Identity (2020). https://doi.org/10.48550/ARXIV.2005.05509
Li, C., Yang, M., Zhang, Y., Lai, K.W.: An intelligent mental health identification method for college students: a mixed-method study. IJERPH 19(22), 14976 (2022). https://doi.org/10.3390/ijerph192214976. Nov.
Bozinovski, S.: Reminder of the first paper on transfer learning in neural networks, 1976. IJCAI 44(3) (Sep. 2020). https://doi.org/10.31449/inf.v44i3.2828
Mizgajski, J., Morzy, M.: Affective recommender systems in online news industry: how emotions influence reading choices. User Model User-Adap Inter 29(2), 345–379 (2019). https://doi.org/10.1007/s11257-018-9213-x. Apr.
Zhang, Z.: Improved adam optimizer for deep neural networks. In: 2018 IEEE/ACM 26th International Symposium on Quality of Service (IWQoS), pp. 1–2. IEEE, Banff, AB, Canada (Jun. 2018). https://doi.org/10.1109/IWQoS.2018.8624183
Simonyan, K., Zisserman, A.: Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv, (Apr. 10, 2015). Accessed: 14 May 2023. [Online]. Available: http://arxiv.org/abs/1409.1556
Feidakis, M.: A review of emotion-aware systems for e-learning in virtual environments. In: Formative Assessment, Learning Data Analytics and Gamification, pp. 217–242. Elsevier (2016). https://doi.org/10.1016/B978-0-12-803637-2.00011-7
Kalsum, T., Anwar, S.M., Majid, M., Khan, B., Ali, S.M.: Emotion recognition from facial expressions using hybrid feature descriptors. IET Image Processing 12(6), 1004–1012 (2018). https://doi.org/10.1049/iet-ipr.2017.0499. Jun.
Acknowledgment
This work is funded by the University of West Attica
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Feidakis, M., Maros, G., Antikantzidis, A. (2023). A Deep Learning Model to Recognise Facial Emotion Expressions. In: Kabassi, K., Mylonas, P., Caro, J. (eds) Novel & Intelligent Digital Systems: Proceedings of the 3rd International Conference (NiDS 2023). NiDS 2023. Lecture Notes in Networks and Systems, vol 783. Springer, Cham. https://doi.org/10.1007/978-3-031-44097-7_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-44097-7_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44096-0
Online ISBN: 978-3-031-44097-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)