Abstract
In this paper, we develop a method to monitor the emotional state of students and teachers during the study process based on facial expressions using machine learning and deep learning techniques. We describe the implementation of the created emotion detection model into the learning process as a web application to determine the emotional state of students and teachers in a digital learning environment in near real-time. Several training methods and models were examined using Python and Keras Tensorflow library and the results were compared against the classifiers Support Vector Machine, Random Forest, and Convolutional Neural Networks (CNN). The average recognition rate of the best model is about 96% and the proposed system is able to recognize emotions in near real-time.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ekman, P.: Basic emotions. In: Dalgleish, T., Power, M.J. (eds.) Handbook of Cognition and Emotion, pp. 45–60. Wiley (1999). https://doi.org/10.1002/0470013494.ch3
Ekman, P.: A methodological discussion of nonverbal behavior. J. Psychol. 43(1), 141–149 (1957). https://doi.org/10.1080/00223980.1957.9713059
Russell, J.: A circumplex model of affect. J. Pers. Soc. Psychol. 39(6), 1161–1178 (1980). https://doi.org/10.1037/h0077714
Scherer, K.: What are emotions? And how can they be measured? Soc. Sci. Inf. 44(4), 695–729 (2005). https://doi.org/10.1177/0539018405058216
Haq, S., Jackson, P.: multimodal emotion recognition. In: Wang, W. (ed.) Machine Audition: Principles, Algorithms and Systems, pp. 398–423. IGI Global (2011). https://doi.org/10.4018/978-1-61520-919-4.ch017
Liu, S., Wang, W.: The application study of learner's face detection and location in the teaching network system based on emotion recognition. In: IEEE 2010 Second International Conference on Networks Security, Wireless Communications and Trusted Computing, vol. 1, pp. 394–397. IEEE Computer Society (2010). https://doi.org/10.1109/NSWCTC.2010.97
Whitehill, J., Serpell, Z., Lin, Y.-C., Foster, A., Movellan, J.: The faces of engagement: automatic recognition of student engagement from facial expressions. IEEE Trans. Affect. Comput. 5(1), 86–98 (2014). https://doi.org/10.1109/TAFFC.2014.2316163
Ayvaz, U., Gürüler, H., Devrim, M.: Use of facial emotion recognition on e-learning systems. Inform. Technol. Learn. Tools 60(4), 95–104 (2017). https://doi.org/10.33407/itlt.v60i4.1743
Lasri, I., Solh, A., Belkacemi, M.: Facial emotion recognition of students using convolutional neural network. In: 2019 Third International Conference on Intelligent Computing in Data Sciences (ICDS), pp. 1–6 (2019). https://doi.org/10.1109/ICDS47004.2019.8942386
Sathik, M., Jonathan, S.G.: Effect of facial expressions on student’s comprehension recognition in virtual educational environments. Springerplus 2(1), 1–9 (2013). https://doi.org/10.1186/2193-1801-2-455
FER-2013. https://www.kaggle.com/datasets/msambare/fer2013. Accessed 29 May 2022
Serengil, S., Ozpinar, A.: LightFace: a hybrid deep face recognition framework. In: 2020 Innovations in Intelligent Systems and Applications Conference (ASYU), pp. 23–27. IEEE (2020). https://doi.org/10.1109/ASYU50717.2020.9259802
TensorFlow. https://www.tensorflow.org/. Accessed 29 May 2022
Keras. https://keras.io/. Accessed 29 May 2022
Google Colaboratory. https://colab.research.google.com/. Accessed 29 May 2022
Vapnik, V.: An overview of statistical learning theory. IEEE Trans. Neural Networks 10(5), 988–999 (1999). https://doi.org/10.1109/72.788640
Liaw, A., Wiener, M.: Classification and regression by random forest. R news 2(3), 18–22 (2002)
Deshpande, A.: A beginner’s guide to understanding convolutional neural networks. Part 2. http://tinyurl.com/y2cw5lmj. Accessed 21 Nov 2016
O’Malley, T., Bursztein, E., Long, J., Chollet, F., Jin, H., Invernizzi, L.: Keras Tuner (2019). https://github.com/keras-team/keras-tuner
Streamlit. https://streamlit.io/. Accessed 29 May 2022/05/29
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Ivleva, N., Pentel, A., Dunajeva, O., Juštšenko, V. (2023). Machine Learning Based Emotion Recognition in a Digital Learning Environment. In: Auer, M.E., Pachatz, W., Rüütmann, T. (eds) Learning in the Age of Digital and Green Transition. ICL 2022. Lecture Notes in Networks and Systems, vol 633. Springer, Cham. https://doi.org/10.1007/978-3-031-26876-2_38
Download citation
DOI: https://doi.org/10.1007/978-3-031-26876-2_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-26875-5
Online ISBN: 978-3-031-26876-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)