Engagement Classification in E-Learning Environments Through Eye Features Using Scaled Dot-Product Attention Convolutional Neural Network

Engagement can be defined as how individuals are involved and interact with a task that requires their attention and emotional state. Analyzing a student’s emotional reactions to four affective states, namely boredom, engagement, confusion, and frustration in e-learning environment can identify their learning behavior. By measuring student engagement and implementing the right prevention, such as rearranging the learning design or building an effective feedback system, students’ learning intake can be increased. A person’s actions and intentions can be identified using eye movement without knowing what the person is looking at. Therefore, this study utilized eye landmarks and gaze features extracted with OpenFace as the input to a Convolutional Neural Network (CNN)-based classifier that enhances the state-of-the-art model with the Scaled Dot-Product Attention mechanism to capture important part of the data. The performance of our model was compared with the state-of-the-art model and machine learning models on three different landmark points amounts, namely 56, 20, and 10 landmark points. The proposed model obtained accuracies of 96.89% on students’ affective states classification using 20 eye landmark points and 95.81% on engagement levels classification using 56 eye landmark points, while the Support Vector Machine obtained 81.4% and 78.73% accuracy on both objectives respectively, and the Dense Neural Network obtained 92.66% and 93.65% accuracy on both objectives respectively. It can be concluded that adding an attention mechanism can boost the model’s performance. Besides, similar accuracy was obtained using eye landmarks of 56 points and 20 points, but reduced more significantly on 10 points.