Abstract
Intelligent tutoring systems have great potential in personalizing the educational experience by processing some key features from the user and educational task to optimize learning, engagement, or other performance measures. This paper presents an approach that uses a combination of facial features from the user of an educational app and contextual features about the progress of the task to predict key events related to user engagement. Our approach trains Gaussian Mixture Models from automatically processed screen-capture videos and propagates the probability of events over the course of an activity. Results show the advantage of including contextual features in addition to facial features when predicting these engagement-related events, which can be used to intervene appropriately during an educational activity.
Supported in part by NSF Grant #IIS-1939047.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
We tried varying \(\lambda \) over time, but that did not improve results.
References
Agarwal, M., Mostow, J.: Semi-supervised learning to perceive children’s affective states in a tablet tutor. In: Tenth Symposium on Educational Advances in Artificial Intelligence(EAAI). New York, NY (2020)
Amos, B., Ludwiczuk, B., Satyanarayanan, M.: Openface: a general-purpose face recognition library with mobile applications. CMU School Comput. Sc. 6 (2016)
Bogina, V., Kuflik, T., Mokryn, O.: Learning item temporal dynamics for predicting buying sessions. In: Proceedings of the 21st International Conference on Intelligent User Interfaces (2016)
Brown, L., Kerwin, R., Howard, A.M.: Applying behavioral strategies for student engagement using a robotic educational agent. In: IEEE International Conference on Systems, Man, and Cybernetics. IEEE (2013)
Burgoon, J.K., Buller, D.B., Hale, J.L., de Turck, M.A.: Relational messages associated with nonverbal behaviors. Human Commun. Res. 10(3), 351–378 (1984)
Ekman, P., Friesen, W.V.: Facial action coding systems. Consulting Psychologists Press (1978)
Fard, M.J., Wang, P., Chawla, S., Reddy, C.K.: A bayesian perspective on early stage event prediction in longitudinal data. IEEE Trans. Knowl. Data Eng. 28(12), 3126–3139 (2016)
Fawaz, H.I., Forestier, G., Weber, J., Idoumghar, L., Muller, P.A.: Deep learning for time series classification: a review. Data Min. Knowl. Discov. 33(4), 917–963 (2019)
Hernández, Y., Noguez, J., Sucar, E., Arroyo-Figueroa, G.: A probabilistic model of affective behavior for intelligent tutoring systems. In: Gelbukh, A., de Albornoz, Á., Terashima-Marín, H. (eds.) MICAI 2005. LNCS (LNAI), vol. 3789, pp. 1175–1184. Springer, Heidelberg (2005). https://doi.org/10.1007/11579427_119
Johns, J., Woolf, B.: A dynamic mixture model to detect student motivation and proficiency. In: Proceedings of the National Conference on Artificial Intelligence, vol. 21. Menlo Park, CA (2006)
Leite, I., Pereira, A., Castellano, G., Mascarenhas, S., Martinho, C., Paiva, A.: Modelling empathy in social robotic companions. In: Ardissono, L., Kuflik, T. (eds.) UMAP 2011. LNCS, vol. 7138, pp. 135–147. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-28509-7_14
Levinkov, E., Fritz, M.: Sequential bayesian model update under structured scene prior for semantic road scenes labeling. In: Proceedings of the IEEE International Conference on Computer Vision (2013)
McReynolds, A.A., Naderzad, S.P., Goswami, M., Mostow, J.: Toward Learning at Scale in Developing Countries: Lessons from the Global Learning XPRIZE Field Study. In: Learning @ Scale. ACM (2020)
Mori, U., Mendiburu, A., Keogh, E., Lozano, J.A.: Reliable early classification of time series based on discriminating the classes over time. Data Min. Knowl. Discov. 31(1), 233–263 (2016). https://doi.org/10.1007/s10618-016-0462-1
Perez, Y.H., Gamboa, R.M., Ibarra, O.M.: Modeling affective responses in intelligent tutoring systems. In: IEEE International Conference on Advanced Learning Technologies, 2004 Proceedings. IEEE (2004)
Sanghvi, J., Castellano, G., Leite, I., Pereira, A., McOwan, P.W., Paiva, A.: Automatic analysis of affective postures and body motion to detect engagement with a game companion. In: Proceedings of the 6th International Conference on Human-Robot Interaction (2011)
Saxena, M., Pillai, R.K., Mostow, J.: Relating children’s automatically detected facial expressions to their behavior in robotutor. In: Thirty-Second AAAI Conference on Artificial Intelligence (2018)
Spaulding, S., Gordon, G., Breazeal, C.: Affect-aware student models for robot tutors. In: Proceedings of the International Conference on Autonomous Agents and Multiagent Systems (2016)
Woolfolk, A.E., Brooks, D.M.: Chapter 5: nonverbal communication in teaching. Review of research in education 10(1), 103–149 (1983)
Xing, Z., Pei, J., Philip, S.Y.: Early classification on time series. Knowl. Inf. Syst. 31(1), 105–127 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Kaushik, R., Simmons, R. (2021). Early Prediction of Student Engagement-Related Events from Facial and Contextual Features. In: Li, H., et al. Social Robotics. ICSR 2021. Lecture Notes in Computer Science(), vol 13086. Springer, Cham. https://doi.org/10.1007/978-3-030-90525-5_26
Download citation
DOI: https://doi.org/10.1007/978-3-030-90525-5_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-90524-8
Online ISBN: 978-3-030-90525-5
eBook Packages: Computer ScienceComputer Science (R0)