Abstract
This paper introduces a system that enable the collection of relevant data related to the emotional behavior and attention of both student and professor during exams. It exploits facial coding techniques to enable the collection of a large amount of data from the automatic analysis of students and professors faces using video analysis, advanced techniques for gaze tracking based on deep Learning, and technologies and the principles related to the Affective Computing branch derived from the research of Paul Ekman. It provides tools that facilitates the interpretation of the collected data by means of a dashboard. A preliminary experiment has been carried out to investigate whether such a system may help in assessing the evaluation setting and support reflection on the evaluation processes in the light of the different situations, so as to improve the adoption of inclusive approaches. Results suggest that information provided by the proposed system can be helpful in assessing the setting and the evaluation process.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
D’Angelo, I., Del Bianco, N. (eds.): Inclusion at the University. Studies and Practices. Milano: FrancoAngeli (2019)
Giaconi, C.: Qualità della Vita e adulti con disabilità. Percorsi di ricerca e prospettive inclusive. FrancoAngeli, Milano (2015)
Perla, L.: Formare il docente alla didattica universitaria: il cantiere dell’innovazione. Riflessioni sull’innovazione didattica universitaria. Interventi alla tavola rotonda GEO (30 giugno 2017), 79–88 (2018)
Giaconi, C., Capellini, S.A., Del Bianco, N., Taddei, A., D’Angelo, I.: Study empowerment for inclusion. Educ. Sci. Soc.-Open Access 9(2), 166–183 (2019)
Pino, M., Mortari, L.: The inclusion of students with dyslexia in higher education: a systematic review using narrative synthesis. Dyslexia 20, 46–369 (2014)
Rivera, C.J., Wood, C.L, James, M., Williams, S.: Improving study outcomes for college students with executive functioning challenges. Career Dev. Transition Exceptional Individuals, 42(3), 139–147 (2019)
Mengoni, M., et al.: Emotional Feedback in evaluation processes: case studies in the University context (in press)
Pentaraki, A., Burkholder, G.J.: Emerging evidence regarding the roles of emotional, behavioural, and cognitive aspects of student engagement in the online classroom. Eur. J. Open Distance E-learning 20(1), 1–21 (2017)
D’Errico, F., Paciello, M., Cerniglia, L.: When emotions enhance students’ engagement in e-learning processes. J. e-Learning Knowl. Soc. 12(4), 9–23 (2016)
Romeo, F.P.: Investimento affettivo nei processi di insegnamento-apprendimento. Tre criteri per la didattica a distanza nelle emergenze. Open Journal of IUL University 2(1), 267–279 (2021)
Sroufe, L.A.: Lo sviluppo delle emozioni. I primi anni di vita. Milano, Raffaello Cortina (2000)
Le Boterf, G.: Costruire le competenze individuali e collettive. Guida, Napoli (2000)
Romeo, F.P.: Sollecitare la resilienza. Emergenze educative e strategie didattiche. Trento, Erickson (2020)
Mengoni, M., et al.: Evaluation strategies at University for students with Dyslexia: a pilot study supported by face emotion recognition. In: 2021 International Conference on Computational Science and Computational Intelligence (CSCI)
Nicholls, M.E.R., Loveless, K.M., Thomas, N.A., Loetscher, T., Churches, O.: Some participants may be better than others: sustained attention and motivation are higher early in semester. Quarterly J. Exp. Psychol. 68(1), 10–18 (2015)
J. Whitehill, Z. Serpell, Y. Lin, A. Foster and J. R. Movellan, The faces of engagement: automatic recognition of student engagement from facial expressions. In: IEEE Transactions on Affective Computing, vol. 5, no. 1, pp. 86–98, 1 January-March 2014. https://doi.org/10.1109/TAFFC.2014.2316163
Tyng, C.M., Amin, H.U., Saad, M.N., Malik, A.S.: The influences of emotion on learning and memory. Front. Psychol. 8, 1454 (2017)
Vuilleumier, P.: How brains beware: neural mechanisms of emotional attention. Trends Cogn. Sci. 9, 585–594 (2005). https://doi.org/10.1016/j.tics.2005.10.011
Um, E., Plass, J.L., Hayward, E.O., Homer, B.D.: Emo-tional design in multimedia learning. J. Educ. Psychol. 104, 485–498 (2012). https://doi.org/10.1037/a0026609
Jung, N., Wranke, C., Hamburger, K., Knauff, M.: How emotions affect logical reasoning: evidence from experiments with mood-manipulated participants, spider phobics, and people with exam anxie-ty. Front. Psychol. 5, 570 (2014). https://doi.org/10.3389/fpsyg.2014.00570
Isen, A.M., Daubman, K.A., Nowicki, G.P.: Positive affect facilitates creative problem solving. J. Pers. Soc. Psychol. 52, 1122–1131 (1987). https://doi.org/10.1037/0022-3514.52.6.1122
Ceccacci, S., Generosi, A., Cimini, G., Faggiano, S., Giraldi, L., Mengoni, M.: Facial coding as a mean to enable continuous monitoring of student's behavior in e-Learning. In: CEUR Workshop Proceedings, vol. 2817 (2021)
Generosi, A., Ceccacci, S., Mengoni, M.: A deep learning-based system to track and analyze customer behavior in retail store, in 2018 IEEE 8th International Conference on Consumer Electronics-Berlin (ICCE-Berlin) (2018). https://doi.org/10.1109/ICCE-Berlin.2018.8576169
Ekman, P., Wallace, V.F.: Manual for the facial action coding system. Consulting Psychologists Press (1978)
Generosi, A., Ceccacci, S., Faggiano, S., Giraldi, L., Mengoni, M.: A toolkit for the automatic analysis of human behavior in HCI applications in the wild. Adv. Sci. Technol. Eng. Syst. 5(6), 185–192 (2020)
Beck, J.: Engagement tracing: using response times to model student disengagement. In: Proceedings of International Conference on Artificial Intelligence and Education, pp. 88–95 (2005)
Johns, J., Woolf, B.: A dynamic mixture model to detect student motivation and proficiency. In: Proceedings of 21st Nat. Conf. Artif. Intell., pp. 2–8 (2006)
Hussain, M., Zhu, W., Zhang, W., Abidi, S.M.R.: Student engagement predictions in an e-learning system and their impact on student course assessment scores. Comput. Intell. Neurosci. (2018). https://doi.org/10.1155/2018/6347186
Goldberg, B.S., Sottilare, R.A., Brawner, K.W., Holden, H.K.: Predicting learner engagement during well-defined and Ill-defined computer-based intercultural interactions. In: D’Mello, S., Graesser, A., Schuller, B., Martin, J.-C. (eds.) ACII 2011. LNCS, vol. 6974, pp. 538–547. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-24600-5_57
Xiao, X., Wang, J.: Understanding and detecting divided attention in mobile MOOC learning. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, pp. 2411–2415 (2017)
Kaur, A., Mustafa, A., Mehta, L., Dhall, A.: Prediction and localization of student engagement in the wild. In: 2018 Digital Image Computing: Techniques and Applications (DICTA), Canberra, Australia, pp. 1–8 (2018). https://doi.org/10.1109/DICTA.2018.8615851
Whitehill, J., Serpell, Z., Lin, Y., Foster, A., Movellan, J.R.: The faces of engagement: automatic recognition of student engagementfrom facial expressions. IEEE Trans. Affect. Comput. 5(1), 86–98 (2014). https://doi.org/10.1109/TAFFC.2014.2316163
Hansen, D.W., Ji, Q.: In the eye of the beholder: a survey of models for eyes and gaze. IEEE Trans. Pattern Anal. Mach. Intell. 32(3), 478–500 (2010). https://doi.org/10.1109/TPAMI.2009.30
Sigut, J., Sidha, S.A.: Iris center corneal reflection method for gaze tracking using visible light. IEEE Trans. Biomed. Eng. 58(2), 411–419 (2010). https://doi.org/10.1109/TBME.2010.2087330
Valenti, R., Gevers, T.: Accurate eye center location and tracking using isophote curvature. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition (2008). https://doi.org/10.1109/CVPR.2008.4587529
Wu, H., Kitagawa, Y., Wada, T., Kato, T., Chen, Q.: Tracking Iris contour with a 3D eye-model for gaze estimation. In: Yagi, Y., Kang, S.B., Kweon, I.S., Zha, H. (eds.) ACCV 2007. LNCS, vol. 4843, pp. 688–697. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-76386-4_65
Zhang, X., Sugano, Y., Fritz, M., Bulling, A.: Appearance-based gaze estimation in the wild. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2019)-. https://doi.org/10.1109/CVPR.2015.7299081
Krafka, K., et al.: Eye tracking for everyone. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016). https://doi.org/10.1109/CVPR.2016.239
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems (2012). https://doi.org/10.1145/3065386
Ceccacci, S., Generosi, A., Giraldi, L., Mengoni, M.: An emotion recognition system for monitoring shopping experience. In: Proceedings of the 11th PErvasive Technologies Related to Assistive Environments Conference, pp. 102–103, June 2018
Talipu, A., Generosi, A., Mengoni, M., Giraldi, L.: Evaluation of deep convolutional neural network architectures for emotion recognition in the wild. In: 2019 IEEE 23rd International Symposium on Consumer Technologies (ISCT), pp. 25–27. IEEE, June 2019
Rossi, P.G., Pentucci, M., Fedeli, L., Giannandrea, L., Pennazio, V.: From the informative feedback to the generative feedback. Educ. Sci. Soc.-Open Access 9(2), 83–107 (2018)
Rossi, P.G.: Didattica enattiva. Complessità, teorie dell'azione, professionalità docente: Complessità, teorie dell'azione, professionalità docente. Milano, FrancoAngeli (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Generosi, A. et al. (2022). Emotion Analysis Platform to Investigate Student-Teacher Interaction. In: Antona, M., Stephanidis, C. (eds) Universal Access in Human-Computer Interaction. User and Context Diversity. HCII 2022. Lecture Notes in Computer Science, vol 13309. Springer, Cham. https://doi.org/10.1007/978-3-031-05039-8_3
Download citation
DOI: https://doi.org/10.1007/978-3-031-05039-8_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-05038-1
Online ISBN: 978-3-031-05039-8
eBook Packages: Computer ScienceComputer Science (R0)