Skip to main content
SpringerLink
Log in

Emotion Analysis Platform to Investigate Student-Teacher Interaction

  • Conference paper
  • First Online:
Universal Access in Human-Computer Interaction. User and Context Diversity (HCII 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13309))

Included in the following conference series:

  • 1249 Accesses

Abstract

This paper introduces a system that enable the collection of relevant data related to the emotional behavior and attention of both student and professor during exams. It exploits facial coding techniques to enable the collection of a large amount of data from the automatic analysis of students and professors faces using video analysis, advanced techniques for gaze tracking based on deep Learning, and technologies and the principles related to the Affective Computing branch derived from the research of Paul Ekman. It provides tools that facilitates the interpretation of the collected data by means of a dashboard. A preliminary experiment has been carried out to investigate whether such a system may help in assessing the evaluation setting and support reflection on the evaluation processes in the light of the different situations, so as to improve the adoption of inclusive approaches. Results suggest that information provided by the proposed system can be helpful in assessing the setting and the evaluation process.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. D’Angelo, I., Del Bianco, N. (eds.): Inclusion at the University. Studies and Practices. Milano: FrancoAngeli (2019)

    Google Scholar 

  2. Giaconi, C.: Qualità della Vita e adulti con disabilità. Percorsi di ricerca e prospettive inclusive. FrancoAngeli, Milano (2015)

    Google Scholar 

  3. Perla, L.: Formare il docente alla didattica universitaria: il cantiere dell’innovazione. Riflessioni sull’innovazione didattica universitaria. Interventi alla tavola rotonda GEO (30 giugno 2017), 79–88 (2018)

    Google Scholar 

  4. Giaconi, C., Capellini, S.A., Del Bianco, N., Taddei, A., D’Angelo, I.: Study empowerment for inclusion. Educ. Sci. Soc.-Open Access 9(2), 166–183 (2019)

    Google Scholar 

  5. Pino, M., Mortari, L.: The inclusion of students with dyslexia in higher education: a systematic review using narrative synthesis. Dyslexia 20, 46–369 (2014)

    Article  Google Scholar 

  6. Rivera, C.J., Wood, C.L, James, M., Williams, S.: Improving study outcomes for college students with executive functioning challenges. Career Dev. Transition Exceptional Individuals, 42(3), 139–147 (2019)

    Google Scholar 

  7. Mengoni, M., et al.: Emotional Feedback in evaluation processes: case studies in the University context (in press)

    Google Scholar 

  8. Pentaraki, A., Burkholder, G.J.: Emerging evidence regarding the roles of emotional, behavioural, and cognitive aspects of student engagement in the online classroom. Eur. J. Open Distance E-learning 20(1), 1–21 (2017)

    Article  Google Scholar 

  9. D’Errico, F., Paciello, M., Cerniglia, L.: When emotions enhance students’ engagement in e-learning processes. J. e-Learning Knowl. Soc. 12(4), 9–23 (2016)

    Google Scholar 

  10. Romeo, F.P.: Investimento affettivo nei processi di insegnamento-apprendimento. Tre criteri per la didattica a distanza nelle emergenze. Open Journal of IUL University 2(1), 267–279 (2021)

    Google Scholar 

  11. Sroufe, L.A.: Lo sviluppo delle emozioni. I primi anni di vita. Milano, Raffaello Cortina (2000)

    Google Scholar 

  12. Le Boterf, G.: Costruire le competenze individuali e collettive. Guida, Napoli (2000)

    Google Scholar 

  13. Romeo, F.P.: Sollecitare la resilienza. Emergenze educative e strategie didattiche. Trento, Erickson (2020)

    Google Scholar 

  14. Mengoni, M., et al.: Evaluation strategies at University for students with Dyslexia: a pilot study supported by face emotion recognition. In: 2021 International Conference on Computational Science and Computational Intelligence (CSCI)

    Google Scholar 

  15. Nicholls, M.E.R., Loveless, K.M., Thomas, N.A., Loetscher, T., Churches, O.: Some participants may be better than others: sustained attention and motivation are higher early in semester. Quarterly J. Exp. Psychol. 68(1), 10–18 (2015)

    Google Scholar 

  16. J. Whitehill, Z. Serpell, Y. Lin, A. Foster and J. R. Movellan, The faces of engagement: automatic recognition of student engagement from facial expressions. In: IEEE Transactions on Affective Computing, vol. 5, no. 1, pp. 86–98, 1 January-March 2014. https://doi.org/10.1109/TAFFC.2014.2316163

  17. Tyng, C.M., Amin, H.U., Saad, M.N., Malik, A.S.: The influences of emotion on learning and memory. Front. Psychol. 8, 1454 (2017)

    Article  Google Scholar 

  18. Vuilleumier, P.: How brains beware: neural mechanisms of emotional attention. Trends Cogn. Sci. 9, 585–594 (2005). https://doi.org/10.1016/j.tics.2005.10.011

    Article  Google Scholar 

  19. Um, E., Plass, J.L., Hayward, E.O., Homer, B.D.: Emo-tional design in multimedia learning. J. Educ. Psychol. 104, 485–498 (2012). https://doi.org/10.1037/a0026609

    Article  Google Scholar 

  20. Jung, N., Wranke, C., Hamburger, K., Knauff, M.: How emotions affect logical reasoning: evidence from experiments with mood-manipulated participants, spider phobics, and people with exam anxie-ty. Front. Psychol. 5, 570 (2014). https://doi.org/10.3389/fpsyg.2014.00570

    Article  Google Scholar 

  21. Isen, A.M., Daubman, K.A., Nowicki, G.P.: Positive affect facilitates creative problem solving. J. Pers. Soc. Psychol. 52, 1122–1131 (1987). https://doi.org/10.1037/0022-3514.52.6.1122

    Article  Google Scholar 

  22. Ceccacci, S., Generosi, A., Cimini, G., Faggiano, S., Giraldi, L., Mengoni, M.: Facial coding as a mean to enable continuous monitoring of student's behavior in e-Learning. In: CEUR Workshop Proceedings, vol. 2817 (2021)

    Google Scholar 

  23. Generosi, A., Ceccacci, S., Mengoni, M.: A deep learning-based system to track and analyze customer behavior in retail store, in 2018 IEEE 8th International Conference on Consumer Electronics-Berlin (ICCE-Berlin) (2018). https://doi.org/10.1109/ICCE-Berlin.2018.8576169

  24. Ekman, P., Wallace, V.F.: Manual for the facial action coding system. Consulting Psychologists Press (1978)

    Google Scholar 

  25. Generosi, A., Ceccacci, S., Faggiano, S., Giraldi, L., Mengoni, M.: A toolkit for the automatic analysis of human behavior in HCI applications in the wild. Adv. Sci. Technol. Eng. Syst. 5(6), 185–192 (2020)

    Article  Google Scholar 

  26. Beck, J.: Engagement tracing: using response times to model student disengagement. In: Proceedings of International Conference on Artificial Intelligence and Education, pp. 88–95 (2005)

    Google Scholar 

  27. Johns, J., Woolf, B.: A dynamic mixture model to detect student motivation and proficiency. In: Proceedings of 21st Nat. Conf. Artif. Intell., pp. 2–8 (2006)

    Google Scholar 

  28. Hussain, M., Zhu, W., Zhang, W., Abidi, S.M.R.: Student engagement predictions in an e-learning system and their impact on student course assessment scores. Comput. Intell. Neurosci. (2018). https://doi.org/10.1155/2018/6347186

    Article  Google Scholar 

  29. Goldberg, B.S., Sottilare, R.A., Brawner, K.W., Holden, H.K.: Predicting learner engagement during well-defined and Ill-defined computer-based intercultural interactions. In: D’Mello, S., Graesser, A., Schuller, B., Martin, J.-C. (eds.) ACII 2011. LNCS, vol. 6974, pp. 538–547. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-24600-5_57

    Chapter  Google Scholar 

  30. Xiao, X., Wang, J.: Understanding and detecting divided attention in mobile MOOC learning. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, pp. 2411–2415 (2017)

    Google Scholar 

  31. Kaur, A., Mustafa, A., Mehta, L., Dhall, A.: Prediction and localization of student engagement in the wild. In: 2018 Digital Image Computing: Techniques and Applications (DICTA), Canberra, Australia, pp. 1–8 (2018). https://doi.org/10.1109/DICTA.2018.8615851

  32. Whitehill, J., Serpell, Z., Lin, Y., Foster, A., Movellan, J.R.: The faces of engagement: automatic recognition of student engagementfrom facial expressions. IEEE Trans. Affect. Comput. 5(1), 86–98 (2014). https://doi.org/10.1109/TAFFC.2014.2316163

    Article  Google Scholar 

  33. Hansen, D.W., Ji, Q.: In the eye of the beholder: a survey of models for eyes and gaze. IEEE Trans. Pattern Anal. Mach. Intell. 32(3), 478–500 (2010). https://doi.org/10.1109/TPAMI.2009.30

    Article  Google Scholar 

  34. Sigut, J., Sidha, S.A.: Iris center corneal reflection method for gaze tracking using visible light. IEEE Trans. Biomed. Eng. 58(2), 411–419 (2010). https://doi.org/10.1109/TBME.2010.2087330

    Article  Google Scholar 

  35. Valenti, R., Gevers, T.: Accurate eye center location and tracking using isophote curvature. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition (2008). https://doi.org/10.1109/CVPR.2008.4587529

  36. Wu, H., Kitagawa, Y., Wada, T., Kato, T., Chen, Q.: Tracking Iris contour with a 3D eye-model for gaze estimation. In: Yagi, Y., Kang, S.B., Kweon, I.S., Zha, H. (eds.) ACCV 2007. LNCS, vol. 4843, pp. 688–697. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-76386-4_65

    Chapter  Google Scholar 

  37. Zhang, X., Sugano, Y., Fritz, M., Bulling, A.: Appearance-based gaze estimation in the wild. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2019)-. https://doi.org/10.1109/CVPR.2015.7299081

  38. Krafka, K., et al.: Eye tracking for everyone. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016). https://doi.org/10.1109/CVPR.2016.239

  39. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems (2012). https://doi.org/10.1145/3065386

  40. Ceccacci, S., Generosi, A., Giraldi, L., Mengoni, M.: An emotion recognition system for monitoring shopping experience. In: Proceedings of the 11th PErvasive Technologies Related to Assistive Environments Conference, pp. 102–103, June 2018

    Google Scholar 

  41. Talipu, A., Generosi, A., Mengoni, M., Giraldi, L.: Evaluation of deep convolutional neural network architectures for emotion recognition in the wild. In: 2019 IEEE 23rd International Symposium on Consumer Technologies (ISCT), pp. 25–27. IEEE, June 2019

    Google Scholar 

  42. Rossi, P.G., Pentucci, M., Fedeli, L., Giannandrea, L., Pennazio, V.: From the informative feedback to the generative feedback. Educ. Sci. Soc.-Open Access 9(2), 83–107 (2018)

    Google Scholar 

  43. Rossi, P.G.: Didattica enattiva. Complessità, teorie dell'azione, professionalità docente: Complessità, teorie dell'azione, professionalità docente. Milano, FrancoAngeli (2011)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Industrial Engineering and Mathematical Sciences, Università Politecnica delle Marche, Ancona, Italy

    Andrea Generosi, Silvia Ceccacci & Maura Mengoni

  2. Department of Education, Cultural Heritage and Tourism, University of Macerata, Macerata, Italy

    Ilaria D’Angelo, Noemi Del Bianco & Catia Giaconi

  3. Emoj Srl, Ancona, Italy

    Gianluca Cimini

Authors
  1. Andrea Generosi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Silvia Ceccacci
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ilaria D’Angelo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Noemi Del Bianco
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Gianluca Cimini
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Maura Mengoni
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Catia Giaconi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Silvia Ceccacci .

Editor information

Editors and Affiliations

  1. Foundation for Research and Technology - Hellas (FORTH), Heraklion, Crete, Greece

    Margherita Antona

  2. University of Crete and Foundation for Research and Technology - Hellas (FORTH), Heraklion, Crete, Greece

    Constantine Stephanidis

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Generosi, A. et al. (2022). Emotion Analysis Platform to Investigate Student-Teacher Interaction. In: Antona, M., Stephanidis, C. (eds) Universal Access in Human-Computer Interaction. User and Context Diversity. HCII 2022. Lecture Notes in Computer Science, vol 13309. Springer, Cham. https://doi.org/10.1007/978-3-031-05039-8_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-05039-8_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-05038-1

  • Online ISBN: 978-3-031-05039-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation