Skip to main content
SpringerLink
Log in

A Review of Automatic Detection of Learner States in Four Typical Learning Scenarios

  • Conference paper
  • First Online:
Adaptive Instructional Systems (HCII 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13332))

Included in the following conference series:

  • 690 Accesses

Abstract

Artificial intelligence technology has already been applied in the education scene, and the automatic detecting technology of learning state has attracted the attention of many researchers. This paper summarizes the main types of learning state that researchers pay attention to at present, including affect, engagement, attention, and cognitive load. Based on four typical learning scenarios: computer-based learning, mobile learning, traditional classroom-based learning, and individual computer-free learning, this paper discusses the shortcomings and development trends of detecting hardware and methods used in this field, and the social problems in obtaining a large amount of personal privacy data.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 64.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 84.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Data Availability

The data used to support the findings of this study are available from the corresponding author upon request.

References

  1. D’Mello, S., Taylor, R., Graesser, A.: Monitoring affective trajectories during complex learning. In: Proceedings of the 29th Annual Meeting of the Cognitive Science Society, pp. 203–208, 01 January 2007

    Google Scholar 

  2. Boekaerts, M.: Cognitive load and self-regulation: attempts to build a bridge. Learn. Instr. 51, 90–97 (2017)

    Article  Google Scholar 

  3. Imtinan, U., Chang, V., Issa, T.: Common mobile learning characteristics-an analysis of mobile learning models and frameworks. In: IADIS International Conference on Mobile Learning 2013, ML 2013, Lisbon, Portugal, 14–16 March 2013, pp. 3–11. IADIS (2013)

    Google Scholar 

  4. Boyle, E.A., Hainey, T., Connolly, T.M., et al.: An update to the systematic literature review of empirical evidence of the impacts and outcomes of computer games and serious games. Comput. Educ. 94, 178–192 (2016)

    Article  Google Scholar 

  5. Easterday, M.W., Aleven, V., Scheines, R., et al.: Using tutors to improve educational games: a cognitive game for policy argument. J. Learn. Sci. 26(2), 226–276 (2017)

    Article  Google Scholar 

  6. Dede, C.: Immersive interfaces for engagement and learning. Science 323(5910), 66–69 (2009). (in English)

    Article  Google Scholar 

  7. Sun, H.J., Huang, M.X., Ngai, G., et al.: Nonintrusive multimodal attention detection. In: 7th International Conference on Advances in Computer-Human Interactions, ACHI 2014, Barcelona, Spain, 23–27 March 2014, pp. 192–199. International Academy, Research and Industry Association, IARIA (2014)

    Google Scholar 

  8. Cabada, R.Z., Estrada, M.L.B., Hernández, F.G., et al.: An affective and Web 3.0-based learning environment for a programming language. Telemat. Inform. 35(3), 611–628 (2018)

    Article  Google Scholar 

  9. Caballé, S.: Towards a multi-modal emotion-awareness e-learning system. In: 2015 International Conference on Intelligent Networking and Collaborative Systems, pp. 280–287. IEEE (2015)

    Google Scholar 

  10. Goh, F., Carroll, A., Gillies, R.M.: A review of the use of portable technologies as observational aids in the classroom. Inf. Learn. Sci. 120(3/4), 228–241 (2019)

    Article  Google Scholar 

  11. Yadegaridehkordi, E., Noor, N.F.B.M., Ayub, M.N.B., et al.: Affective computing in education: a systematic review and future research. Comput. Educ. 142, 103649 (2019)

    Google Scholar 

  12. Keele, S.: Guidelines for performing systematic literature reviews in software engineering. Technical report, ver. 2.3 EBSE Technical Report. EBSE (2007)

    Google Scholar 

  13. Elfenbein, H.A., Ambady, N.: On the universality and cultural specificity of emotion recognition: a meta-analysis. Psychol. Bull. 128(2), 203–235 (2002). https://doi.org/10.1037/0033-2909.128.2.203. (in English)

    Article  Google Scholar 

  14. Dmello, S., Calvo, R.A.: Beyond the basic emotions: what should affective computing compute? In: 31st Annual CHI Conference on Human Factors in Computing Systems: CHI EA 2013, Paris, France, 27 April–2 May 2013, vol. 2013, pp. 2287–2294. Association for Computing Machinery (2013)

    Google Scholar 

  15. Russell, J.A.: Core affect and the psychological construction of emotion. Psychol. Rev. 110(1), 145–172 (2003). (in English)

    Article  Google Scholar 

  16. Graesser, A., McDaniel, B., Chipman, P., et al.: Detection of emotions during learning with AutoTutor. In: Proceedings of the 28th Annual Meetings of the Cognitive Science Society, pp. 285–290. Citeseer (2006)

    Google Scholar 

  17. Silvia, P.J.: Looking past pleasure: anger, confusion, disgust, pride, surprise, and other unusual aesthetic emotions. Psychol. Aesthet. Creat. Arts 3(1), 48 (2009)

    Article  Google Scholar 

  18. Pekrun, R.: Academic emotions. Handb. Motiv. Sch. 2, 120–144 (2016)

    Google Scholar 

  19. Calvo, R.A., D’Mello, S., Gratch, J.M., et al.: The Oxford Handbook of Affective Computing. Oxford University Press, USA (2015)

    Book  Google Scholar 

  20. Woolf, B., Burleson, W., Arroyo, I., et al.: Affect-aware tutors: recognising and responding to student affect. Int. J. Learn. Technol. 4(3–4), 129–164 (2009)

    Article  Google Scholar 

  21. Tyng, C.M., Amin, H.U., Saad, M.N.M., Malik, A.S.: The influences of emotion on learning and memory. Front. Psychol. 8, 1454 (2017). https://doi.org/10.3389/fpsyg.2017.01454. (in English)

    Article  Google Scholar 

  22. D’Mello, S., Graesser, A.: Dynamics of affective states during complex learning. Learn. Instr. 22(2), 145–157 (2012). https://doi.org/10.1016/j.learninstruc.2011.10.001

    Article  Google Scholar 

  23. Clore, G.L., Huntsinger, J.R.: How emotions inform judgment and regulate thought. Trends Cogn. Sci. 11(9), 393–399 (2007). https://doi.org/10.1016/j.tics.2007.08.005

    Article  Google Scholar 

  24. Pekrun, R.: Emotions as drivers of learning and cognitive development. In: Calvo, R.A., D’Mello, S.K. (eds.) New Perspectives on Affect and Learning Technologies, pp. 23–39. Springer, New York (2011). https://doi.org/10.1007/978-1-4419-9625-1_3

    Chapter  Google Scholar 

  25. D’Mello, S., Lehman, B., Pekrun, R., Graesser, A.: Confusion can be beneficial for learning. Learn. Instr. 29, 153–170 (2014). https://doi.org/10.1016/j.learninstruc.2012.05.003

    Article  Google Scholar 

  26. Peters, C., Castellano, G., de Freitas, S.: An exploration of user engagement in HCI. In: Proceedings of the International Workshop on Affective-Aware Virtual Agents and Social Robots, p. 9. ACM (2009)

    Google Scholar 

  27. Christenson, S.L., Reschly, A.L., Wylie, C. (eds.): Handbook of Research on Student Engagement. Springer, Boston (2012). https://doi.org/10.1007/978-1-4614-2018-7

    Book  Google Scholar 

  28. Kamath, A., Biswas, A., Balasubramanian, V.: A crowdsourced approach to student engagement recognition in e-learning environments. In: IEEE Winter Conference on Applications of Computer Vision, WACV 2016, Lake Placid, NY, USA, 7–10 March 2016. Institute of Electrical and Electronics Engineers Inc. (2016)

    Google Scholar 

  29. Kahn, W.A.: Psychological conditions of personal engagement and disengagement at work. Acad. Manag. J. 33(4), 692–724 (1990)

    Article  Google Scholar 

  30. Papadopoulos, F., Corrigan, L.J., Jones, A., et al.: Learner modelling and automatic engagement recognition with robotic tutors. In: 2013 5th Humaine Association Conference on Affective Computing and Intelligent Interaction, ACII 2013, Geneva, Switzerland, 2–5 September 2013, pp. 740–744. IEEE Computer Society (2013)

    Google Scholar 

  31. Czikszentmihalyi, M.: Flow: The Psychology of Optimal Experience. Harper & Row, New York (1990)

    Google Scholar 

  32. Geisler, W.S., Cormack, L.K.: Models of overt attention. In: Oxford Handbook of Eye Movements, pp. 439–454 (2011)

    Google Scholar 

  33. Stanley, D.: Measuring attention using Microsoft Kinect (2013)

    Google Scholar 

  34. Howard-Jones, P., Murray, S.: Ideational productivity, focus of attention, and context. Creat. Res. J. 15(2–3), 153–166 (2003)

    Article  Google Scholar 

  35. Killingsworth, M.A., Gilbert, D.T.: A wandering mind is an unhappy mind. Science 330(6006), 932 (2010)

    Article  Google Scholar 

  36. Smilek, D., Carriere, J.S.A., Allan Cheyne, J.: Out of mind, out of sight: eye blinking as indicator and embodiment of mind wandering. Psychol. Sci. 21(6), 786–789 (2010). https://doi.org/10.1177/0956797610368063

    Article  Google Scholar 

  37. Randall, J.G., Oswald, F.L., Beier, M.E.: Mind-wandering, cognition, and performance: a theory-driven meta-analysis of attention regulation. Psychol Bull 140(6), 1411 (2014)

    Article  Google Scholar 

  38. Smallwood, J., Davies, J.B., Heim, D., et al.: Subjective experience and the attentional lapse: task engagement and disengagement during sustained attention. Conscious. Cogn. 13(4), 657–690 (2004)

    Article  Google Scholar 

  39. Smallwood, J., Schooler, J.W.: The restless mind (2013)

    Google Scholar 

  40. Feng, S., D’Mello, S., Graesser, A.C.: Mind wandering while reading easy and difficult texts. Psychon. Bull. Rev. 20(3), 586–592 (2012). https://doi.org/10.3758/s13423-012-0367-y

    Article  Google Scholar 

  41. Sweller, J.: Human cognitive architecture. In: Handbook of Research on Educational Communications and Technology, pp. 369–381 (2008)

    Google Scholar 

  42. Paas, F., Renkl, A., Sweller, J.: Cognitive load theory: instructional implications of the interaction between information structures and cognitive architecture. Instr. Sci. 32(1/2), 1–8 (2004). https://doi.org/10.1023/B:TRUC.0000021806.17516.d0

    Article  Google Scholar 

  43. Ozcinar, Z.: The topic of instructional design in research journals: a citation analysis for the years 1980–2008. Australas. J. Educ. Technol. 25(4) (2009)

    Google Scholar 

  44. Paas, F., Tuovinen, J.E., Tabbers, H., Van Gerven, P.W.M.: Cognitive load measurement as a means to advance cognitive load theory. Educ. Psychol. 38(1), 63–71 (2010). https://doi.org/10.1207/S15326985EP3801_8

    Article  Google Scholar 

  45. Skulmowski, A., Rey, G.D.: Measuring cognitive load in embodied learning settings. Front. Psychol. 8, 1191 (2017)

    Article  Google Scholar 

  46. Jeon, J., Park, J.-C., Jo, Y., et al.: A real-time facial expression recognizer using deep neural network. In: 10th International Conference on Ubiquitous Information Management and Communication, IMCOM 2016, Danang, Vietnam, 4–6 January 2016. Association for Computing Machinery, Inc., ACM SIGAPP (2016)

    Google Scholar 

  47. Verma, B., Choudhary, A.: A framework for driver emotion recognition using deep learning and Grassmann manifolds. In: 21st IEEE International Conference on Intelligent Transportation Systems, ITSC 2018, Maui, HI, USA, 4–7 November 2018, vol. 2018, pp. 1421–1426. Institute of Electrical and Electronics Engineers Inc. (2018)

    Google Scholar 

  48. Hasani, B., Mahoor, M.H.: Facial expression recognition using enhanced deep 3D convolutional neural networks. In: 30th IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2017, Honolulu, HI, USA, 21–26 July 2017, vol. 2017, pp. 2278–2288. IEEE Computer Society (2017)

    Google Scholar 

  49. Sawyer, R., Smith, A., Rowe, J., et al.: Enhancing student models in game-based learning with facial expression recognition. In: 25th ACM International Conference on User Modeling, Adaptation, and Personalization, UMAP 2017, Bratislava, Slovakia, 9–12 July 2017, pp. 192–201. Association for Computing Machinery, Inc. (2017)

    Google Scholar 

  50. McDaniel, B.T., D’Mello, S.K., King, B.G., et al.: Facial features for affective state detection in learning environments. In: Proceedings of the 29th Annual Cognitive Science Society (2007)

    Google Scholar 

  51. Liu, S., Wang, W.: The application study of learner’s face detection and location in the teaching network system based on emotion recognition. In: 2nd International Conference on Networks Security, Wireless Communications and Trusted Computing, NSWCTC 2010, Wuhan, Hubei, China, 24–25 April 2010, vol. 1, pp. 394–397. IEEE Computer Society (2010)

    Google Scholar 

  52. Bosch, N., et al.: Using video to automatically detect learner affect in computer-enabled classrooms. ACM Trans. Interact. Intell. Syst. 6(2), 1–26 (2016). https://doi.org/10.1145/2946837

    Article  Google Scholar 

  53. Kapoor, A., Picard, R.W.: Multimodal affect recognition in learning environments. In: 13th ACM International Conference on Multimedia, MM 2005, Singapore, Singapore, 6–11 November 2005, pp. 677–682. Association for Computing Machinery (2005)

    Google Scholar 

  54. Psaltis, A., Apostolakis, K.C., Dimitropoulos, K., et al.: Multimodal student engagement recognition in prosocial games. IEEE Trans. Games 10(3), 292–303 (2018)

    Article  Google Scholar 

  55. Yang, D., Abeer Alsadoon, P.W.C., Prasad, A.K., Singh, A.E.: An emotion recognition model based on facial recognition in virtual learning environment. Procedia Comput. Sci. 125, 2–10 (2018). https://doi.org/10.1016/j.procs.2017.12.003

    Article  Google Scholar 

  56. Kaur, A., Mustafa, A., Mehta, L., et al.: Prediction and localization of student engagement in the wild. In: 2018 International Conference on Digital Image Computing: Techniques and Applications, DICTA 2018, Canberra, ACT, Australia, 10–13 December 2018, pp. APRS; Australian Government, Department of Defence, Defence Science and Technology Group; Canon Information Systems Research Australia Pty Ltd (CiSRA); IAPRO; IEEE; UNSW Canberra. Institute of Electrical and Electronics Engineers Inc. (2018)

    Google Scholar 

  57. Monkaresi, H., Bosch, N., Calvo, R.A., et al.: Automated detection of engagement using video-based estimation of facial expressions and heart rate. IEEE Trans. Affect. Comput. 8(1), 15–28 (2017)

    Article  Google Scholar 

  58. Asteriadis, S., Karpouzis, K., Kollias, S.: Visual focus of attention in non-calibrated environments using gaze estimation. Int. J. Comput. Vision 107(3), 293–316 (2014)

    Article  MathSciNet  Google Scholar 

  59. Yang, F., Jiang, Z., Wang, C., et al.: Student eye gaze tracking during MOOC teaching. In: 2018 Joint 10th International Conference on Soft Computing and Intelligent Systems (SCIS) and 19th International Symposium on Advanced Intelligent Systems (ISIS), pp. 875–880 (2018)

    Google Scholar 

  60. Fridman, L., Reimer, B., Mehler, B., et al.: Cognitive load estimation in the wild. In: 2018 CHI Conference on Human Factors in Computing Systems, CHI 2018, Montreal, QC, Canada, 21–26 April 2018, vol. 2018. Association for Computing Machinery, ACM SIGCHI (2018)

    Google Scholar 

  61. Chen, S., Epps, J., Chen, F.: An investigation of pupil-based cognitive load measurement with low cost infrared webcam under light reflex interference. In: 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2013, Osaka, Japan, 3–7 July 2013, pp. 3202–3205. Institute of Electrical and Electronics Engineers Inc. (2013)

    Google Scholar 

  62. Bixler, R., D’Mello, S.: Detecting boredom and engagement during writing with keystroke analysis, task appraisals, and stable traits. In: 18th International Conference on Intelligent User Interfaces, IUI 2013, Santa Monica, CA, USA, 19–22 March 2013, pp. 225–233. Association for Computing Machinery (2013)

    Google Scholar 

  63. Wang, J., Huang, M.X., Ngai, G., et al.: Are you stressed? Your eyes and the mouse can tell. In: 7th International Conference on Affective Computing and Intelligent Interaction, ACII 2017, San Antonio, TX, USA, 23–26 October 2017, vol. 2018, pp. 222–228. Institute of Electrical and Electronics Engineers Inc. (2017)

    Google Scholar 

  64. Lali, P., Naghizadeh, M., Nasrollahi, H., et al.: Your mouse can tell about your emotions. In: 4th International Conference on Computer and Knowledge Engineering, ICCKE 2014, Azadi Square, Mashhad, Iran, 29–30 October 2014, pp. 47–51. Institute of Electrical and Electronics Engineers Inc. (2014)

    Google Scholar 

  65. Shikder, R., Rahaman, S., Afroze, F., et al.: Keystroke/mouse usage based emotion detection and user identification. In: 2017 International Conference on Networking, Systems and Security, NSysS 2017, Dhaka, Bangladesh, 5–8 January 2017, pp. 96–104. Institute of Electrical and Electronics Engineers Inc. (2017)

    Google Scholar 

  66. Fu, Y., Leong, H.V., Ngai, G., Huang, M.X., Chan, S.C.F.: Physiological mouse: toward an emotion-aware mouse. Univ. Access Inf. Soc. 16(2), 365–379 (2016). https://doi.org/10.1007/s10209-016-0469-9

    Article  Google Scholar 

  67. Saha, S., Datta, S., Konar, A., et al.: A study on emotion recognition from body gestures using Kinect sensor. In: 3rd International Conference on Communication and Signal Processing, ICCSP 2014, Melmaruvathur, Tamil Nadu, India, 3–5 April 2014, pp. 56–60. Institute of Electrical and Electronics Engineers Inc. (2014)

    Google Scholar 

  68. Piana, S., Staglianò, A., Odone, F., Camurri, A.: Adaptive body gesture representation for automatic emotion recognition. ACM Trans. Interact. Intell. Syst. 6(1), 1–31 (2016). https://doi.org/10.1145/2818740

    Article  Google Scholar 

  69. Kaza, K., et al.: Body motion analysis for emotion recognition in serious games. In: Antona, M., Stephanidis, C. (eds.) UAHCI 2016. LNCS, vol. 9738, pp. 33–42. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-40244-4_4

    Chapter  Google Scholar 

  70. Wei, H., Scanlon, P., Li, Y., et al.: Real-time head nod and shake detection for continuous human affect recognition. In: 2013 14th International Workshop on Image Analysis for Multimedia Interactive Services, WIAMIS 2013, Paris, France, 3–5 July 2013. IEEE Computer Society (2013)

    Google Scholar 

  71. Mota, S., Picard, R.W.: Automated posture analysis for detecting learner’s interest level. In: Conference on Computer Vision and Pattern Recognition Workshop, CVPRW 2003, Madison, WI, USA, 16–22 June 2003, vol. 5. IEEE Computer Society (2003)

    Google Scholar 

  72. Bixler, R., Blanchard, N., Garrison, L., et al.: Automatic detection of mind wandering during reading using gaze and physiology. In: ACM International Conference on Multimodal Interaction, ICMI 2015, Seattle, WA, USA, 9–13 November 2015, pp. 299–306. Association for Computing Machinery, Inc. (2015)

    Google Scholar 

  73. Bixler, R., D’Mello, S.: Automatic gaze-based user-independent detection of mind wandering during computerized reading. User Model. User-Adap. Inter. 26(1), 33–68 (2015). https://doi.org/10.1007/s11257-015-9167-1

    Article  Google Scholar 

  74. Yonetani, R., Kawashima, H., Matsuyama, T.: Multi-mode saliency dynamics model for analyzing gaze and attention. In: 7th Eye Tracking Research and Applications Symposium, ETRA 2012, Santa Barbara, CA, USA, 28–30 March 2012, pp. 115–122. Association for Computing Machinery (2012)

    Google Scholar 

  75. D’Mello, S., Olney, A., Williams, C., et al.: Gaze tutor: a gaze-reactive intelligent tutoring system. Int. J. Hum. Comput. Stud. 70(5), 377–398 (2012)

    Article  Google Scholar 

  76. Girardi, D., Lanubile, F., Novielli, N.: Emotion detection using noninvasive low cost sensors. In: 7th International Conference on Affective Computing and Intelligent Interaction, ACII 2017, San Antonio, TX, USA, 23–26 October 2017, vol. 2018, pp. 125–130. Institute of Electrical and Electronics Engineers Inc. (2017)

    Google Scholar 

  77. Yang, L., Jiang, D., Han, W., et al.: DCNN and DNN based multi-modal depression recognition. In: 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII), pp. 484-489 (2017)

    Google Scholar 

  78. Kim, J., Truong, K.P., Englebienne, G., et al.: Learning spectro-temporal features with 3D CNNs for speech emotion recognition. In: 7th International Conference on Affective Computing and Intelligent Interaction, ACII 2017, San Antonio, TX, USA, 23–26 October 2017, vol. 2018, pp. 383–388. Institute of Electrical and Electronics Engineers Inc. (2017)

    Google Scholar 

  79. Chen, K., Yue, G., Yu, F., Shen, Y., Zhu, A.: Research on speech emotion recognition system in e-learning. In: Shi, Y., van Albada, G.D., Dongarra, J., Sloot, P.M.A. (eds.) ICCS 2007. LNCS, vol. 4489, pp. 555–558. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-72588-6_91

    Chapter  Google Scholar 

  80. Tzinis, E., Potamianos, A.: Segment-based speech emotion recognition using recurrent neural networks. In: 7th International Conference on Affective Computing and Intelligent Interaction, ACII 2017, San Antonio, TX, USA, 23–26 October 2017, vol. 2018, pp. 190–195. Institute of Electrical and Electronics Engineers Inc. (2017)

    Google Scholar 

  81. Slater, S., Ocumpaugh, J., Baker, R., et al.: Using natural language processing tools to develop complex models of student engagement. In: 7th International Conference on Affective Computing and Intelligent Interaction, ACII 2017, San Antonio, TX, USA, 23–26 October 2017, vol. 2018, pp. 542–547. Institute of Electrical and Electronics Engineers Inc. (2017)

    Google Scholar 

  82. Demmans Epp, C., Munteanu, C., Axtell, B., et al.: Finger tracking: facilitating non-commercial content production for mobile E-reading applications. In: 19th International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI 2017, Vienna, Austria, 4–7 September 2017. Association for Computing Machinery, Inc., ACM Special Interest Group on Computer-Human Interaction (SIGCHI) (2017)

    Google Scholar 

  83. Ghosh, S., Ganguly, N., Mitra, B., et al.: Evaluating effectiveness of smartphone typing as an indicator of user emotion. In: 7th International Conference on Affective Computing and Intelligent Interaction, ACII 2017, San Antonio, TX, USA, 23–26 October 2017, vol. 2018, pp. 146–151. Institute of Electrical and Electronics Engineers Inc. (2017)

    Google Scholar 

  84. Ciman, M., Wac, K., Gaggi, O.: iSensestress: assessing stress through human-smartphone interaction analysis. In: 2015 9th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth), pp. 84–91 (2015)

    Google Scholar 

  85. Wu, Y.-H., Lin, S.-J., Yang, D.-L.: A mobile emotion recognition system based on speech signals and facial images. In: 2013 17th International Computer Science and Engineering Conference, ICSEC 2013, Bangkok, Thailand, 4–6 September 2013, pp. 212–217. IEEE Computer Society (2013)

    Google Scholar 

  86. Pham, P., Wang, J.: Predicting learners’ emotions in mobile MOOC learning via a multimodal intelligent tutor. In: Nkambou, R., Azevedo, R., Vassileva, J. (eds.) Intelligent Tutoring Systems, pp. 150–159. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-91464-0_15

    Chapter  Google Scholar 

  87. Zatarain-Cabada, R., Barrón-Estrada, M.L., Alor-Hernández, G., Reyes-García, C.A.: Emotion recognition in intelligent tutoring systems for android-based mobile devices. In: Gelbukh, A., Espinoza, F.C., Galicia-Haro, S.N. (eds.) MICAI 2014. LNCS (LNAI), vol. 8856, pp. 494–504. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-13647-9_44

    Chapter  Google Scholar 

  88. Benta, K.-I., Cremene, M., Vaida, M.-F.: A multimodal affective monitoring tool for mobile learning. In: 14th RoEduNet International Conference - Networking in Education and Research, RoEduNet NER 2015, Craiova, Romania, 24–26 September 2015, pp. 34–38. Institute of Electrical and Electronics Engineers Inc. (2015)

    Google Scholar 

  89. Pham, P., Wang, J.: AttentiveLearner: improving mobile MOOC learning via implicit heart rate tracking. In: Conati, C., Heffernan, N., Mitrovic, A., Verdejo, M.F. (eds.) AIED 2015. LNCS (LNAI), vol. 9112, pp. 367–376. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-19773-9_37

    Chapter  Google Scholar 

  90. Pham, P., Wang, J.: AttentiveLearner2: a multimodal approach for improving MOOC learning on mobile devices. In: André, E., et al. (eds.) Artificial Intelligence in Education, pp. 561–564. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-61425-0_64

    Chapter  Google Scholar 

  91. Rachuri, K.K., Musolesi, M., Mascolo, C., et al.: EmotionSense: a mobile phones based adaptive platform for experimental social psychology research. In: Proceedings of the 2010 ACM Conference on Ubiquitous Computing, UbiComp 2010, pp. 281–290. Association for Computing Machinery (2010)

    Google Scholar 

  92. Barron-Estrada, M.L., Zatarain-Cabada, R., Aispuro-Gallegos, C.G.: Multimodal recognition of emotions with application to mobile learning. In: 18th IEEE International Conference on Advanced Learning Technologies, ICALT 2018, Bombay, India, 9–13 July 2018, pp. 416–418. Institute of Electrical and Electronics Engineers Inc. (2018)

    Google Scholar 

  93. Wang, R., Chen, F., Chen, Z., et al.: Studentlife: assessing mental health, academic performance and behavioral trends of college students using smartphones. In: 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing, UbiComp 2014, Seattle, USA, 13–17 September 2014, pp. 3–14. Association for Computing Machinery, Inc. (2014)

    Google Scholar 

  94. Noroozi, F., Kaminska, D., Corneanu, C., et al.: Survey on emotional body gesture recognition. IEEE Trans. Affect. Comput. 12, 505–523 (2018)

    Article  Google Scholar 

  95. Tseng, C.-H., Chen, Y.-H.: A camera-based attention level assessment tool designed for classroom usage. J. Supercomput. 74(11), 5889–5902 (2017). https://doi.org/10.1007/s11227-017-2122-7

    Article  Google Scholar 

  96. Ayvaz, U., Guruler, H.: Real-time detection of students’ emotional states in the classroom

    Google Scholar 

  97. Orencilerin Sinif ici Duygusal Durumlarinin Gercek Zamanli Tespit Edilmesi. In: 25th Signal Processing and Communications Applications Conference, SIU 2017, Antalya, Turkey, 15–18 May 2017. Institute of Electrical and Electronics Engineers Inc. (2017)

    Google Scholar 

  98. Gupta, S.K., Ashwin, T.S., Guddeti, R.M.R.: Students affective content analysis in smart classroom environment using deep learning techniques. Multimed. Tools Appl. 78(18), 25321–25348 (2019)

    Article  Google Scholar 

  99. Whitehill, J., Serpell, Z., Lin, Y., et al.: The faces of engagement: automatic recognition of student engagementfrom facial expressions. IEEE Trans. Affect. Comput. 5(1), 86–98 (2014)

    Article  Google Scholar 

  100. Lim, J.H., Teh, E.Y., Geh, M.H., et al.: Automated classroom monitoring with connected visioning system. In: 9th Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2017, Kuala Lumpur, Malaysia, 12–15 December 2017, vol. 2018, pp. 386–393. Institute of Electrical and Electronics Engineers Inc. (2017)

    Google Scholar 

  101. Bosch, N., D’Mello, S.K., Baker, R.S., et al.: Detecting student emotions in computer-enabled classrooms. In: 25th International Joint Conference on Artificial Intelligence, IJCAI 2016, New York, NY, USA, 9–15 July 2016, vol. 2016, pp. 4125–4129. International Joint Conferences on Artificial Intelligence (2016)

    Google Scholar 

  102. Burnik, U., Zaletelj, J., Koir, A.: Kinect based system for student engagement monitoring. In: 1st IEEE Ukraine Conference on Electrical and Computer Engineering, UKRCON 2017, Kyiv, Ukraine, 29 May–2 June 2017, pp. 1229–1232. Institute of Electrical and Electronics Engineers Inc. (2017)

    Google Scholar 

  103. Kim, Y., Soyata, T., Behnagh, R.F.: Towards emotionally aware AI smart classroom: current issues and directions for engineering and education. IEEE Access 6, 5308–5331 (2018)

    Article  Google Scholar 

  104. Nikopoulou, R., Spyrou, E., Vernikos, I., et al.: Emotion recognition from speech: a classroom experiment. In: 11th ACM International Conference on PErvasive Technologies Related to Assistive Environments, PETRA 2018, Corfu, Greece, 26–29 June 2018, pp. 104–105. Association for Computing Machinery (2018)

    Google Scholar 

  105. Waber, B.N., Olguin Olguin, D., Kim, T., et al.: Understanding organizational behavior with wearable sensing technology (2008). SSRN 1263992

    Google Scholar 

Download references

Acknowledgement

This work is partly supported by National Key R&D Program of China (Grant No. 2019YFB1703800), Natural Science Basic Research Plan in Shanxi Province of China (Grant No. 2016JM6054), the Programme of Introducing Talents of Discipline to Universities(111 Project), China(Grant No. B13044).

Author information

Authors and Affiliations

  1. Northwestern Polytechnical University, Xi’an, China

    Guanfeng Wang, Chen Gong & Shuxia Wang

Authors
  1. Guanfeng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chen Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shuxia Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shuxia Wang .

Editor information

Editors and Affiliations

  1. Soar Technology, Inc., Orlando, FL, USA

    Robert A. Sottilare

  2. Fraunhofer FKIE, Wachtberg, Germany

    Jessica Schwarz

Ethics declarations

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, G., Gong, C., Wang, S. (2022). A Review of Automatic Detection of Learner States in Four Typical Learning Scenarios. In: Sottilare, R.A., Schwarz, J. (eds) Adaptive Instructional Systems. HCII 2022. Lecture Notes in Computer Science, vol 13332. Springer, Cham. https://doi.org/10.1007/978-3-031-05887-5_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-05887-5_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-05886-8

  • Online ISBN: 978-3-031-05887-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation