Skip to main content
SpringerLink
Log in

Trends in the use of affective computing in e-learning environments

  • Published:
Education and Information Technologies Aims and scope Submit manuscript

Abstract

Considering that emotions have a great impact on motivation, reasoning, and decision making, affective computing methods, that were designed to attempt to understand and respond to human emotional states, have been used in more than one field including e-learning. Thus, a systematic literature review was conducted on 4 search engines resulting in a set of papers that were filtered in a systematic way until we obtained a corpus of 27 papers. Data were extracted to answer four research questions concerning the use and efficacy of affective computing in e-learning in recent years. We found out that the majority of studies about emotion recognition use uni-modal systems in which facial expressions emotion detection is the most present. The major research purpose is designing/building systems, approaches, methods, detectors for emotion recognition. For the e-learning environments, the most present is conversational agents. The emotions detected or used are basic emotions, non-basic emotions, learning-centered emotions, trait emotions, or a combination of two or three of them. This systematic literature review also provides the major findings, challenges, and future research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Data availability

Not applicable.

Code availability

Not applicable.

References

  • Akputu, O. K., Seng, K. P., Lee, Y., & Ang, L. M. (2018). Emotion recognition using multiple kernel learning toward E-learning applications. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), 14(1), 1. https://doi.org/10.1145/3131287

  • Al Osman, H., Dong, H., & El Saddik, A. (2016). Ubiquitous biofeedback serious game for stress management. IEEE Access, 4, 1274–1286. https://doi.org/10.1109/ACCESS.2016.2548980

    Article  Google Scholar 

  • Alexandra, J. M., Andres, L., Ocumpaugh, J., Baker, R. S., Slater, S., Paquette, L., ... & Moore, A. (2019, March). affect sequences and learning in Betty’s brain. In Proceedings of the 9th International Conference on Learning Analytics & Knowledge, Tempe, AZ, USA (pp. 4–8).

  • Bachtiar, F. A., Sulistyo, G. H., Cooper, E. W., & Katsuari, K. (2017, January). Affect, personality, and learning styles in online reading comprehension. In Proceedings of the 5th International Conference on Information and Education Technology (pp. 78–83). ACM. https://doi.org/10.1145/3029387.3029422

  • Bahreini, K., Nadolski, R., & Westera, W. (2016). Towards real-time speech emotion recognition for affective e-learning. Education and Information Technologies, 21(5), 1367–1386. https://doi.org/10.1007/s10639-015-9388-2

    Article  Google Scholar 

  • Bahreini, K., van der Vegt, W., & Westera, W. (2019). A fuzzy logic approach to reliable real-time recognition of facial emotions. Multimedia Tools and Applications, 78(14), 18943–18966. https://doi.org/10.1007/s11042-019-7250-z

    Article  Google Scholar 

  • Baker, R., & Ocumpaugh, J. (2015). Interaction-based affect detection in educational software (pp. 233–245). Oxford University Press.

    Google Scholar 

  • Baldassarri, S., Hupont, I., Abadía, D., & Cerezo, E. (2015). Affective-aware tutoring platform for interactive digital television. Multimedia Tools and Applications, 74(9), 3183–3206. https://doi.org/10.1007/s11042-013-1779-z

    Article  Google Scholar 

  • Barbalet, J. M. (2001). Emotion, social theory, and social structure: A macrosociological approach. Cambridge University Press.

  • Bontchev, B., & Vassileva, D. (2016, November). Assessing engagement in an emotionally-adaptive applied game. In Proceedings of the Fourth International Conference on Technological Ecosystems for Enhancing Multiculturality (pp. 747–754). ACM. https://doi.org/10.1145/3012430.3012602

  • Bosch, N., D'mello, S. K., Ocumpaugh, J., Baker, R. S., & Shute, V. (2016). Using video to automatically detect learner affect in computer-enabled classrooms. ACM Transactions on Interactive Intelligent Systems (TiiS), 6(2), 17. https://doi.org/10.1145/2946837

  • Bower, G. H., & Cohen, P. R. (1982). Emotional influences in memory and thinking: Data and theory. Affect and cognition, 1.

  • Brigham, T. J. (2017). Merging technology and emotions: Introduction to affective computing. Medical Reference Services Quarterly, 36(4), 399–407.

    Article  MathSciNet  Google Scholar 

  • Buck, R. (1994). Social and emotional functions in facial expression and communication: The readout hypothesis. Biological Psychology, 38(2–3), 95–115. https://doi.org/10.1016/0301-0511(94)90032-9

    Article  Google Scholar 

  • Calvo, R. A., & D’Mello, S. (2010). Affect detection: An interdisciplinary review of models, methods, and their applications. IEEE Transactions on Affective Computing, 1(1), 18–37. https://doi.org/10.1109/T-AFFC.2010.1

    Article  Google Scholar 

  • Chen, J., Luo, N., Liu, Y., Liu, L., Zhang, K., & Kolodziej, J. (2016). A hybrid intelligence-aided aproach to affect-sensitive e-learning. Computing, 98(1–2), 215–233. https://doi.org/10.1007/s00607-014-0430-9

    Article  MathSciNet  Google Scholar 

  • Clore, G. L., & Palmer, J. (2009). Affective guidance of intelligent agents: How emotion controls cognition. Cognitive Systems Research, 10(1), 21–30. https://doi.org/10.1016/j.cogsys.2008.03.002

    Article  Google Scholar 

  • Dirkx, J. M. (2001). The power of feelings: Emotion, imagination, and the construction of meaning in adult learning. New Directions for Adult and Continuing Education, 2001(89), 63–72.

    Article  Google Scholar 

  • D'mello, S., & Graesser, A. (2012). AutoTutor and affective AutoTutor: Learning by talking with cognitively and emotionally intelligent computers that talk back. ACM Transactions on Interactive Intelligent Systems (TiiS), 2(4), 23. https://doi.org/10.1145/2395123.2395128

  • Eid, M., Schneider, C., & Schwenkmezger, P. (1999). Do you feel better or worse? The validity of perceived deviations of mood states from mood traits. European Journal of Personality, 13(4), 283–306. https://doi.org/10.1002/(SICI)1099-0984(199907/08)13:4%3c283::AID-PER341%3e3.0.CO;2-0

    Article  Google Scholar 

  • Ekman, P., Friesen, W. V., O'sullivan, M., Chan, A., Diacoyanni-Tarlatzis, I., Heider, K., ... & Scherer, K. (1987). Universals and cultural differences in the judgments of facial expressions of emotion. Journal of Personality and Social Psychology, 53(4), 712. https://doi.org/10.1037/0022-3514.53.4.712

  • Fehr, B., & Russell, J. A. (1984). Concept of emotion viewed from a prototype perspective. Journal of Experimental Psychology: General, 113(3), 464. https://doi.org/10.1037/0096-3445.113.3.464

    Article  Google Scholar 

  • Forsyth, C. M., Graesser, A., Olney, A. M., Millis, K., Walker, B., & Cai, Z. (2015, June). Moody agents: affect and discourse during learning in a serious game. In International Conference on Artificial Intelligence in Education (pp. 135–144). Springer, Cham. https://doi.org/10.1007/978-3-319-19773-9_14

  • Frijda, N. H., Manstead, A. S., & Bem, S. (2000). The influence of emotions on beliefs. Emotions and beliefs: How feelings influence thoughts, 1–9.

  • Fwa, H. L. (2018). An architectural design and evaluation of an affective tutoring system for novice programmers. International Journal of Educational Technology in Higher Education, 15(1), 38. https://doi.org/10.1186/s41239-018-0121-2

    Article  Google Scholar 

  • Graesser, A. C., & D’Mello, S. (2012). Moment-to-moment emotions during reading. The Reading Teacher, 66(3), 238–242. https://doi.org/10.1002/TRTR.01121

    Article  Google Scholar 

  • Grafsgaard, J. F., Wiggins, J. B., Boyer, K. E., Wiebe, E. N., & Lester, J. C. (2013, September). Automatically recognizing facial indicators of frustration: a learning-centric analysis. In 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction (pp. 159–165). IEEE. https://doi.org/10.1109/ACII.2013.33

  • Grawemeyer, B., Mavrikis, M., Holmes, W., Gutierrez-Santos, S., Wiedmann, M., & Rummel, N. (2016, April). Affecting off-task behaviour: how affect-aware feedback can improve student learning. In Proceedings of the sixth international conference on learning analytics & knowledge (pp. 104–113). ACM. https://doi.org/10.1145/2883851.2883936

  • Harley, J. M., Carter, C. C., Papaionnou, N., Bouchet, F., Landis, R. S., Azevedo, R., & Karabachian, L. (2015, June). Examining the predictive relationship between personality and emotion traits and learners’ agent-direct emotions. In International Conference on Artificial Intelligence in Education (pp. 145–154). Springer, Cham. https://doi.org/10.1007/978-3-319-19773-9_15

  • Harley, J. M., Carter, C. K., Papaionnou, N., Bouchet, F., Landis, R. S., Azevedo, R., & Karabachian, L. (2016). Examining the predictive relationship between personality and emotion traits and students’ agent-directed emotions: Towards emotionally-adaptive agent-based learning environments. User Modeling and User-Adapted Interaction, 26(2), 177–219. https://doi.org/10.1007/s11257-016-9169-7

    Article  Google Scholar 

  • Hung, J. C., Lin, K. C., & Lai, N. X. (2019). Recognizing learning emotion based on convolutional neural networks and transfer learning. Applied Soft Computing, 84, 105724. https://doi.org/10.1016/j.asoc.2019.105724

    Article  Google Scholar 

  • Isen, A. M. (2004). Some perspectives on positive feelings and emotions: Positive affect facilitates thinking and problem solving. In Feelings and Emotions: The Amsterdam Symposium, Jun, 2001, Amsterdam, Netherlands. Cambridge University Press.

  • Kamour, M. (2012). Importance of emotional intelligence among open learning and distance learning students. In EDULEARN12 Proceedings (pp. 2895–2899). IATED.

  • Kitchenham, B. (2004). Procedures for performing systematic reviews. Keele, UK, Keele University, 33(2004), 1–26.

    Google Scholar 

  • Kitchenham, B., & Charters, S. (2007). Guidelines for performing systematic literature reviews in software engineering.

  • Kleinginna, P. R., & Kleinginna, A. M. (1981). A categorized list of emotion definitions, with suggestions for a consensual definition. Motivation and Emotion, 5(4), 345–379. https://doi.org/10.1007/BF00992553

    Article  Google Scholar 

  • Kołakowska, A. (2013, June). A review of emotion recognition methods based on keystroke dynamics and mouse movements. In 2013 6th international conference on human system interactions (HSI) (pp. 548–555). IEEE.

  • Krithika, L. B., & GG, L. P. (2016). Student emotion recognition system (SERS) for e-learning improvement based on learner concentration metric. Procedia Computer Science, 85, 767-776.

  • Levy, Y. (2007). Comparing dropouts and persistence in e-learning courses. Computers & Education, 48(2), 185–204.

    Article  Google Scholar 

  • Lin, H. C. K., Wu, C. H., & Hsueh, Y. P. (2014). The influence of using affective tutoring system in accounting remedial instruction on learning performance and usability. Computers in Human Behavior, 41, 514–522. https://doi.org/10.1016/j.chb.2014.09.052

    Article  Google Scholar 

  • Liu, Y., Fu, Q., & Fu, X. (2009). The interaction between cognition and emotion. Chinese Science Bulletin, 54(22), 4102. https://doi.org/10.1007/s11434-009-0632-2

    Article  Google Scholar 

  • Matejka, M. M., Kazzer, P., Seehausen, M., Bajbouj, M., Klann-Delius, G., Margrit, G., ... & Prehn, K. (2013). Talking about emotion: prosody and skin conductance indicate emotion regulation. Frontiers in Psychology, 4, 260.

  • Moore, J. L., Dickson-Deane, C., & Galyen, K. (2011). e-Learning, online learning, and distance learning environments: Are they the same? The Internet and Higher Education, 14(2), 129–135. https://doi.org/10.1016/j.iheduc.2010.10.001

    Article  Google Scholar 

  • Pardos, Z. A., Baker, R. S., San Pedro, M. O., Gowda, S. M., & Gowda, S. M. (2013, April). Affective states and state tests: Investigating how affect throughout the school year predicts end of year learning outcomes. In Proceedings of the third international conference on learning analytics and knowledge (pp. 117–124). ACM. https://doi.org/10.1145/2460296.2460320

  • Pekrun, R., Goetz, T., Titz, W., & Perry, R. P. (2002). Academic emotions in students’ self-regulated learning and achievement: A program of qualitative and quantitative research. Educational Psychologist, 37(2), 91–105.

    Article  Google Scholar 

  • Phelps, E. A. (2006). Emotion and cognition: Insights from studies of the human amygdala. Annual Review of Psychology, 57, 27–53. https://doi.org/10.1146/annurev.psych.56.091103.070234

    Article  Google Scholar 

  • Poria, S., Cambria, E., Bajpai, R., & Hussain, A. (2017). A review of affective computing: From unimodal analysis to multimodal fusion. Information Fusion, 37, 98–125. https://doi.org/10.1016/j.inffus.2017.02.003

    Article  Google Scholar 

  • Qualter, P., Gardner, K. J., & Whiteley, H. E. (2007). Emotional intelligence: Review of research and educational implications. Pastoral Care in Education, 25(1), 11–20.

    Article  Google Scholar 

  • Rimé, B. (2009). Emotion elicits the social sharing of emotion: Theory and empirical review. Emotion Review, 1(1), 60–85.

    Article  Google Scholar 

  • Salmeron-Majadas, S., Arevalillo-Herráez, M., Santos, O. C., Saneiro, M., Cabestrero, R., Quirós, P., ... & Boticario, J. G. (2015, June). Filtering of spontaneous and low intensity emotions in educational contexts. In International Conference on Artificial Intelligence in Education (pp. 429–438). Springer, Cham. https://doi.org/10.1007/978-3-319-19773-9_43

  • Salovey, P., & Grewal, D. (2005). The science of emotional intelligence. Current Directions in Psychological Science, 14(6), 281–285.

    Article  Google Scholar 

  • Salovey, P., & Mayer, J. D. (1990). Emotional intelligence. Imagination, Cognition and Personality, 9(3), 185–211.

    Article  Google Scholar 

  • Santos, O. C., Rodriguez-Ascaso, A., Boticario, J. G., Salmeron-Majadas, S., Quirós, P., & Cabestrero, R. (2013, July). Challenges for inclusive affective detection in educational scenarios. In International Conference on Universal Access in Human-Computer Interaction (pp. 566–575). Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39188-0_61

  • Schuller, B. W. (2018). Speech emotion recognition: Two decades in a nutshell, benchmarks, and ongoing trends. Communications of the ACM, 61(5), 90–99.

    Article  Google Scholar 

  • Souza, N., & Perry, G. (2018). Identification of affective states in MOOCs: A systematic literature review. Int. J. Innov. Educ. Res., 6(12), 39–55. https://doi.org/10.31686/ijier.vol6.iss12.1250

  • Spann, C. A., Shute, V. J., Rahimi, S., & D’Mello, S. K. (2019). The productive role of cognitive reappraisal in regulating affect during game-based learning. Computers in Human Behavior. https://doi.org/10.1016/j.chb.2019.03.002

    Article  Google Scholar 

  • Spaulding, S., Gordon, G., & Breazeal, C. (2016, May). Affect-aware student models for robot tutors. In Proceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems (pp. 864–872). International Foundation for Autonomous Agents and Multiagent Systems.

  • Tyng, C. M., Amin, H. U., Saad, M. N., & Malik, A. S. (2017). The influences of emotion on learning and memory. Frontiers in Psychology, 8, 1454. https://doi.org/10.3389/fpsyg.2017.01454

    Article  Google Scholar 

  • Vogel, S., & Schwabe, L. (2016). Learning and memory under stress: implications for the classroom. npj Science of Learning, 1(1), 1–10. https://doi.org/10.1038/npjscilearn.2016.11

  • Weidt, F., & Silva, R. (2016). Systematic Literature Review in Computer Science-A Practical Guide. Relatórios Técnicos do DCC/UFJF, 1.

  • Woolf, B., Burleson, W., Arroyo, I., Dragon, T., Cooper, D., & Picard, R. (2009). Affect-aware tutors: Recognising and responding to student affect. International Journal of Learning Technology, 4(3–4), 129–164. https://doi.org/10.1504/IJLT.2009.028804

    Article  Google Scholar 

  • Wu, C. H., Huang, Y. M., & Hwang, J. P. (2016). Review of affective computing in education/learning: Trends and challenges. British Journal of Educational Technology, 47(6), 1304–1323. https://doi.org/10.1111/bjet.12324

    Article  Google Scholar 

  • Xing, B., Zhang, L., Gao, J., Yu, R., & Lyu, R. (2016, November). Barrier-free affective communication in MOOC study by analyzing pupil diameter variation. In SIGGRAPH ASIA 2016 Symposium on Education (pp. 1–8). https://doi.org/10.1145/2993352.2993362

  • Yadegaridehkordi, E., Noor, N. F. B. M., Ayub, M. N. B., Affal, H. B., & Hussin, N. B. (2019). Affective computing in education: A systematic review and future research. Computers & Education, 142, 103649. https://doi.org/10.1016/j.compedu.2019.103649

    Article  Google Scholar 

  • Zatarain-Cabada, R., Barrón-Estrada, M. L., & Ríos-Félix, J. M. (2016, October). Affective learning system for algorithmic logic applying gamification. In Mexican International Conference on Artificial Intelligence (pp. 536–547). Springer, Cham. https://doi.org/10.1007/978-3-319-62428-0_44

Download references

Author information

Authors and Affiliations

  1. Research Laboratory of Technologies of Information and Communication & Electrical Engineering (LaTICE), University of Tunis, Tunis, Tunisia

    Nesreen Mejbri, Fathi Essalmi & Mohamed Jemni

  2. Management Information Systems Department, College of Business, University of Jeddah, Jeddah, Saudi Arabia

    Fathi Essalmi & Bader A. Alyoubi

Authors
  1. Nesreen Mejbri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fathi Essalmi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohamed Jemni
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bader A. Alyoubi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nesreen Mejbri.

Ethics declarations

Conflicts of interests

The authors have no conflicts of interest to declare that are relevant to the content of this article.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mejbri, N., Essalmi, F., Jemni, M. et al. Trends in the use of affective computing in e-learning environments. Educ Inf Technol 27, 3867–3889 (2022). https://doi.org/10.1007/s10639-021-10769-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10639-021-10769-9

Keywords

Search

Navigation