Skip to main content

Advertisement

Comparative study on emotions analysis from facial expressions in children with and without learning disabilities in virtual learning environment

  • Published:
Education and Information Technologies Aims and scope Submit manuscript

Abstract

Children with Learning Disabilities (LDs) show some emotional difficulties and behavioral problems in classroom compared with their peers without LDs. Emotions constitute an important part of the learning process. Recent evidence suggests that the use of Information and Communication Technology (ICT) in special education permits to remove barriers in learning for the target children. Besides, it offers a learning environment for a diversity of emotional experiences. In this present study, we explored the benefits of ICT use to identify the ways in which emotions are involved during the learning process in Virtual Learning Environments (VLE). We conducted a user study with 42 children divided into two groups; experimental group (n = 14) and age matched control group (n = 28) to compare their emotional experiences in VLE. We used advances in Artificial Intelligence (AI) to detect children’s emotions through their facial expressions by analyzing seven basic facial emotion expressions (angry, disgust, fear, happy, sad, surprise and neutral) while playing an educational game. The initial results indicate that emotions are present in VLE and they appear to suggest that children with LDs experience the same emotions as their peers without LDs in VLE. Besides, they show that children with LDs experience less negative emotions compared to literature evidence about the presence of a higher level of negative emotions in classroom.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Abrams, J. C. (1986). On learning disabilities: Affective considerations. Journal of Reading, Writing, and Learning Disabilities International, 2(3), 189–196. https://doi.org/10.1080/0748763860020303.

    Article  Google Scholar 

  • Adam, T., & Tatnall, A. (2010). Use of ICT to assist students with learning difficulties: An Actor-Network Analysis. In Key competencies in the knowledge society (pp. 1–11). Springer.

  • Adam, T., & Tatnall, A. (2017). The value of using ICT in the education of school students with learning difficulties. Education and Information Technologies, 22(6), 2711–2726.

    Article  Google Scholar 

  • Adebisi, R. O., Liman, N. A., & Longpoe, P. K. (2015). Using assistive Technology in Teaching Children with learning disabilities in the 21st century. Journal of Education and Practice, 6(24), 14–20.

    Google Scholar 

  • Ahmad, W. F. W., Akhir, E. A. P., & Azmee, S. (2010). Game-based learning courseware for children with learning disabilities (pp. 1–4). IEEE. https://doi.org/10.1109/ITSIM.2010.5561303.

  • AL Zyoudi, M. (2010). Differences in self-concept among student with and without learning disabilities in Al Karak District in Jordan. International Journal of Special Education, 25(2), 72–77.

    Google Scholar 

  • Alexander, S., Sarrafzadeh, A., & Hill, S. (2006). Easy with eve: A functional affective tutoring system. In Workshop on Motivational and Affective Issues in ITS. 8th International Conference on ITS (pp. 5–12). Citeseer.

  • Bahreini, K., Nadolski, R., & Westera, W. (2016). Towards multimodal emotion recognition in e-learning environments. Interactive Learning Environments, 24(3), 590–605.

    Article  Google Scholar 

  • Ben Ammar, M., Neji, M., Alimi, A. M., & Gouardères, G. (2010). The affective tutoring system. Expert Systems with Applications, 37(4), 3013–3023. https://doi.org/10.1016/j.eswa.2009.09.031.

    Article  Google Scholar 

  • Benmarrakchi, F., El Kafi, J., & Hore, A. E. (2015). A different learning way for pupils with specific learning disabilities. International Journal of Computers & Technology, 14, 6157–6162.

    Article  Google Scholar 

  • Benmarrakchi, F. E., El Kafi, J., & Elhore, A. (2016). Supporting dyslexic’s learning style preferences in adaptive virtual learning environment (pp. 1–6). IEEE. https://doi.org/10.1109/ICEMIS.2016.7745294.

  • Benmarrakchi, F. E., El Kafi, J., & Elhore, A. (2017a). Communication Technology for Users with specific learning disabilities. Procedia Computer Science, 110, 258–265. https://doi.org/10.1016/j.procs.2017.06.093.

    Article  Google Scholar 

  • Benmarrakchi, F. E., El Kafi, J., & Elhore, A. (2017b). User modeling approach for dyslexic students in virtual learning environments. International Journal of Cloud Applications and Computing (IJCAC), 7(2), 1–9.

    Article  Google Scholar 

  • Benmarrakchi, F., El Kafi, J., Elhore, A., & Haie, S. (2017c). Exploring the use of the ICT in supporting dyslexic students’ preferred learning styles: A preliminary evaluation. Education and Information Technologies, 22(6), 2939–2957. https://doi.org/10.1007/s10639-016-9551-4.

    Article  Google Scholar 

  • Bower, G. H. (1981). Mood and memory. American Psychologist, 36(2), 129–148. https://doi.org/10.1037/0003-066X.36.2.129.

    Article  Google Scholar 

  • Bowlby, J. (1969). Attachment, Vol. 1 of attachment and loss. New York: Basic Books.

    Google Scholar 

  • Bryan, T., Burstein, K., & Ergul, C. (2004). The social-emotional side of learning disabilities: A science-based presentation of the state of the art. Learning Disability Quarterly, 27(1), 45–51. https://doi.org/10.2307/1593631.

    Article  Google Scholar 

  • Burgstahler, S. (2003). The role of Technology in Preparing Youth with disabilities for postsecondary education and employment. Journal of Special Education Technology, 18(4), 7–19. https://doi.org/10.1177/016264340301800401.

    Article  Google Scholar 

  • Bursuck, W. (1989). A comparison of students with learning disabilities to low achieving and higher achieving students on three dimensions of social competence. Journal of Learning Disabilities, 22(3), 188–194. https://doi.org/10.1177/002221948902200310.

    Article  Google Scholar 

  • Calleja, G. (2011). In-game: From immersion to incorporation. MIT Press.

  • Carver, C. S., & Scheier, M. F. (2012). Perspectives on personality. Pearson education.

  • Cleveland-Innes, M., & Campbell, P. (2012). Emotional presence, learning, and the online learning environment. The International Review of Research in Open and Distributed Learning, 13(4), 269–292. https://doi.org/10.19173/irrodl.v13i4.1234.

    Article  Google Scholar 

  • Cortiella, C., & Horowitz, S. H. (2014). The state of learning disabilities: Facts, trends and emerging issues (pp. 2–45). New York: National Center for Learning Disabilities.

    Google Scholar 

  • Cosden, M., Elliott, K., Noble, S., & Kelemen, E. (1999). Self-understanding and self-esteem in children with learning disabilities. Learning Disability Quarterly, 22(4), 279–290. https://doi.org/10.2307/1511262.

    Article  Google Scholar 

  • Crutzen, R., van ‘t Riet, J., & Short, C. E. (2015). Enjoyment: A conceptual exploration and overview of experimental evidence in the context of games for health. Games for Health Journal, 5(1), 15–20. https://doi.org/10.1089/g4h.2015.0059.

    Article  Google Scholar 

  • D’Mello, S., Picard, R. W., & Graesser, A. (2007). Toward an affect-sensitive AutoTutor. IEEE Intelligent Systems, 22(4), 53–61. https://doi.org/10.1109/MIS.2007.79.

    Article  Google Scholar 

  • Dillon, P. (2004). Trajectories and tensions in the theory of information and communication Technology in Education. British Journal of Educational Studies, 52(2), 138–150. https://doi.org/10.1111/j.1467-8527.2004.00259.x.

    Article  Google Scholar 

  • Ekman, P., & Friesen, W. V. (1971). Constants across cultures in the face and emotion. Journal of Personality and Social Psychology, 17(2), 124–129. https://doi.org/10.1037/h0030377.

    Article  Google Scholar 

  • Ekman, I., Chanel, G., Järvelä, S., Kivikangas, J. M., Salminen, M., & Ravaja, N. (2012). Social interaction in games: Measuring physiological linkage and social presence. Simulation & Gaming, 43(3), 321–338. https://doi.org/10.1177/1046878111422121.

    Article  Google Scholar 

  • Forgas, J. P. (2012). Feeling and speaking: Affective influences on communication strategies and language use. Social Cognition and Communication, 63–81.

  • Forgas, J. P. (2017). Mood effects on cognition: Affective influences on the content and process of information processing and behavior. In Emotions and affect in human factors and human-computer interaction (pp. 89–122). Elsevier. https://doi.org/10.1016/B978-0-12-801851-4.00003-3.

    Chapter  Google Scholar 

  • Gajadhar, B. J., de Kort, Y. A. W., & IJsselsteijn, W. A. (2008). Shared Fun Is Doubled Fun: Player Enjoyment as a Function of Social Setting. In P. Markopoulos, B. de Ruyter, W. IJsselsteijn, & D. Rowland (Eds.), Fun and Games (pp. 106–117). Berlin Heidelberg: Springer.

    Chapter  Google Scholar 

  • Gallegos, J., Langley, A., & Villegas, D. (2012). Anxiety, depression, and coping skills among Mexican school children: A comparison of students with and without learning disabilities. Learning Disability Quarterly, 35(1), 54–61. https://doi.org/10.1177/0731948711428772.

    Article  Google Scholar 

  • Graetz, K. A. (2006). The psychology of learning environments. Learning Spaces, 6.

  • Gross, J. J. (2010). The Future’s so bright, I Gotta Wear shades. Emotion Review, 2(3), 212–216. https://doi.org/10.1177/1754073910361982.

    Article  Google Scholar 

  • Hammill, D. D. (1990). On defining learning disabilities: An emerging consensus. Journal of Learning Disabilities, 23(2), 74–84. https://doi.org/10.1177/002221949002300201.

    Article  Google Scholar 

  • Hammoumi, O. E., Benmarrakchi, F.E., Ouherrou, N., El Kafi, J., & ElHore, A. (2018). Emotion Recognition in E-learning Systems. In 2018 6th International Conference on Multimedia Computing and Systems (ICMCS) (pp. 1–6). https://doi.org/10.1109/ICMCS.2018.8525872.

  • Hassan, A. E. H. (2015). Emotional and behavioral problems of children with learning disabilities. Journal of Educational Policy and Entrepreneurial Research (JEPER), 2(10), 66–74.

    Google Scholar 

  • Hill, K. T., & Wigfield, A. (1984). Test anxiety: A major educational problem and what can be done about it. The Elementary School Journal, 85(1), 105–126.

    Article  Google Scholar 

  • Horvat, M., Kukolja, D., & Ivanec, D. (2015). Comparing affective responses to standardized pictures and videos: A study report. ArXiv:1505.07398 [Cs]. Retrieved from https://arxiv.org/ftp/arxiv/papers/1505/1505.07398.pdf.

  • Ismaili, J. (2017). Mobile learning as alternative to assistive technology devices for special needs students. Education and Information Technologies, 22(3), 883–899.

    Article  Google Scholar 

  • Jacko, J. A. (2012). Human computer interaction handbook: Fundamentals, evolving technologies, and emerging applications, third edition. CRC Press.

  • Kah, A. E., & Lakhouaja, A. (2018). Developing effective educative games for Arabic children primarily dyslexics. Education and Information Technologies, 23, 1–20. https://doi.org/10.1007/s10639-018-9750-2.

    Article  Google Scholar 

  • Kanade, T., Cohn, J. F., & Tian, Y. (2000). Comprehensive database for facial expression analysis. In Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580) (pp. 46–53). https://doi.org/10.1109/AFGR.2000.840611.

  • Kistner, J., Haskett, M., White, K., & Robbins, F. (1987). Perceived competence and self-worth of LD and normally achieving students. Learning Disability Quarterly, 10(1), 37–44. https://doi.org/10.2307/1510753.

    Article  Google Scholar 

  • Kort, B., Reilly, R., & Picard, R. W. (2001). An affective model of interplay between emotions and learning: reengineering educational pedagogy-building a learning companion. In Proceedings IEEE International Conference on Advanced Learning Technologies (pp. 43–46). https://doi.org/10.1109/ICALT.2001.943850.

  • Linnenbrink-Garcia, L., & Pekrun, R. (2011). Students’ emotions and academic engagement: Introduction to the special issue. Contemporary Educational Psychology, 36(1), 1–3. https://doi.org/10.1016/j.cedpsych.2010.11.004.

    Article  Google Scholar 

  • Lucey, P., Cohn, J. F., Kanade, T., Saragih, J., Ambadar, Z., & Matthews, I. (2010). The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops (pp. 94–101). https://doi.org/10.1109/CVPRW.2010.5543262.

  • Lundqvist, D., Flykt, A., & Öhman, A. (1998). The Karolinska directed emotional faces (KDEF). CD ROM from Department of Clinical Neuroscience, Psychology Section, Karolinska Institutet, (1998).

  • Maag, J. W., & Reid, R. (2006). Depression among students with learning disabilities: Assessing the risk. Journal of Learning Disabilities, 39(1), 3–10. https://doi.org/10.1177/00222194060390010201.

    Article  Google Scholar 

  • Margalit, M. (1991). Understanding loneliness among students with learning disabilities. Behaviour Change, 8(04), 167–173. https://doi.org/10.1017/S0813483900006641.

    Article  Google Scholar 

  • Margalit, M., & Ben-Dov, I. (1995). Learning disabilities and social environments: Kibbutz versus City comparisons of loneliness and social competence. International Journal of Behavioral Development, 18(3), 519–536. https://doi.org/10.1177/016502549501800308.

    Article  Google Scholar 

  • Nelson, J. M., & Harwood, H. (2011). Learning disabilities and anxiety: A meta-analysis. Journal of Learning Disabilities, 44(1), 3–17. https://doi.org/10.1177/0022219409359939.

    Article  Google Scholar 

  • Pekrun, R. (1992). The impact of emotions on learning and achievement: Towards a theory of cognitive/motivational mediators. Applied Psychology, 41(4), 359–376. https://doi.org/10.1111/j.1464-0597.1992.tb00712.x.

    Article  Google Scholar 

  • Pekrun, R., Goetz, T., Titz, W., & Perry, R. P. (2002). Academic emotions in students’ self-regulated learning and achievement: A program of qualitative and quantitative research. Educational Psychologist, 37(2), 91–105.

    Article  Google Scholar 

  • Petrovica, S., Anohina-Naumeca, A., & Ekenel, H. K. (2017). Emotion recognition in affective tutoring systems: Collection of ground-truth data. Procedia Computer Science, 104, 437–444. https://doi.org/10.1016/j.procs.2017.01.157.

    Article  Google Scholar 

  • Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39(6), 1161–1178. https://doi.org/10.1037/h0077714.

    Article  Google Scholar 

  • Russell, J. A., & Barrett, L. F. (1999). Core affect, prototypical emotional episodes, and other things called emotion: Dissecting the elephant. Journal of Personality and Social Psychology, 76(5), 805–819.

    Article  Google Scholar 

  • Ryan, R. M., & Deci, E. L. (2000). Intrinsic and extrinsic motivations: Classic definitions and new directions. Contemporary Educational Psychology, 25(1), 54–67. https://doi.org/10.1006/ceps.1999.1020.

    Article  Google Scholar 

  • Sandanayake, T. C., Madurapperuma, A. P., & Dias, D. (2011). Affective E learning model for Recognising learner emotions. International Journal of Information and Education Technology, 1(4), 315.

    Article  Google Scholar 

  • Sarrafzadeh, A., Hosseini, H. G., Fan, C., & Overmyer, S. P. (2003). Facial expression analysis for estimating learner’s emotional state in intelligent tutoring systems. In Proceedings 3rd IEEE International Conference on Advanced Technologies (pp. 336–337). https://doi.org/10.1109/ICALT.2003.1215111.

  • Sarrafzadeh, A., Alexander, S., Dadgostar, F., Fan, C., & Bigdeli, A. (2008). “How do you know that I don’t understand?” a look at the future of intelligent tutoring systems. Computers in Human Behavior, 24(4), 1342–1363. https://doi.org/10.1016/j.chb.2007.07.008.

    Article  Google Scholar 

  • Sathik, M., & Jonathan, S. G. (2013). Effect of facial expressions on student’s comprehension recognition in virtual educational environments. SpringerPlus, 2(1), 455. https://doi.org/10.1186/2193-1801-2-455.

    Article  Google Scholar 

  • Shen, L., Leon, E., Callaghan, V., & Shen, R. (2007). Exploratory research on an affective elearning model. In International Workshop on Blended Learning (pp. 15–17).

  • Shen, L., Wang, M., & Shen, R. (2009). Affective e-learning: Using" emotional" data to improve learning in pervasive learning environment. Journal of Educational Technology & Society, 12(2).

  • Sorour, A. S., Mohamed, N. A., & El-Maksoud, M. M. A. (2014). Emotional and behavioral problems of primary school children with and without learning disabilities: A comparative study. Journal of Education and Practice, 5(8), 1–11.

    Google Scholar 

  • Stemmler, G., & Wacker, J. (2010). Personality, emotion, and individual differences in physiological responses. Biological Psychology, 84(3), 541–551. https://doi.org/10.1016/j.biopsycho.2009.09.012.

    Article  Google Scholar 

  • Tam, H. E., & Hawkins, R. (2012). Self-concept and depression levels of students with dyslexia in Singapore.

  • Toro, P. A., Weissberg, R. P., Guare, J., & Liebenstein, N. L. (1990). A comparison of children with and without learning disabilities on social problem-solving skill, school behavior, and family background. Journal of Learning Disabilities, 23(2), 115–120. https://doi.org/10.1177/002221949002300207.

    Article  Google Scholar 

  • Vaughn, S., Haager, D., Hogan, A., & Kouzekanani, K. (1992). Self-concept and peer acceptance in students with learning disabilities: A four- to five-year prospective study. Journal of Educational Psychology, 84(1), 43–50. https://doi.org/10.1037/0022-0663.84.1.43.

    Article  Google Scholar 

  • Viola, P., & Jones, M. (2001). Rapid object detection using a boosted cascade of simple features. In Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001 (Vol. 1, pp. I-511-I–518 vol.1). https://doi.org/10.1109/CVPR.2001.990517.

  • Vodopivec, J. L., & Bagon, S. (2016). Motivation for using ICT and pupils with learning difficulties. International Journal of Emerging Technologies in Learning (IJET), 11(10), 70–75.

    Article  Google Scholar 

  • Wiener, J., & Schneider, B. H. (2002). A multisource exploration of the friendship patterns of children with and without learning disabilities. Journal of Abnormal Child Psychology, 30(2), 127–141.

    Article  Google Scholar 

  • Yannakakis, G. N., & Paiva, A. (2014). Emotion in games. Handbook on Affective Computing, 459–471.

Download references

Acknowledgments

This work was financially supported by an Excellence Grant accorded to Nihal Ouherrou (3UCD2018) and Oussama Elhammoumi (11UAE2017) by the National Center of Scientific and Technical Research (CNRST)-Minister of National Education, Higher Education, Staff Training and Scientific Research, Morocco.

The authors would like to acknowledge the president and staff at Speech-Language Pathology Service-Health center, El Jadida Morocco and also children who have participated in this study. The authors would like also to thank the speech therapist Ilham Elhousni for her valuable suggestions and recommendations and the stuff at the primary school l’Ange Bleu El Jadida for their cooperation.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Nihal Ouherrou.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ouherrou, N., Elhammoumi, O., Benmarrakchi, F. et al. Comparative study on emotions analysis from facial expressions in children with and without learning disabilities in virtual learning environment. Educ Inf Technol 24, 1777–1792 (2019). https://doi.org/10.1007/s10639-018-09852-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10639-018-09852-5

Keywords