Skip to main content
Springer Nature Link
Log in

Adaptive Feedback Based on Student Emotion in a System for Programming Practice

  • Conference paper
  • First Online:
Intelligent Tutoring Systems (ITS 2018)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 10858))

Included in the following conference series:

Abstract

We developed a system for programming practice that provides adaptive feedback based on the presence of confusion on the student. The system provides two types of adaptive feedback. First, it can control the complexity of the exercises presented to the student. Second, it can offer guides for the exercises when needed. These feedback are based on the presence of confusion, which is detected based on the student’s compilations, typing activity, and facial expressions using a hidden Markov model trained on data collected from introductory programming course students. In this paper we discuss the system, the approach for detecting confusion, and the types of adaptive feedback displayed. We tested our system on Japanese university students and discuss the results and their feedback. This study can lay the foundation for the development of intelligent programming tutors that can generate personalized learning content based on the state of the individual learner.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Affectiva developer portal. https://developer.affectiva.com/. Accessed 04 Jan 2018

  2. Codecademy. https://www.codecademy.com. Accessed 04 Jan 2018

  3. Code.org. https://code.org. Accessed 04 Jan 2018

  4. Programming education at elementary school level - ministry of education, culture, sports, science and technology Japan. http://www.mext.go.jp/b_menu/shingi/chousa/shotou/122/attach/1372525.htm. Accessed 04 Jan 2018

  5. Ade-Ibijola, Abejide, Ewert, Sigrid, Sanders, Ian: Introducing Code Adviser: A DFA-driven Electronic Programming Tutor. In: Drewes, Frank (ed.) CIAA 2015. LNCS, vol. 9223, pp. 307–312. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-22360-5_25

    Chapter  Google Scholar 

  6. Arawjo, I., Wang, C.Y., Myers, A.C., Andersen, E., Guimbretière, F.: Teaching programming with gamified semantics. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pp. 4911–4923. ACM (2017)

    Google Scholar 

  7. Balanskat, A., Engelhardt, K.: Computer programming and coding: priorities, school curricula and initiatives across Europe, European schoolnet (2015)

    Google Scholar 

  8. Barros, J.P., Estevens, L., Dias, R., Pais, R., Soeiro, E.: Using lab exams to ensure programming practice in an introductory programming course. ACM SIGCSE Bull. 35(3), 16–20 (2003)

    Article  Google Scholar 

  9. Ben-Ari, M.: Visualization of programming. Improv. Comput. Sci. Educ. 52 (2013)

    Google Scholar 

  10. Bosch, Nigel, D’Mello, Sidney, Mills, Caitlin: What Emotions Do Novices Experience during Their First Computer Programming Learning Session? In: Lane, H.Chad, Yacef, Kalina, Mostow, Jack, Pavlik, Philip (eds.) AIED 2013. LNCS (LNAI), vol. 7926, pp. 11–20. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39112-5_2

    Chapter  Google Scholar 

  11. Cabada, R.Z., Estrada, M.L.B., Hernández, F.G., Bustillos, R.O.: An affective learning environment for Java. In: 2015 IEEE 15th International Conference on Advanced Learning Technologies (ICALT), pp. 350–354. IEEE (2015)

    Google Scholar 

  12. Cooper, S., Dann, W., Pausch, R.: Alice: a 3-D tool for introductory programming concepts. J. Comput. Sci. Coll. 15, 107–116 (2000). Consortium for Computing Sciences in Colleges

    Google Scholar 

  13. Digital Promise: Computational thinking for a computational world (2017)

    Google Scholar 

  14. DMello, S., Jackson, T., Craig, S., Morgan, B., Chipman, P., White, H., Person, N., Kort, B., el Kaliouby, R., Picard, R., et al.: Autotutor detects and responds to learners affective and cognitive states. In: Workshop on Emotional and Cognitive Issues at the International Conference on Intelligent Tutoring Systems, pp. 306–308 (2008)

    Google Scholar 

  15. DMello, S.K., Lehman, B., Graesser, A.: A motivationally supportive affect-sensitive autotutor. In: Calvo, R., D’Mello, S. (eds.) New Perspectives on Affect and Learning Technologies, vol. 3, pp. 113–126. Springer, Heidelberg (2011). https://doi.org/10.1007/978-1-4419-9625-1_9

  16. Ekman, P., Friesen, W.V.: Unmasking the face: a guide to recognizing emotions from facial cues (1975)

    Google Scholar 

  17. Frasson, C., Chalfoun, P.: Managing learners affective states in intelligent tutoring systems. In: Nkambou, R., Bourdeau, J., Mizoguchi, R. (eds.) Advances in Intelligent Tutoring Systems. SCI, vol. 308, pp. 339–358. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-14363-2_17

    Google Scholar 

  18. Fulton, K.: Upside down and inside out: flip your classroom to improve student learning. Learn. Leading Technol. 39(8), 12–17 (2012)

    Google Scholar 

  19. Gerdes, A., Heeren, B., Jeuring, J., van Binsbergen, L.T.: Ask-elle: an adaptable programming tutor for haskell giving automated feedback. Int. J. Artif. Intell. Educ. 27(1), 65–100 (2017)

    Article  Google Scholar 

  20. Grafsgaard, Joseph F., Boyer, Kristy Elizabeth, Lester, James C.: Predicting Facial Indicators of Confusion with Hidden Markov Models. In: D’Mello, Sidney, Graesser, Arthur, Schuller, Björn, Martin, Jean-Claude (eds.) ACII 2011. LNCS, vol. 6974, pp. 97–106. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-24600-5_13

    Chapter  Google Scholar 

  21. Grafsgaard, J.F., Wiggins, J.B., Boyer, K.E., Wiebe, E.N., Lester, J.C.: Automatically recognizing facial indicators of frustration: a learning-centric analysis. In: 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction (ACII), pp. 159–165. IEEE (2013)

    Google Scholar 

  22. Grafsgaard, Joseph F., Wiggins, Joseph B., Boyer, Kristy Elizabeth, Wiebe, Eric N., Lester, James C.: Embodied Affect in Tutorial Dialogue: Student Gesture and Posture. In: Lane, H.Chad, Yacef, Kalina, Mostow, Jack, Pavlik, Philip (eds.) AIED 2013. LNCS (LNAI), vol. 7926, pp. 1–10. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39112-5_1

    Chapter  Google Scholar 

  23. Keuning, H., Heeren, B., Jeuring, J.: Strategy-based feedback in a programming tutor. In: Proceedings of the Computer Science Education Research Conference, pp. 43–54. ACM (2014)

    Google Scholar 

  24. Lahtinen, E., Ala-Mutka, K., Järvinen, H.M.: A study of the difficulties of novice programmers. In: ACM Sigcse Bulletin, vol. 37, pp. 14–18. ACM (2005)

    Article  Google Scholar 

  25. Le, N.T.: A classification of adaptive feedback in educational systems for programming. Systems 4(2), 22 (2016)

    Article  Google Scholar 

  26. Lee, Diane Marie C., Rodrigo, Ma Mercedes T., Baker, Ryan S.J.d, Sugay, Jessica O., Coronel, Andrei: Exploring the Relationship between Novice Programmer Confusion and Achievement. In: D’Mello, Sidney, Graesser, Arthur, Schuller, Björn, Martin, Jean-Claude (eds.) ACII 2011. LNCS, vol. 6974, pp. 175–184. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-24600-5_21

    Chapter  Google Scholar 

  27. Melis, E., Andres, E.: Global feedback in activemath. J. Comput. Math. Sci. Teach. 24(2), 197 (2005)

    Google Scholar 

  28. Myers, B.A.: Taxonomies of visual programming and program visualization. J. Vis. Lang. Comput. 1(1), 97–123 (1990)

    Article  Google Scholar 

  29. Okpo, J., Masthoff, J., Dennis, M., Beacham, N.: Conceptualizing a framework for adaptive exercise selection with personality as a major learner characteristic. In: Adjunct Publication of the 25th Conference on User Modeling, Adaptation and Personalization, pp. 293–298. ACM (2017)

    Google Scholar 

  30. Piaget, J., Cook, M.: The Origins of Intelligence in Children, vol. 8. International Universities Press, New York (1952)

    Book  Google Scholar 

  31. Resnick, M., Maloney, J., Monroy-Hernández, A., Rusk, N., Eastmond, E., Brennan, K., Millner, A., Rosenbaum, E., Silver, J., Silverman, B., et al.: Scratch: programming for all. Commun. ACM 52(11), 60–67 (2009)

    Article  Google Scholar 

  32. Rivers, K., Koedinger, K.R.: Data-driven hint generation in vast solution spaces: a self-improving python programming tutor. Int. J. Artif. Intell. Educ. 27(1), 37–64 (2017)

    Article  Google Scholar 

  33. Rodrigo, M.M.T., Baker, R.S., Jadud, M.C., Amarra, A.C.M., Dy, T., Espejo-Lahoz, M.B.V., Lim, S.A.L., Pascua, S.A., Sugay, J.O., Tabanao, E.S.: Affective and behavioral predictors of novice programmer achievement. In: ACM SIGCSE Bulletin, vol. 41, pp. 156–160. ACM (2009)

    Article  Google Scholar 

  34. Salden, R.J., Paas, F., Van Merriënboer, J.J.: Personalised adaptive task selection in air traffic control: effects on training efficiency and transfer. Learn. Instr. 16(4), 350–362 (2006)

    Article  Google Scholar 

  35. Thompson, N., McGill, T.J.: Genetics with jean: the design, development and evaluation of an affective tutoring system. Educ. Technol. Res. Dev. 65(2), 279–299 (2017)

    Article  Google Scholar 

  36. Tiam-Lee, T.J., Sumi, K.: Analyzing facial expressions and hand gestures in filipino students’ programming sessions. In: 2017 International Conference on Culture and Computing (Culture and Computing), pp. 75–81. IEEE (2017)

    Google Scholar 

  37. Tiam-Lee, T.J., Sumi, K.: A comparison of Filipino and Japanese facial expressions and hand gestures in relation to affective states in programming sessions. In: Workshop on Computation: Theory and Practice 2017 (2017)

    Google Scholar 

  38. Woolf, B., Burleson, W., Arroyo, I., Dragon, T., Cooper, D., Picard, R.: Affect-aware tutors: recognising and responding to student affect. Int. J. Learn. Technol. 4(3–4), 129–164 (2009)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Future University Hakodate, Hakodate, Hokkaido, 041-8655, Japan

    Thomas James Tiam-Lee & Kaoru Sumi

Authors
  1. Thomas James Tiam-Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kaoru Sumi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Thomas James Tiam-Lee or Kaoru Sumi .

Editor information

Editors and Affiliations

  1. Université du Québec, Montreal, Québec, Canada

    Roger Nkambou

  2. NCSU, Raleigh, North Carolina, USA

    Roger Azevedo

  3. University of Saskatchewan, Saskatoon, Saskatchewan, Canada

    Julita Vassileva

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Tiam-Lee, T.J., Sumi, K. (2018). Adaptive Feedback Based on Student Emotion in a System for Programming Practice. In: Nkambou, R., Azevedo, R., Vassileva, J. (eds) Intelligent Tutoring Systems. ITS 2018. Lecture Notes in Computer Science(), vol 10858. Springer, Cham. https://doi.org/10.1007/978-3-319-91464-0_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-91464-0_24

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-91463-3

  • Online ISBN: 978-3-319-91464-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics