Skip to main content
SpringerLink
Log in

Predicting Learning from Student Affective Response to Tutor Questions

  • Conference paper
  • First Online:
Intelligent Tutoring Systems (ITS 2016)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 9684))

Included in the following conference series:

Abstract

Modeling student learning during tutorial interaction is a central problem in intelligent tutoring systems. While many modeling techniques have been developed to address this problem, most of them focus on cognitive models in conjunction with often-complex domain models. This paper presents an analysis suggesting that observing students’ multimodal behaviors may provide deep insight into student learning at critical moments in a tutorial session. In particular, this work examines student facial expression, electrodermal activity, posture, and gesture immediately following inference questions posed by human tutors. The findings show that for human-human task-oriented tutorial dialogue, facial expression and skin conductance response following tutor inference questions are highly predictive of student learning gains. These findings suggest that with multimodal behavior data, intelligent tutoring systems can make more informed adaptive decisions to support students effectively.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Hartley, D., Mitrović, A.: Supporting learning by opening the student model. In: Cerri, S.A., Gouardéres, G., Paraguaçu, F. (eds.) ITS 2002. LNCS, vol. 2363, pp. 453–462. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  2. Gluga, R.: Long term student learner modeling and curriculum mapping. In: Aleven, V., Kay, J., Mostow, J. (eds.) ITS 2010, Part II. LNCS, vol. 6095, pp. 227–229. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  3. Corbett, A.T., Anderson, J.R.: Student modeling and mastery learning in a computer-based programming tutor. In: Proceedings of the 2nd International Conference on Intelligent Tutoring Systems, pp. 413–420 (1992)

    Google Scholar 

  4. Stevens, R., Soller, A., Cooper, M., Sprang, M.: Modeling the development of problem solving skills in chemistry with a web-based tutor. In: Lester, J.C., Vicari, R.M., Paraguaçu, F. (eds.) ITS 2004. LNCS, vol. 3220, pp. 580–591. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  5. Corbett, A.T., Anderson, J.R.: Knowledge tracing: modeling the acquisition of procedural knowledge. User Model. User-Adap. Inter. 4, 253–278 (1995)

    Article  Google Scholar 

  6. Pavlik Jr., P.I., Cen, H., Koedinger, K.R.: Performance factors analysis - a new alternative to knowledge tracing. In: Proceedings of the 14th International Conference on Artificial Intelligence in Education, pp. 531–538 (2009)

    Google Scholar 

  7. Grafsgaard, J.F., Wiggins, J.B., Boyer, K.E., Wiebe, E.N., Lester, J.C.: Automatically recognizing facial expression: predicting engagement and frustration. In: Proceedings of the 6th International Conference on Educational Data Mining, pp. 43–50 (2013)

    Google Scholar 

  8. Grafsgaard, J.F., Wiggins, J.B., Boyer, K.E., Wiebe, E.N., Lester, J.C.: Automatically recognizing facial indicators of frustration: a learning-centric analysis. In: Proceedings of the Humane Association Conference on Affective Computing and Intelligent Interaction, pp. 159–165 (2013)

    Google Scholar 

  9. Scherer, S., Weibel, N., Morency, L.P., Oviatt, S.: Multimodal prediction of expertise and leadership in learning groups. In: Proceedings of the 1st International Workshop on Multimodal Learning Analytics (2012)

    Google Scholar 

  10. Biel, J.I., Teijeiro-Mosquera, L., Gatica-Perez, D.: FaceTube: predicting personality from facial expressions of emotion in online conversational video. In: Proceedings of the 14th International Conference on Multimodal Interaction, pp. 53–56 (2012)

    Google Scholar 

  11. Oviatt, S., Cohen, A.: Written and multimodal representations as predictors of expertise and problem-solving success in mathematics. In: Proceedings of the 15th International Conference on Multimodal Interaction, pp. 599–606 (2013)

    Google Scholar 

  12. D’Mello, S.K., Graesser, A.C.: Multimodal semi-automated affect detection from conversational cues, gross body language, and facial features. User Model. User-Adap. Inter. 20, 147–187 (2010)

    Article  Google Scholar 

  13. Arroyo, I., Cooper, D.G., Burleson, W., Woolf, B.P., Muldner, K., Christopherson, R.: Emotion sensors go to school. In: Proceedings of the 14th International Conference on Artificial Intelligence in Education, pp. 17–24 (2009)

    Google Scholar 

  14. Stein, N.L., Levine, L.J.: Making sense out of emotion: the representation and use of goal-structured knowledge. In: Psychological and Biological Approaches to Emotion, pp. 45–73 (1990)

    Google Scholar 

  15. Piaget, J.: The Origins of Intelligence. International University Press, New York (1952)

    Book  Google Scholar 

  16. Hardy, M., Wiebe, E.N., Grafsgaard, J.F., Boyer, K.E., Lester, J.C.: Physiological responses to events during training: Use of skin conductance to inform future adaptive learning systems. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp. 2101–2105 (2013)

    Google Scholar 

  17. Ha, E.Y., Grafsgaard, J.F., Mitchell, C.M., Boyer, K.E., Lester, J.C.: Combining verbal and nonverbal features to overcome the ‘Information Gap’ in task-oriented dialogue. In: Proceedings of the 13th Annual SIGDIAL Meeting on Discourse and Dialogue, pp. 247–256 (2012)

    Google Scholar 

  18. Mitchell, C.M., Ha, E.Y., Boyer, K.E., Lester, J.C.: Learner characteristics and dialogue: recognising effective and student-adaptive tutorial strategies. Int. J. Learn. Technol. 8(4), 382–403 (2013)

    Article  Google Scholar 

  19. Vail, A.K., Boyer, K.E.: Adapting to personality over time: examining the effectiveness of dialogue policy progressions in task-oriented interaction. In: Proceedings of the 15th Annual SIGDIAL Meeting on Discourse and Dialogue, pp. 41–50 (2014)

    Google Scholar 

  20. Vail, A.K., Boyer, K.E.: Identifying effective moves in tutorial dialogue: on the refinement of speech act annotation schemes. In: Proceedings of the 12th International Conference on Intelligent Tutoring Systems, pp. 199–209 (2014)

    Google Scholar 

  21. Graesser, A.C., Olde, B.A.: How does one know whether a person understands a device? The quality of the questions the person asks when the device breaks down. J. Educ. Psychol. 95(3), 524–536 (2003)

    Article  Google Scholar 

  22. Littlewort, G., Whitehill, J., Wu, T., Fasel, I., Frank, M., Javier, M., Bartlett, M.: The computer expression recognition toolbox (CERT). In: Proceedings of the IEEE International Conference on Automatic Face & Gesture Recognition and Workshops, pp. 298–305 (2011)

    Google Scholar 

  23. Lepper, M.R., Woolverton, M.: The wisdom of practice: Lessons learned from the study of highly effective tutors. Impact of Psychological Factors on Education, Improving Academic Achievement, pp. 135–158 (2002)

    Google Scholar 

  24. Boucsein, W.: Electrodermal Activity. Springer Science & Business Media, NewYork (2012)

    Book  Google Scholar 

  25. Benedek, M., Kaernbach, C.: A continuous measure of phasic electrodermal activity. J. Neurosci. Methods 190, 80–91 (2010)

    Article  Google Scholar 

  26. Vural, E., Cetin, M., Ercil, A., Littlewort, G., Bartlett, M., Movellan, J.: Drowsy driver detection through facial movement analysis. In: Proceedings of the 12th International Conference on Human-Computer Interaction, pp. 6–18 (2007)

    Google Scholar 

  27. Mortillaro, M., Mehu, M., Scherer, K.R.: Subtly different positive emotions can be distinguished by their facial expressions. Soc. Psychol. Pers. Sci. 2(3), 262–271 (2011)

    Article  Google Scholar 

  28. Goldin, I.M., Carlson, R.: Learner differences and hint content. In: Lane, H.C., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS, vol. 7926, pp. 522–531. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  29. San Pedro, M.O.Z., Baker, R.S.J., Gowda, S.M., Heffernan, N.T.: Towards an understanding of affect and knowledge from student interaction with an intelligent tutoring system. In: Lane, H.C., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS, vol. 7926, pp. 41–50. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  30. D’Mello, S., Lehman, B., Pekrun, R., Graesser, A.: Confusion can be beneficial for learning. Learn. Instr. 29, 153–170 (2004)

    Article  Google Scholar 

Download references

Acknowledgements

The authors wish to thank the members of the LearnDialogue and Intellimedia groups at North Carolina State University for their helpful input. This work is supported in part by the Department of Computer Science at North Carolina State University and the National Science Foundation through Grants IIS-1409639, CNS-1453520, and a Graduate Research Fellowship. Any opinions, findings, conclusions, or recommendations expressed in this report are those of the participants, and do not necessarily represent the official views, opinions, or policy of the National Science Foundation.

Author information

Authors and Affiliations

  1. Department of Computer Science, North Carolina State University, Raleigh, NC, USA

    Alexandria K. Vail & James C. Lester

  2. Department of Psychology, North Carolina State University, Raleigh, NC, USA

    Joseph F. Grafsgaard

  3. Department of STEM Education, North Carolina State University, Raleigh, NC, USA

    Eric N. Wiebe

  4. Department of Computer and Information Science and Engineering, University of Florida, Gainesville, FL, USA

    Kristy Elizabeth Boyer

Authors
  1. Alexandria K. Vail
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joseph F. Grafsgaard
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kristy Elizabeth Boyer
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Eric N. Wiebe
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. James C. Lester
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alexandria K. Vail .

Editor information

Editors and Affiliations

  1. Roma Tre University , Rome, Italy

    Alessandro Micarelli

  2. Carnegie Mellon University , Pittsburgh, Pennsylvania, USA

    John Stamper

  3. Neoanalysis Ltd , Athens, Greece

    Kitty Panourgia

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Vail, A.K., Grafsgaard, J.F., Boyer, K.E., Wiebe, E.N., Lester, J.C. (2016). Predicting Learning from Student Affective Response to Tutor Questions. In: Micarelli, A., Stamper, J., Panourgia, K. (eds) Intelligent Tutoring Systems. ITS 2016. Lecture Notes in Computer Science(), vol 9684. Springer, Cham. https://doi.org/10.1007/978-3-319-39583-8_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-39583-8_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-39582-1

  • Online ISBN: 978-3-319-39583-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation