Abstract
This paper concerns the multimodal inference of complex mental states in the intelligent tutoring domain. The research aim is to provide intervention strategies in response to a detected mental state, with the goal being to keep the student in a positive affect realm to maximize learning potential. The research follows an ethnographic approach in the determination of affective states that naturally occur between students and computers. The multimodal inference component will be evaluated from video and audio recordings taken during classroom sessions. Further experiments will be conducted to evaluate the affect component and educational impact of the intelligent tutor.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Paleari, M., Lisetti, C.: Toward Multimodal Fusion of Affective Cues. In: Proceedings of the1st ACM International Workshop on Human-Centered Multimedia (2006)
Ekman, P., Friesen, W.: Nonverbal behaviour in pschotherapy research. Research in Pschotherapy 3, 179–216 (1968)
Ingleton, C.: Emotion in learning: a neglected dynamic. Cornerstones of Higher Education 22, 86–99 (2000)
Wolcott, L.: The distance teacher as reflective practitioner. Educational Technology 1, 39–43 (1995)
Murray, T.: Authoring Intelligent Tutoring Systems: An Analysis of the State of the Art. Internal Journal of Artificial Intelligence in Education 10, 98–129 (1999)
Ohlsson, S.: Some Principles of Intelligent Tutoring. Instructional Science 14, 293–326 (1986)
Mostow, J., Aist, G.: Giving help and praise in a reading tutor with imperfect listening - because automated speech recognition means never being able to say you’re certain. CALICO Journal 16, 407–424 (1999)
Beck, J.E., Jia, P., Sison, J., Mostow, J.: Predicting student help-request behavior in an intelligent tutor for reading. In: Proceedings of the 9th International Conference on User Modeling (2003)
Herrington, J., Oliver, R., Reeves, T.C.: Patterns of engagement in authentic online learning environments. Australian Journal of Educational Technology 19, 59–71 (2003)
Robison, J., McQuiggan, S., Lester, J.: Evaluating the consequences of affective feedback in intelligent tutoring systems, pp. 1–6 (2009)
D’Mello, S., Taylor, R.S., Graesser, A.: Monitoring Affective Trajectories during Complex Learning. In: Proceedings of the 29th Annual Meeting of the Cognitive Science Society, Austin, TX, pp. 203–208 (2007)
Litman, D., Forbes, K.: Recognizing Emotions from Student Speech in Tutoring Dialogues. In: Proceedings of the ASRU 2003 (2003)
D’Mello, S., Jackson, T., Craig, S., Morgan, B., Chipman, P., White, H., Person, N., Kort, B., el Kaliouby, R., Picard, R.W., Graesser, A.: AutoTutor Detects and Responds to Learners Affective and Cognitive States. In: Workshop on Emotional and Cognitive Issues at the International Conference of Intelligent Tutoring Systems (2008)
Baron-Cohen, S., Golan, O., Wheelwright, S., Hill, J.J.: Mind Reading: The Interactive Guide to Emotions. Jessica Kingsley Publishers, London (2004)
Ekman, P., Friesen, W.V.: Facial Action Coding System: a technique for the measurement of facial movement. Consulting Psychologists Press, Palo Alto (1978)
el Kaliouby, R., Robinson, P.: Real-time Inference of Complex Mental States from Facial Expressions and Head Gestures. In: Real-Time Vision for Human-Computer Interaction, pp. 181–200. Springer, Heidelberg (2005)
Sobol-Shikler, T., Robinson, P.: Classification of complex information: inference of co-occurring affective states from their expressions in speech. IEEE Transactions on Pattern Analysis and Machine Intelligence 32, 1284–1297 (2010)
Pfister, T., Robinson, P.: Speech emotion classification and public speaking skill assessment. In: Salah, A.A., Gevers, T., Sebe, N., Vinciarelli, A. (eds.) HBU 2010. LNCS, vol. 6219, pp. 151–162. Springer, Heidelberg (2010)
Sharma, R., Pavlovic, V.I., Huang, T.S.: Toward a multi- modal human computer interface. In: Beun, R.-J. (ed.) Multimodal Cooperative Communication, pp. 89–112. Springer, Heidelberg (2001)
Zeng, Z., Pantic, M., Huang, T.S.: Emotion Recognition Based on Multimodal Information. In: Affective Information Processing (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Banda, N., Robinson, P. (2011). Multimodal Affect Recognition in Intelligent Tutoring Systems. In: D’Mello, S., Graesser, A., Schuller, B., Martin, JC. (eds) Affective Computing and Intelligent Interaction. ACII 2011. Lecture Notes in Computer Science, vol 6975. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24571-8_21
Download citation
DOI: https://doi.org/10.1007/978-3-642-24571-8_21
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-24570-1
Online ISBN: 978-3-642-24571-8
eBook Packages: Computer ScienceComputer Science (R0)