Skip to main content
SpringerLink
Log in

Improving Automatic Affect Recognition on Low-Level Speech Features in Intelligent Tutoring Systems

  • Conference paper
  • First Online:
Design for Teaching and Learning in a Networked World (EC-TEL 2015)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 9307))

Included in the following conference series:

  • 4237 Accesses

Abstract

Currently, a lot of research in the field of intelligent tutoring systems is concerned with recognising student’s emotions and affects. The recognition is done by extracting features from information sources like speech, typing and mouse clicking behaviour or physiological sensors. According to the state-of-the-art support vector machines are the best performing classification models for those kinds of features. However, single classification models often do not deliver the best possible performance. Hence, we propose an approach for further improving the affect recognition performance, which is based on ideas from ensemble approaches and feature selection methods. The approach is proven by experiments on low-level speech features extracted from data which was collected in a study with German students solving mathematical tasks. In these experiments the proposed approach reached on average an affect recognition performance improvement of about 59 % in comparison to a single SVM.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

References

  1. Arroyo, I., Woolf, B.P., Burelson, W., Muldner, K., Rai, D., Tai, M.: A multimedia adaptive tutoring system for mathematics that addresses cognition, metacognition and affect. Int. J. Artif. Intell. Edu. 24, 387–426 (2014)

    Article  Google Scholar 

  2. Baker, R.S.J.D., Gowda, S., Wixon, M., Kalka, J., Wagner, A., Salvi, A., Aleven, V., Kusbit, G., Ocumpaugh, J. and Rossi, L.: Towards sensor-free affect detection in cognitive tutor algebra. In: Proceedings of the 5th International Conference on Educational Data Mining (EDM 2012), pp. 126–133 (2012)

    Google Scholar 

  3. Boser, B.E., Guyon, I., Vapnik, V.: A training algorithm for optimal margin classifiers. In: Proceedings of the Fifth Annual Workshop on Computational Learning Theory, pp. 144–152. ACM Press (1992)

    Google Scholar 

  4. Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2(3), 1–27 (2011)

    Article  Google Scholar 

  5. Cortes, C., Vapnik, V.: Support-vector network. Mach. Learn. 20, 273–297 (1995)

    Article  MATH  Google Scholar 

  6. Epp, C., Lippold, M., Mandryk, R.L.: Identifying emotional states using keystroke dynamics. In: Proceedings of the 2011 Annual Conference on Human Factors in Computing Systems (CHI 2011), pp. 715–724 (2011)

    Google Scholar 

  7. Grawemeyer, B., Gutierrez-Santos, S., Holmes, W., Mavrikis, M., Rummel, N., Mazziotti, C. Janning, R.: Talk, tutor, explore, learn: intelligent tutoring and exploration for robust learning. In: Proceedings of the 17th International Conference on Artificial Intelligence in Education (AIED) (2015)

    Google Scholar 

  8. Hsu, C.W., Chang, C.C., Lin, C.J.: A practical guide to support vector classification. Technical report, Department of Computer Science, National Taiwan University (2011). http://www.csie.ntu.edu.tw/cjlin/

  9. Hu, X., Tang, L., Tang, J., Liu, H.: Exploiting social relations for sentiment analysis in microblogging. In: Proceedings of the Sixth ACM WSDM Conference (WSDM 2013) (2013)

    Google Scholar 

  10. Janning, R., Schatten, C., Schmidt-Thieme, L.: Multimodal affect recognition for adaptive intelligent tutoring systems. In: Extended Proceedings of the 7th International Conference on Educational Data Mining (EDM 2014), pp. 171–178 (2014)

    Google Scholar 

  11. Janning, R., Schatten, C., Schmidt-Thieme, L.: Feature analysis for affect recognition supporting task sequencing in adaptive intelligent tutoring systems. In: Proceedings of the European Conference on Technology Enhanced Learning (EC-TEL 2014), pp. 179–192 (2014)

    Google Scholar 

  12. Janning, R., Schatten, C., Schmidt-Thieme, L., Backfried, G.: An SVM plait for improving affect recognition in intelligent tutoring systems. In: Proceedings of the IEEE International Conference on Tools with Artificial Intelligence (ICTAI) (2014)

    Google Scholar 

  13. Luz, S.: Automatic identification of experts and performance prediction in the multimodal math data corpus through analysis of speech interaction. In: Second International Workshop on Multimodal Learning Analytics, Sydney Australia (2013)

    Google Scholar 

  14. Mavrikis, M.: Data-driven modelling of students interactions in an ILE. In: Proceedings of the International Conference on Educational Data Mining (EDM 2008), pp. 87–96 (2008)

    Google Scholar 

  15. D’Mello, S.K., Craig, S.D., Witherspoon, A., McDaniel, B., Graesser, A.: Automatic detection of learners affect from conversational cues. User Model User-Adap Inter. 18, 45–80 (2008). doi:10.1007/s11257-007-9037-6

    Article  Google Scholar 

  16. D’Mello, S.K., Graesser, A.: Language and discourse are powerful signals of student emotions during tutoring. IEEE Trans. Learn. Technol. 5(4), 304–317 (2012)

    Article  Google Scholar 

  17. Moore, J.D., Tian, L., Lai, C.: Word-level emotion recognition using high-level features. In: Gelbukh, A. (ed.) CICLing 2014, Part II. LNCS, vol. 8404, pp. 17–31. Springer, Heidelberg (2014)

    Chapter  Google Scholar 

  18. Morency, L.P., Oviatt, S., Scherer, S., Weibel, N., Worsley, M.: ICMI 2013 grand challenge workshop on multimodal learning analytics. In: Proceedings of the 15th ACM on International Conference on Multimodal Interaction (ICMI 2013), pp. 373–378 (2013)

    Google Scholar 

  19. Pardos, Z.A., Baker, R.S.J.D., San Pedro, M., Gowda, S.M., Gowda, S.M.: Affective states and state tests: investigating how affect and engagement during the school year predict end-of-year learning outcomes. Inaugural issue J. Learn. Anal. 1(1), 107–128 (2014)

    Google Scholar 

  20. Purandare, A., Litman, D.: Humor: prosody analysis and automatic recognition for F * R * I * E * N * D * S *. In: Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing (EMNLP 2006), pp. 208–215 (2006)

    Google Scholar 

  21. Qi, F., Bao, C., Liu, Y.: A novel two-step SVM classifier for voiced/unvoiced/silence classification of speech. In: International Symposium on Chinese Spoken Language Processing, pp. 77–80 (2004)

    Google Scholar 

  22. Sadegh, M., Ibrahim, R., Othman, Z.A.: Opinion mining and sentiment analysis: a survey. Int. J. Comput. Technol. 2(3), 171–178 (2012)

    Article  Google Scholar 

  23. Saif, H., He, Y., Alani, H.: Semantic sentiment analysis of Twitter. In: Cudré-Mauroux, P., Heflin, J., Sirin, E., Tudorache, T., Euzenat, J., Hauswirth, M., Parreira, J.X., Hendler, J., Schreiber, G., Bernstein, A., Blomqvist, E. (eds.) ISWC 2012, Part I. LNCS, vol. 7649, pp. 508–524. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  24. San Pedro, M.O.C., Baker, R.S.J.D., Bowers, A., Heffernan, N.: Predicting college enrollment from student interaction with an intelligent tutoring system in middle school. In: Proceedings of the 6th International Conference on Educational Data Mining (EDM 2013), pp. 177–184 (2013)

    Google Scholar 

  25. Schuller, B., Batliner, A., Steidl, S., Seppi, D.: Recognising realistic emotions and affect in speech: state of the art and lessons learnt from the first challenge. Speech Commun. 53, 1062–1087 (2011)

    Article  Google Scholar 

  26. Ting, K.M., Witten, I.H.: Issues in stacked generalization. J. Artif. Intel. Res. 10, 271–289 (1999)

    MATH  Google Scholar 

  27. Worsley, M., Blikstein, P.: What’s an expert? using learning analytics to identify emergent markers of expertise through automated speech, sentiment and sketch analysis. In: Proceedings of the 4th International Conference on Educational Data Mining (EDM 2011), pp. 235–240 (2011)

    Google Scholar 

Download references

Acknowledgements

The research leading to the results reported here has received funding from the European Union Seventh Framework Programme (FP7/2007 – 2013) under grant agreement no. 318051 – iTalk2Learn project (www.italk2learn.eu).

Author information

Authors and Affiliations

  1. Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Hildesheim, Germany

    Ruth Janning, Carlotta Schatten & Lars Schmidt-Thieme

Authors
  1. Ruth Janning
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carlotta Schatten
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lars Schmidt-Thieme
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ruth Janning .

Editor information

Editors and Affiliations

  1. University of Bath Spa, Bath, UK

    Gráinne Conole

  2. Jožef Stefan Institute, Ljubljana, Slovenia

    Tomaž Klobučar

  3. Technische Universität Darmstadt, Darmstadt, Germany

    Christoph Rensing

  4. Technische Universität Darmstadt, Darmstadt, Germany

    Johannes Konert

  5. IAE Lyon, Université Jean Moulin Lyon 3, Lyon, France

    Elise Lavoué

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Janning, R., Schatten, C., Schmidt-Thieme, L. (2015). Improving Automatic Affect Recognition on Low-Level Speech Features in Intelligent Tutoring Systems. In: Conole, G., Klobučar, T., Rensing, C., Konert, J., Lavoué, E. (eds) Design for Teaching and Learning in a Networked World. EC-TEL 2015. Lecture Notes in Computer Science(), vol 9307. Springer, Cham. https://doi.org/10.1007/978-3-319-24258-3_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-24258-3_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-24257-6

  • Online ISBN: 978-3-319-24258-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation