Abstract
One should expect the unexpected when deploying intelligent tutoring systems. This paper describes a case study in collecting, warehousing, and analyzing field usage data from two language and culture learning environments, to understand what happened when they were deployed. A data warehousing system, Hoahu, was used to process the raw data and transform it into a relational database to facilitate queries and analysis. The system also supported data annotation by subject matter experts to facilitate comparison of automated assessments against human raters. Errors and inconsistencies in the data were identified and corrected. The resulting data warehouse has proven valuable for understanding the trajectory of learning over extended periods of time and analyzing the strengths and weaknesses of complex interactive subsystems such as spoken dialog systems.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Jack Mostow, J., Beck, J.: What, How, and Why should Tutors Log? In: Proceedings of EDM 2009, pp. 269–278 (2009)
Johnson, W.L., Wu, S.: Assessing aptitude for learning with a serious game for foreign language and culture. In: Woolf, B.P., Aïmeur, E., Nkambou, R., Lajoie, S. (eds.) ITS 2008. LNCS, vol. 5091, pp. 520–529. Springer, Heidelberg (2008)
Koedinger, K., Cunningham, K., Skogsholm, A., Leber, B.: An open repository and analysis tools for fine-grained, longitudinal learner data. In: Proceedings of EDM 2008, pp. 157–166 (2008)
Razzaq, L., Patvarczki, J., Almeida, S.F., Vartak, M., Feng, M., Heffernan, N.T., Koedinger, K.: The ASSISTment builder: Supporting the Life-cycle of ITS Content Creation. IEEE Transactions on Learning Technologies, Special Issue on Real-World Applications of Intelligent Tutoring Systems 2(2), 157–166 (2009)
Jordan, P., Litman, D., Lipschultz, M., Drummond, J.: Evidence of Misunderstandings in Tutorial Dialogue and their Impact on Learning. In: Artificial Intelligence in Education, pp. 125–132. IOS Press, Amsterdam (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Johnson, W.L., Ashish, N., Bodnar, S., Sagae, A. (2010). Expecting the Unexpected: Warehousing and Analyzing Data from ITS Field Use. In: Aleven, V., Kay, J., Mostow, J. (eds) Intelligent Tutoring Systems. ITS 2010. Lecture Notes in Computer Science, vol 6095. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-13437-1_68
Download citation
DOI: https://doi.org/10.1007/978-3-642-13437-1_68
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-13436-4
Online ISBN: 978-3-642-13437-1
eBook Packages: Computer ScienceComputer Science (R0)