skip to main content
10.1145/2723576.2723608acmotherconferencesArticle/Chapter ViewAbstractPublication PageslakConference Proceedingsconference-collections
short-paper

Tracking student progress in a game-like learning environment with a Monte Carlo Bayesian knowledge tracing model

Published: 16 March 2015 Publication History

Abstract

The Bayesian Knowledge Tracing (BKT) model is a popular model used for tracking student progress in learning systems such as an intelligent tutoring system. However, the model is not free of problems. Well-recognized problems include the identifiability problem and the empirical degeneracy problem. Unfortunately, these problems are still poorly understood and how they should be dealt with in practice is unclear. Here, we analyze the mathematical structure of the BKT model, identify a source of the difficulty, and construct a simple Monte Carlo BKT model to analyze the problem in real data. Using the student activity data obtained from the ramp task module at the Concord Consortium, we find that the Monte Carlo BKT analysis is capable of detecting the identifiability problem and the empirical degeneracy problem, and, more generally, gives an excellent summary of the student learning data. In particular, the student activity monitoring parameter M emerges as the central parameter.

References

[1]
R. S. Baker, A. T. Corbett, S. M. Gowda, A. Z. Wagner, B. A. MacLaren, L. R. Kauffman, A. P. Mitchell, and S. Giguere. Contextual slip and prediction of student performance after use of an intelligent tutor. In User Modeling, Adaptation, and Personalization, pages 52--63. Springer, 2010.
[2]
R. S. J. d. Baker, A. T. Corbett, and V. Aleven. More accurate student modeling through contextual estimation of slip and guess probabilities in Bayesian Knowledge Tracing. In B. P. Woolf, E. Aïmeur, R. Nkambou, and S. Lajoie, editors, Intelligent Tutoring Systems, number 5091 in Lecture Notes in Computer Science, pages 406--415. Springer Berlin Heidelberg, Jan. 2008.
[3]
J. E. Beck. Difficulties in inferring student knowledge from observations (and why you should care). In Educational Data Mining: Supplementary Proceedings of the 13th International Conference of Artificial Intelligence in Education, pages 21--30, 2007.
[4]
J. E. Beck and K.-m. Chang. Identifiability: A fundamental problem of student modeling. In C. Conati, K. McCoy, and G. Paliouras, editors, User Modeling 2007, number 4511 in Lecture Notes in Computer Science, pages 137--146. Springer Berlin Heidelberg, Jan. 2007.
[5]
A. T. Corbett and J. R. Anderson. Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction, 4(4): 253--278, Dec. 1994.
[6]
J. D. Gobert, M. Sao Pedro, J. Raziuddin, and R. S. Baker. From log files to assessment metrics: Measuring students' science inquiry skills using educational data mining. Journal of the Learning Sciences, 22(4): 521--563, Sept. 2013.
[7]
H.-S. Lee, G.-H. Gweon, C. Dorsey, R. Tinker, W. Finzer, D. Damelin, N. Kimball, A. Pallant, and T. Lord. How does Bayesian Knowledge Tracing model emergence of knowledge about a mechanical system? In LAK15 Conference Proceedings, LAK15, 2015.
[8]
B. v. d. Sande. Properties of the Bayesian Knowledge Tracing model. JEDM - Journal of Educational Data Mining, 5(2): 1--10, July 2013.

Cited By

View all
  • (2024)Adaptation of the Multi-Concept Multivariate Elo Rating System to Medical Students' Training DataProceedings of the 14th Learning Analytics and Knowledge Conference10.1145/3636555.3636858(123-133)Online publication date: 18-Mar-2024
  • (2024)Systematic Review and Analysis of EDM for Predicting the Academic Performance of StudentsJournal of The Institution of Engineers (India): Series B10.1007/s40031-024-00998-0105:4(1021-1071)Online publication date: 4-Feb-2024
  • (2023)A Remote View into the Classroom: Analyzing Teacher Use of Digitally Enhanced Educative Curriculum Materials in Support of Student LearningJournal of Science Teacher Education10.1080/1046560X.2023.220459135:2(127-152)Online publication date: 26-May-2023
  • Show More Cited By

Index Terms

  1. Tracking student progress in a game-like learning environment with a Monte Carlo Bayesian knowledge tracing model

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Other conferences
      LAK '15: Proceedings of the Fifth International Conference on Learning Analytics And Knowledge
      March 2015
      448 pages
      ISBN:9781450334174
      DOI:10.1145/2723576
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 16 March 2015

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. Bayesian knowledge tracing
      2. Monte Carlo
      3. educational data mining

      Qualifiers

      • Short-paper

      Funding Sources

      Conference

      LAK '15

      Acceptance Rates

      LAK '15 Paper Acceptance Rate 20 of 74 submissions, 27%;
      Overall Acceptance Rate 236 of 782 submissions, 30%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)12
      • Downloads (Last 6 weeks)1
      Reflects downloads up to 11 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Adaptation of the Multi-Concept Multivariate Elo Rating System to Medical Students' Training DataProceedings of the 14th Learning Analytics and Knowledge Conference10.1145/3636555.3636858(123-133)Online publication date: 18-Mar-2024
      • (2024)Systematic Review and Analysis of EDM for Predicting the Academic Performance of StudentsJournal of The Institution of Engineers (India): Series B10.1007/s40031-024-00998-0105:4(1021-1071)Online publication date: 4-Feb-2024
      • (2023)A Remote View into the Classroom: Analyzing Teacher Use of Digitally Enhanced Educative Curriculum Materials in Support of Student LearningJournal of Science Teacher Education10.1080/1046560X.2023.220459135:2(127-152)Online publication date: 26-May-2023
      • (2020)The Invisible Breadcrumbs of Digital LearningHandbook of Research on Digital Learning10.4018/978-1-5225-9304-1.ch019(302-316)Online publication date: 2020
      • (2018)Learning meets assessmentBehaviormetrika10.1007/s41237-018-0070-z45:2(457-474)Online publication date: 20-Oct-2018

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media