Revealing the Learning in Learning Curves

Abstract

Most work on learning curves for ITSs has focused on the knowledge components (or skills) included in the curves, aggregated across students. But an aggregate learning curve need not have the same form as subsets of its underlying data, so learning curves for subpopulations of students may take different forms. We show that disaggregating a skill’s aggregate learning curve into separate learning curves for different student subpopulations can reveal learning: 70% of the skills that did not show learning and were identified as candidates for improvement did show learning when disaggregated. This phenomenon appears to be in part a characteristic of mastery learning. Disaggregated learning curves can reconcile an apparent mismatch between the tutor’s runtime assessment of student knowledge and the post hoc assessment provided by the aggregate learning curve. More precise learning curves can be used to refine Bayesian knowledge tracing parameters and to improve skill model assessment metrics.