From testing to training: Evaluating automated diagnosis in statistics and algebra

Abstract

Much of the controversy over the role of testing in learning stems from its inability to differentiate among sources of error, the focus on outcome rather than problem solving, and the lack of useful information for further learning. To a large extent, these difficulties stem from the constrained form of testing typically used (i.e., multiple choice). Intelligent diagnosis of less constrained problem-solving scenarios is one way to integrate testing and learning. GIDE, a goal-based diagnostic system, provides a step in the direction of such integration. Initial empirical assessment in the domains of elementary statistics and algebra word problems suggests that GIDE's analysis retains information comparable to that provided by multiple-choice items and reflects a global similarity to human evaluators.