Shift Error Detection in Standardized Exams

Abstract

Computer-graded multiple choice examinations are a familiar and dreaded part of most student’s lives. Many test takers are particularly fearful of form-filling shift errors, where absent-mindedly marking the answer to (say) question 32 in position 31 causes a long run of answers to be successively displaced. Test-taking strategies where students answer questions out of sequence (such as answering easy questions first) seem particularly likely to cause unrecognized shift errors. Such errors can result in many correct answers being marked wrong, and significantly lower the exam taker’s score.

In this paper, we consider the question of whether these shift errors can be accurately recognized and corrected for. Our results suggest that students are right to fear such errors, and that a non-trivial fraction of multiple-choice exams appear to contain significant shift errors. In particular, we found evidence that 1%–2% of multiple-choice exam papers at SUNY Stony Brook contain likely shift errors, each causing the loss of about 10% of the student's grade. Given the importance of standardized examinations, clerical mistakes should not be allowed to have such an impact on the student's score. If our observed shift error rate holds across the millions of examinations given annually, this is a serious but yet unstudied problem. Indeed, based on the preliminary results in the paper, we have begun working with The College Board to study the prevalence of uncorrected shift-errors in SAT examinations.

This work is partially supported by NSF Grant CCR-9625669 and ONR Award N00149710589.

Partially supported by the 1998 GAAN graduate fellowship P200A97030199.