Binglin Chen
ABSTRACT
To design good assessments, it is useful to have an estimate of the difficulty of a novel exam question before running an exam. In this paper, we study a collection of a few hundred automatic item generators (short computer programs that generate a variety of unique item instances) and show that their exam difficulty can be roughly predicted from student performance on the same generator during pre-exam practice. Specifically, we show that the rate that students correctly respond to a generator on an exam is on average within 5% of the correct rate for those students on their last practice attempt. This study is conducted with data from introductory undergraduate Computer Science and Mechanical Engineering courses.
- Binglin Chen, Matthew West, and Craig Zilles. 2018. How much randomization is needed to deter collaborative cheating on asynchronous exams?. In Learning at Scale. Google Scholar
Digital Library
- Hermann Ebbinghaus. 1913. Memory: A contribution to experimental psychology. Number 3. University Microfilms.Google ScholarCross Ref
- M.J. Gierl and T.M. Haladyna. 2013. Automatic Item Generation: Theory and practice. Routledge.Google Scholar
- Ray Hembree. 1988. Correlates, causes, effects, and treatment of test anxiety. Review of educational research 58, 1 (1988), 47--77.Google Scholar
- Nate Kornell, Matthew Jensen Hays, and Robert A Bjork. 2009. Unsuccessful retrieval attempts enhance subsequent learning. Journal of Experimental Psychology: Learning, Memory, and Cognition 35, 4 (2009), 989.Google ScholarCross Ref
- Constance A Mara and Robert A Cribbie. 2012. Paired-samples tests of equivalence. Communications in Statistics-Simulation and Computation 41, 10 (2012), 1928--1943.Google ScholarCross Ref
- Matthew West, Geoffrey L. Herman, and Craig Zilles. 2015. PrairieLearn: Mastery-based Online Problem Solving with Adaptive Scoring and Recommendations Driven by Machine Learning. In 2015 ASEE Annual Conference & Exposition. ASEE Conferences, Seattle, Washington. https://peer.asee.org/24575.Google ScholarCross Ref
- Craig Zilles, Matthew West, and David Mussulman. 2016. Student Behavior in Selecting an Exam Time in a Computer-Based Testing Facility. In 2016 ASEE Annual Conference & Exposition. ASEE Conferences, New Orleans, Louisiana. https://peer.asee.org/25896.Google ScholarCross Ref
- Craig Zilles, Matthew West, David Mussulman, and Timothy Bretl. 2018. Making testing less trying: Lessons learned from operating a Computer-Based Testing Facility. In 2018 IEEE Frontiers in Education (FIE) Conference. San Jose, California.Google ScholarDigital Library
Comments