Yang Song
Lucas Layman
ABSTRACT
In an entry-level programming course, the instructor needs to assess students' prior knowledge or to evaluate their learning at important milestones. The Second CS1 Assessment (SCS1) is one of the best-known validated tests of programming knowledge. It is a multiple-choice test written in pseudo-code, covering concepts commonly presented in Computer Science 1 (CS1) courses. However, SCS1 is known to be an unwieldy assessment - the questions tend to be difficult and thereby may not provide good discrimination between students with average competencies against the weak ones. In our project, we created a set of short deliberate-practice questions for CS1 topics and used them to evaluate students' prior knowledge at the beginning of a Computer Science 2 (CS2) course. This set of deliberate-practice questions consists of 127 multiple-choice questions.
Both SCS1 questions and our short deliberate-practice questions were given to CS2 students as pre-CS2 assessments on an online practice system called HawkQB. Using Item Response Theory (IRT), we analyzed students' responses in both sets of questions as pre-CS2 assessments to examine whether the SCS1 questions are suitable to be used as a pre-CS2 assessment instrument. We found that the SCS1 questions are of greater difficulty and low discrimination compare with our short deliberate-practice questions based on the data from students in four sections of CS2 in fall 2019. Considering the time gap between CS1 and CS2 may vary, we question the suitability of SCS1 as a pre-CS2 assessment used in a formal test setup in the first or second week of CS2 courses.
- R. H. Austing, B. H. Barnes, D. T. Bonnette, G. L. Engel, and G. Stokes, "Curriculum'78: Recommendations for the Undergraduate Program in Computer Science---A Report of the ACM Curriculum Committee on Computer Science," Commun. ACM, vol. 22, no. 3, pp. 147--166, 1979.Google Scholar
Digital Library
- A. Yadav, D. Burkhart, D. Moix, E. Snow, P. Bandaru, and L. Clayborn, "Sowing the Seeds: A Landscape Study on Assessment in Secondary Computer Science Education," Comp Sci Teach. Assn NY NY, 2015.Google Scholar
- M. C. Parker, M. Guzdial, and S. Engleman, "Replication, Validation, and Use of a Language Independent CS1 Knowledge Assessment," in Proceedings of the 2016 ACM conference on international computing education research, 2016, pp. 93--101.Google Scholar
- K. A. Ericsson, "The Influence of Experience and Deliberate Practice on the Development of Superior Expert Performance," Camb. Handb. Expert. Expert Perform., vol. 38, pp. 685--705, 2006.Google Scholar
- Y. Song, "Alignment of Deliberate Practice to Micro-credentials in an Introductory Computer Science Course," in 2019 IEEE Frontiers in Education Conference (FIE), Cincinnati, 2019.Google Scholar
- R. J. De Ayala, The Theory and Practice of Item Response Theory. Guilford Publications, 2013.Google Scholar
Index Terms
- Suitability of SCS1 as a Pre-CS2 Assessment Instrument: A Comparison with Short Deliberate-practice Questions
Recommendations
An IT program assessment process
SIGITE '07: Proceedings of the 8th ACM SIGITE conference on Information technology educationThis paper describes the assessment process designed and implemented for an undergraduate program in IT that was recently accredited by the Computing Accreditation Commission of ABET, Inc., under the general criteria for computing. It places special ...
The Assessment of Learning Outcomes in Information Assurance Curriculum
InfoSecCD '13: Proceedings of the 2013 on InfoSecCD '13: Information Security Curriculum Development ConferenceWe present an overview of the design and conduct of learning outcomes assessment for an undergraduate information assurance curriculum. The described program of assessment provides for an evidence-based, decision-making process that can be used to ...
Principled Assessment of Student Learning in High School Computer Science
ICER '17: Proceedings of the 2017 ACM Conference on International Computing Education ResearchAs K-12 computer science (CS) initiatives scale throughout the U.S., educators face increasing pressure from their school systems to provide evidence about student learning on hard-to-measure CS outcomes. At the same time, researchers studying ...
Comments