Paul Denny
Andrew Luxton-Reilly
Beth Simon
ABSTRACT
Common exam practice centres around two question types: code tracing (reading) and code writing. It is commonly believed that code tracing is easier than code writing, but it seems obvious that different skills are needed for each. These problems also differ in their value on an exam. Pedagogically, code tracing on paper is an authentic task whereas code writing on paper is less so. Yet, few instructors are willing to forgo the code writing question on an exam. Is there another way, easier to grade, that captures the "problem solving through code creation process" we wish to examine? In this paper we propose Parson's puzzle-style problems for this purpose. We explore their potential both qualitatively, through interviews, and quantitatively through a set of CS1 exams. We find notable correlation between Parsons scores and code writing scores. We find low correlation between code writing and tracing and between Parsons and tracing. We also make the case that marks from a Parsons problem make clear what students don't know (specifically, in both syntax and logic) much less ambiguously than marks from a code writing problem. We make recommendations on the design of Parsons problems for the exam setting, discuss their potential uses and urge further investigations of Parsons problems for assessment of CS1 students.
- Clear, T., Edwards, J., Lister, R., Simon, B., Thompson, E. and Whalley, J. The teaching of novice computer programmers: bringing the scholarly-research approach to Australia. In Proc. Of ACE 2008, Wollongong, NSW, Australia. CRPIT, 78. Simon and Hamilton, M., Eds., ACS. 63--68, 2008. Google Scholar
Digital Library
- Garner, S., The Learning of Plans in Programming: A Program Completion Approach. In Proc. of ICCE'02. IEEE Computer Society, Washington, DC, 2002. Google ScholarDigital Library
- Hieggelke, C., Maloney, D., Kanim, S., O'Kuma, T. E&M TIPERs: Electricity and Magnetism Tasks Inspired by Physics Education Research. Pearson, Prentice Hall, 2006.Google Scholar
- Lister, R., Adams, E. S., Fitzgerald, S., Fone, W., Hamer, J., Lindholm, M., McCartney, R., Mostrom, J. E., Sanders, K., Seppala, O., Simon, B., & Thomas, L. A Multi-National Study of Reading and Tracing Skills in Novice Programmers. SIGCSE Bulletin, 36(4): 119--150, 2004. Google ScholarDigital Library
- Lister, R., Simon, B., Thompson, E., Whalley, J. L., & Prasad, C. Not seeing the forest for the trees: novice programmers and the SOLO taxonomy, In Proc. of ITICSE 2006, Bologna, Italy. 118--122, 2006. Google ScholarDigital Library
- McCracken, M., Almstrum, V., Diaz, D., Guzdial, M., Hagen, D., Kolikant, Y., Laxer, C., Thomas, L., Utting, I., & Wilusz, T. A Multi-Institutional, Multi-National Study of Assessment of Programming Skills of First-year CS Students. SIGCSE Bulletin, 33(4).125--140, 2001. Google ScholarDigital Library
- O'Kuma, T., Maloney, D., Hieggelke, C. Ranking Task Exercises in Physics. Prentice Hall, 2000.Google Scholar
- Parsons, D. and Haden, P. Parsons' programming puzzles: a fun and effective learning tool for first programming courses. In Proc. of ACE 2006, Hobart, Australia, 157--163, January 16 - 19, 2006. Google ScholarDigital Library
- Selden, J. and Selden, A. Unpacking the logic of mathematical statements. Educational Studies in Mathematics, 29(2), 123--151, 1995.Google ScholarCross Ref
- Selden, A. & Selden J. Validations of proofs considered as texts: Can undergraduates tell whether an argument proves a theorem? Journal for Research in Mathematics Education 34, 1, (pp.4--36), 2003.Google ScholarCross Ref
- Thompson, E., Luxton-Reilly, A., Whalley, J., Hu, M. and Robbins, P. Bloom's Taxonomy for CS assessment. In Proc. of ACE 2008, Wollongong, NSW, Australia. 155--162, 2008. Google ScholarDigital Library
- Whalley, J., Clear, T. & Lister, R. The Many Ways of the BRACElet Project. Bull. of Applied Computing and Information Technology Vol. 5, Issue 1. ISSN 1176-4120, June 2007. Retrieved April 24, 2008 from http://www.naccq.co.nz/bacit/0501/2007Whalley_BRACELET_Ways.htmGoogle Scholar
Index Terms
- Evaluating a new exam question: Parsons problems
Recommendations
Solving parsons problems versus fixing and writing code
Koli Calling '17: Proceedings of the 17th Koli Calling International Conference on Computing Education ResearchPrior research has shown that Parsons problems are an engaging type of code completion problem that can be used to teach syntactic and semantic language constructs. They can also be used in summative assessments to reduce marking time and grading ...
The Code Mangler: Evaluating Coding Ability Without Writing any Code
SIGCSE '17: Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science EducationMarking coding exam questions for introductory computer science courses is notoriously resource intensive and difficult to perform consistently. Students can be easily led astray by minor misunderstandings in the wording of questions, and graders often ...
Tracing vs. Writing Code: Beyond the Learning Hierarchy
SIGCSE '18: Proceedings of the 49th ACM Technical Symposium on Computer Science EducationMuch work has been done on the achievement gap between code tracing and code writing in CS1 students. The generally accepted explanation for this gap is that tracing and writing form separate steps in a learning scaffolding; students must first learn to ...
Comments