Paul Denny
Jacqueline Whalley
Juho Leinonen
ABSTRACT
Programming assignments are a common form of assessment in introductory courses and often require substantial work to complete. Students must therefore plan and manage their time carefully, especially leading up to published deadlines. Although time management is an important metacognitive skill that students must develop, it is rarely taught explicitly. Prior research has explored various approaches for reducing procrastination and other unproductive behaviours in students, but these are often ineffective or impractical in large courses. In this work, we investigate a scalable intervention that incentivizes students to begin work early. We provide automatically generated feedback to students who submit their work-in-progress prior to two fixed deadlines scheduled earlier than the final deadline for the assignment. Although voluntary, we find that many students welcome this early feedback and improve the quality of their work across each iteration. Especially for at-risk students, who have failed an earlier module in the course, engaging with the early feedback opportunities results in significantly better work at the time of final submission.
- Tapio Auvinen. 2015. Harmful Study Habits in Online Learning Environments with Automatic Assessment. In 2015 International Conference on Learning and Teaching in Computing and Engineering. 50–57.Google Scholar
Digital Library
- Brett A. Becker, Paul Denny, Raymond Pettit, Durell Bouchard, Dennis J. Bouvier, Brian Harrington, Amir Kamil, Amey Karkare, Chris McDonald, Peter-Michael Osera, Janice L. Pearce, and James Prather. 2019. Compiler Error Messages Considered Unhelpful: The Landscape of Text-Based Programming Error Message Research. In Proceedings of the Working Group Reports on Innovation and Technology in Computer Science Education(ITiCSE-WGR ’19). ACM, New York, NY, USA, 177–210. https://doi.org/10.1145/3344429.3372508Google ScholarDigital Library
- Brent J. Bowman and William A. Newman. 1990. Software Metrics as a Programming Training Tool. J. Syst. Softw. 13, 2 (Oct. 1990), 139–147. https://doi.org/10.1016/0164-1212(90)90119-7Google ScholarDigital Library
- Rachel Cardell-Oliver. 2011. How Can Software Metrics Help Novice Programmers?. In Proceedings of the Thirteenth Australasian Computing Education Conference - Volume 114(ACE ’11). Australian Computer Society, Inc., AUS, 55–62.Google Scholar
- Rachel Cardell-Oliver, Lu Zhang, Rieky Barady, You Hai Lim, Asad Naveed, and Terry Woodings. 2010. Automated Feedback for Quality Assurance in Software Engineering Education. In Proceedings of the 2010 21st Australian Software Engineering Conference(ASWEC ’10). IEEE Computer Society, USA, 157–164. https://doi.org/10.1109/ASWEC.2010.24Google ScholarDigital Library
- Rohan Roy Choudhury, HeZheng Yin, Joseph Moghadam, and Armando Fox. 2016. AutoStyle: Toward Coding Style Feedback At Scale. In Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work and Social Computing Companion(CSCW ’16 Companion). Association for Computing Machinery, New York, NY, USA, 21–24. https://doi.org/10.1145/2818052.2874315Google ScholarDigital Library
- Sophie H. Cormack, Laurence A. Eagle, and Mark S. Davies. 2020. A large-scale test of the relationship between procrastination and performance using learning analytics. Assessment & Evaluation in Higher Education 0, 0 (2020), 1–14. https://doi.org/10.1080/02602938.2019.1705244 arXiv:https://doi.org/10.1080/02602938.2019.1705244Google ScholarCross Ref
- Paul Denny, Brett A. Becker, Michelle Craig, Greg Wilson, and Piotr Banaszkiewicz. 2019. Research This! Questions That Computing Educators Most Want Computing Education Researchers to Answer. In Proceedings of the 2019 ACM Conference on International Computing Education Research(ICER ’19). Association for Computing Machinery, New York, NY, USA, 259–267. https://doi.org/10.1145/3291279.3339402Google ScholarDigital Library
- Paul Denny, Andrew Luxton-Reilly, Michelle Craig, and Andrew Petersen. 2018. Improving Complex Task Performance Using a Sequence of Simple Practice Tasks. In Proceedings of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education(ITiCSE 2018). Association for Computing Machinery, New York, NY, USA, 4–9. https://doi.org/10.1145/3197091.3197141Google ScholarDigital Library
- Paul Denny, Andrew Luxton-Reilly, Ewan Tempero, and Jacob Hendrickx. 2011. CodeWrite: Supporting Student-Driven Practice of Java. In Proceedings of the 42nd ACM Technical Symposium on Computer Science Education(SIGCSE ’11). Association for Computing Machinery, New York, NY, USA, 471–476. https://doi.org/10.1145/1953163.1953299Google ScholarDigital Library
- Paul Denny, James Prather, and Brett A. Becker. 2020. Error Message Readability and Novice Debugging Performance. In Proceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science Education(ITiCSE ’20). Association for Computing Machinery, New York, NY, USA, 480–486. https://doi.org/10.1145/3341525.3387384Google ScholarDigital Library
- Stephen H. Edwards, Joshua Martin, and Clfford A. Shaffer. 2015. Examining Classroom Interventions to Reduce Procrastination. In Proceedings of the 2015 ACM Conference on Innovation and Technology in Computer Science Education(ITiCSE ’15). Association for Computing Machinery, New York, NY, USA, 254–259. https://doi.org/10.1145/2729094.2742632Google ScholarDigital Library
- Stephen H. Edwards and Krishnan Panamalai Murali. 2017. CodeWorkout: Short Programming Exercises with Built-in Data Collection. In Proceedings of the 2017 ACM Conference on Innovation and Technology in Computer Science Education(ITiCSE ’17). Association for Computing Machinery, New York, NY, USA, 188–193. https://doi.org/10.1145/3059009.3059055Google ScholarDigital Library
- Stephen H. Edwards and Manuel A. Perez-Quinones. 2008. Web-CAT: Automatically Grading Programming Assignments. In Proceedings of the 13th Annual Conference on Innovation and Technology in Computer Science Education(ITiCSE ’08). Association for Computing Machinery, New York, NY, USA, 328. https://doi.org/10.1145/1384271.1384371Google ScholarDigital Library
- Stephen H. Edwards, Jason Snyder, Manuel A. Pérez-Quiñones, Anthony Allevato, Dongkwan Kim, and Betsy Tretola. 2009. Comparing Effective and Ineffective Behaviors of Student Programmers. In Proceedings of the Fifth International Workshop on Computing Education Research Workshop(ICER ’09). Association for Computing Machinery, New York, NY, USA, 3–14. https://doi.org/10.1145/1584322.1584325Google ScholarDigital Library
- Tommy Färnqvist and Fredrik Heintz. 2016. Competition and Feedback through Automated Assessment in a Data Structures and Algorithms Course. In Proceedings of the 2016 ACM Conference on Innovation and Technology in Computer Science Education(ITiCSE ’16). Association for Computing Machinery, New York, NY, USA, 130–135. https://doi.org/10.1145/2899415.2899454Google ScholarDigital Library
- James B. Fenwick, Cindy Norris, Frank E. Barry, Josh Rountree, Cole J. Spicer, and Scott D. Cheek. 2009. Another Look at the Behaviors of Novice Programmers. In Proceedings of the 40th ACM Technical Symposium on Computer Science Education(SIGCSE ’09). Association for Computing Machinery, New York, NY, USA, 296–300. https://doi.org/10.1145/1508865.1508973Google ScholarDigital Library
- Yoshiko Goda, Masanori Yamada, Hiroshi Kato, Takeshi Matsuda, Yutaka Saito, and Hiroyuki Miyagawa. 2015. Procrastination and other learning behavioral types in e-learning and their relationship with learning outcomes. Learning and Individual Differences 37 (2015), 72 – 80. https://doi.org/10.1016/j.lindif.2014.11.001Google ScholarCross Ref
- Keith Gregory and Sue Morón-García. 2009. Assignment submission, student behaviour and experience. Engineering Education 4, 1 (2009), 16–28. https://doi.org/10.11120/ened.2009.04010016Google ScholarCross Ref
- Petri Ihantola, Tuukka Ahoniemi, Ville Karavirta, and Otto Seppälä. 2010. Review of Recent Systems for Automatic Assessment of Programming Assignments. In Proceedings of the 10th Koli Calling International Conference on Computing Education Research(Koli Calling ’10). Association for Computing Machinery, New York, NY, USA, 86–93. https://doi.org/10.1145/1930464.1930480Google ScholarDigital Library
- Petri Ihantola, Arto Vihavainen, Alireza Ahadi, Matthew Butler, Jürgen Börstler, Stephen H. Edwards, Essi Isohanni, Ari Korhonen, Andrew Petersen, Kelly Rivers, Miguel Ángel Rubio, Judy Sheard, Bronius Skupas, Jaime Spacco, Claudia Szabo, and Daniel Toll. 2015. Educational Data Mining and Learning Analytics in Programming: Literature Review and Case Studies. In Proceedings of the 2015 ITiCSE on Working Group Reports(ITICSE-WGR ’15). Association for Computing Machinery, New York, NY, USA, 41–63. https://doi.org/10.1145/2858796.2858798Google ScholarDigital Library
- Kalle Ilves, Juho Leinonen, and Arto Hellas. 2018. Supporting self-regulated learning with visualizations in online learning environments. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education. 257–262.Google ScholarDigital Library
- Michael S. Irwin and Stephen H. Edwards. 2019. Can Mobile Gaming Psychology Be Used to Improve Time Management on Programming Assignments?. In Proceedings of the ACM Conference on Global Computing Education(CompEd ’19). Association for Computing Machinery, New York, NY, USA, 208–214. https://doi.org/10.1145/3300115.3309517Google ScholarDigital Library
- Lucy Jiang, Robert Rewcastle, Paul Denny, and Ewan Tempero. 2020. CompareCFG: Providing Visual Feedback on Code Quality Using Control Flow Graphs. In Proceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science Education(ITiCSE ’20). Association for Computing Machinery, New York, NY, USA, 493–499. https://doi.org/10.1145/3341525.3387362Google ScholarDigital Library
- Ayaan M. Kazerouni, Stephen H. Edwards, and Clifford A. Shaffer. 2017. Quantifying Incremental Development Practices and Their Relationship to Procrastination. In Proceedings of the 2017 ACM Conference on International Computing Education Research(ICER ’17). Association for Computing Machinery, New York, NY, USA, 191–199. https://doi.org/10.1145/3105726.3106180Google ScholarDigital Library
- Hieke Keuning, Johan Jeuring, and Bastiaan Heeren. 2018. A Systematic Literature Review of Automated Feedback Generation for Programming Exercises. ACM Trans. Comput. Educ. 19, 1, Article Article 3 (Sept. 2018), 43 pages. https://doi.org/10.1145/3231711Google ScholarDigital Library
- Pardha Koyya, Lee Young, and Jeong Yang. 2013. Feedback for Programming Assignments Using Software-Metrics and Reference Code. ISRN Software Engineering(2013). https://doi.org/10.1155/2013/805963Google ScholarCross Ref
- Lauri Malmi, Ville Karavirta, Ari Korhonen, and Jussi Nikander. 2005. Experiences on Automatically Assessed Algorithm Simulation Exercises with Different Resubmission Policies. J. Educ. Resour. Comput. 5, 3 (Sept. 2005), 7–es. https://doi.org/10.1145/1163405.1163412Google ScholarDigital Library
- Keir Mierle, Kevin Laven, Sam Roweis, and Greg Wilson. 2005. Mining Student CVS Repositories for Performance Indicators. In Proceedings of the 2005 International Workshop on Mining Software Repositories(MSR ’05). Association for Computing Machinery, New York, NY, USA, 1–5. https://doi.org/10.1145/1083142.1083150Google ScholarDigital Library
- Nick Parlante. 2007. Nifty Reflections. SIGCSE Bull. 39, 2 (June 2007), 25–26. https://doi.org/10.1145/1272848.1272876Google ScholarDigital Library
- James Prather, Brett A. Becker, Michelle Craig, Paul Denny, Dastyni Loksa, and Lauren Margulieux. 2020. What Do We Think We Think We Are Doing?: Metacognition and Self-Regulation in Programming. In Proceedings of the 2020 ACM Conference on International Computing Education Research(ICER ’20). Association for Computing Machinery, New York, NY, USA, 12. https://doi.org/10.1145/3372782.3406263Google ScholarDigital Library
- Paul Ramsden. 1992. Learning to Teach in Higher Education. Routledge, London.Google Scholar
- Jaime Spacco, Paul Denny, Brad Richards, David Babcock, David Hovemeyer, James Moscola, and Robert Duvall. 2015. Analyzing Student Work Patterns Using Programming Exercise Data. In Proceedings of the 46th ACM Technical Symposium on Computer Science Education(SIGCSE ’15). Association for Computing Machinery, New York, NY, USA, 18–23. https://doi.org/10.1145/2676723.2677297Google ScholarDigital Library
- Jaime Spacco and William Pugh. 2006. Helping Students Appreciate Test-Driven Development (TDD). In Companion to the 21st ACM SIGPLAN Symposium on Object-Oriented Programming Systems, Languages, and Applications(OOPSLA ’06). Association for Computing Machinery, New York, NY, USA, 907–913. https://doi.org/10.1145/1176617.1176743Google ScholarDigital Library
- Claudia Szabo, Nickolas Falkner, Antti Knutas, and Mohsen Dorodchi. 2018. Understanding the Effects of Lecturer Intervention on Computer Science Student Behaviour. In Proceedings of the 2017 ITiCSE Conference on Working Group Reports(ITiCSE-WGR ’17). Association for Computing Machinery, New York, NY, USA, 105–124. https://doi.org/10.1145/3174781.3174787Google ScholarDigital Library
- Rebecca Vivian, Katrina Falkner, and Nickolas Falkner. 2013. Computer Science Students’ Causal Attributions for Successful and Unsuccessful Outcomes in Programming Assignments. In Proceedings of the 13th Koli Calling International Conference on Computing Education Research(Koli Calling ’13). Association for Computing Machinery, New York, NY, USA, 125–134. https://doi.org/10.1145/2526968.2526982Google ScholarDigital Library
- Jacqueline L. Whalley and Anne Philpott. 2011. A Unit Testing Approach to Building Novice Programmers’ Skills and Confidence. In Proceedings of the Thirteenth Australasian Computing Education Conference - Volume 114(ACE ’11). Australian Computer Society, Inc., AUS, 113–118. https://dl.acm.org/doi/10.5555/2459936.2459950Google Scholar
- Wikipedia. 2020. Sokoban – Wikipedia. https://en.wikipedia.org/wiki/SokobanGoogle Scholar
- Salla Willman, Rolf Lindén, Erkki Kaila, Teemu Rajala, Mikko-Jussi Laakso, and Tapio Salakoski. 2015. On study habits on an introductory course on programming. Computer Science Education 25, 3 (2015), 276–291. https://doi.org/10.1080/08993408.2015.1073829Google ScholarCross Ref
Index Terms
- Promoting Early Engagement with Programming Assignments Using Scheduled Automated Feedback
Index terms have been assigned to the content through auto-classification.
Recommendations
A Comparison of Immediate and Scheduled Feedback in Introductory Programming Projects
SIGCSE 2022: Proceedings of the 53rd ACM Technical Symposium on Computer Science Education - Volume 1How students are assessed has a powerful effect on their strategies for studying and their learning. When designing assessments, instructors should consider how different approaches for providing feedback to students could encourage positive learning ...
Does the Early Bird Catch the Worm? Earliness of Students' Work and its Relationship with Course Outcomes
ITiCSE '21: Proceedings of the 26th ACM Conference on Innovation and Technology in Computer Science Education V. 1Intuitively, it seems plausible that students who start their work earlier and work on more days than their peers should perform better in any course. But does the early bird really catch the worm? In this article, we examine introductory programming ...
The Effects of Procrastination Interventions on Programming Project Success
ICER '15: Proceedings of the eleventh annual International Conference on International Computing Education ResearchIn computer science, procrastination and related problems with managing programming projects are viewed as primary causes of student attrition. Unfortunately, the most successful techniques for reducing procrastination (such as courses in study skills) ...
Comments