Matthew Peveler
ABSTRACT
Submitty (http://submitty.org) is an open source programming assignment submission system from the Rensselaer Center for Open Source Software (RCOS) at Rensselaer Polytechnic Institute (RPI). Students can submit their code via a web interface in a variety of ways, where it is then tested with a highly configurable and customizable automated grader. Students receive immediate feedback from the grader, and can resubmit to correct errors as needed. Through an online interface, TAs can access detailed grading results and supplement the automated scores with manual grading (numeric and written feedback) of overall program structure, good use of comments, reasonable error checking, etc. and any non-programming components of the assignment. The instructor can also configure the system to allow for a configurable late day policy on a per assignment and per student basis. We currently use Submitty in eight different courses (spanning from introductory through advanced topics) serving over 1500 students and 35+ instructors and TAs each week. We will present a range of "case study" assignment configurations in a hands-on demo, going from simple through complex, using a variety of different automated grading methods including per-character and per-line output difference checkers, external unit testing frameworks (such as JUnit), memory debugging tools (Valgrind and DrMemory), code coverage (e.g., Emma), static analysis tools, and custom graders. Submitty can be customized per test case as appropriate to apply resource limits (running time, number of processes, output file size, etc.) and to display or hide from students the program output, autograding results, and testing logs.
- Petri Ihantola, Tuukka Ahoniemi, Ville Karavirta, and Otto Seppälä. 2010. Review of recent systems for automatic assessment of programming assignments. In Proceedings of the 10th Koli Calling International Conference on Computing Education Research (Koli Calling '10). Submitty, Rensselear Center for Open Source Software, https://github.com/Submitty/ , http://submitty.org . Google Scholar
Digital Library
Index Terms
- Submitty: An Open Source, Highly-Configurable Platform for Grading of Programming Assignments (Abstract Only)
Recommendations
A survey of Australian teachers' self-efficacy and assessment approaches for the K-12 digital technologies curriculum
WiPSCE '18: Proceedings of the 13th Workshop in Primary and Secondary Computing EducationAs K-12 computer science (CS) education has been introduced to a number of countries around the world, the CS education community has been busy working to understand the learning and teaching of computing at these year levels, as well as how to build ...
SCAPA: Development of a Questionnaire Assessing Self-Concept and Attitudes Toward Programming
ITiCSE '20: Proceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science EducationThere is a constantly growing number of initiatives asserting the relevance of programming already in primary education and offering respective interventions with the goal to foster interest in and positive attitudes toward programming. To evaluate to ...
Grading Programming Assignments with an Automated Grading and Feedback Assistant
Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners’ and Doctoral ConsortiumAbstractOver the last few years, Computer Science class sizes have increased, resulting in a higher grading workload. Universities often use multiple graders to quickly deliver the grades and associated feedback to manage this workload. While using ...
Comments