Autograding Python Code with the Pedal Framework: Feedback Beyond Unit Tests

The ever-increasing enrollments in programming courses has driven the need for sophisticated grading tools that can provide students with precise, insightful, and timely feedback. This SIGCSE workshop presents an interactive session on our powerful, open-source Python autograding framework, Pedal. As a free library, Pedal is available on a wide range of grading platforms, including GradeScope and BlockPy - anything that allows installation of a pure Python library. Pedal supports but goes beyond traditional unit testing, providing advanced code analysis techniques, such as type checking, liveness checking, structural code pattern matching, and more. Pedal has a large collection of assertions to evaluate dynamic program traces, query the Abstract Syntax Tree, re-execute student code under varying conditions, mock inputs, and capture outputs. Every feedback message is treated as a first-class object, empowering educators to fine-tune feedback as desired. Pedal is not just a toolset but a comprehensive pipeline optimized for feedback selection, resolution, and evaluation. With functionalities like command-line batch execution, exhaustive metadata tracking, and A/B testing, educators and researchers can analyze and refine their feedback strategies. With Pedal's successful deployment across multiple courses and institutions over the years, this workshop will offer attendees firsthand experience and a plethora of real-world examples. By the end of this workshop, participants will be proficient in leveraging Pedal, even venturing into the realm of creating interactive activities using the framework.