skip to main content
10.1145/3408877.3432481acmconferencesArticle/Chapter ViewAbstractPublication PagessigcseConference Proceedingsconference-collections
research-article
Public Access

A Specification Language for Matching Mistake Patterns with Feedback

Published: 05 March 2021 Publication History

Abstract

Pattern-based feedback detects incorrect code patterns in students' programs and provides feedback that can be personalized to the details of the matched code. Currently, a high level of instructor effort is required because the pattern detection must be expressed using complex programmatic interfaces. A specification language for pattern-based feedback is presented that mitigates this cost. Examples from actual student code illustrate the language's design and expressiveness. The language's implementation and testing is briefly described. Reflections are given on the the language design, where it is effectively used, and lessons learned from experience with its use. While our implementation is targeted at Python, other programming languages could be targeted using a similar approach.

References

[1]
Austin Cory Bart, Javier Tibau, Eli Tilevich, Clifford A Shaffer, and Dennis Kafura. 2017. BlockPy: An Open Access Data-Science Environment for Introductory Programmers. Computer, Vol. 50, 5 (2017), 18--26.
[2]
Brett A Becker, Paul Denny, Raymond Pettit, Durell Bouchard, Dennis J Bouvier, Brian Harrington, Amir Kamil, Amey Karkare, Chris McDonald, Peter-Michael Osera, et almbox. 2019. Compiler Error Messages Considered Unhelpful: The Landscape of Text-Based Programming Error Message Research. In Proceedings of the Working Group Reports on Innovation and Technology in Computer Science Education. ACM, 177--210.
[3]
Brett A Becker, Kyle Goslin, and Graham Glanville. 2018. The Effects of Enhanced Compiler Error Messages on a Syntax Error Debugging Test. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education. ACM, 640--645.
[4]
Ricardo Caceffo, Guilherme Gama, Raysa Benatti, Tales Aparecida, Tania Caldas, and Rodolfo Azevedo. 2018. A concept inventory for cs1 introductory programming courses in c. Technical Report. Tech. rep., University of Campinas, SP, Brasil.
[5]
Vicki Blum Cohen. 1985. A reexamination of feedback in computer-based instruction: Implications for instructional design. Educational Technology, Vol. 25, 1 (1985), 33--37.
[6]
Luke Gusukuma, Austin Cory Bart, and Dennis Kafura. 2020. Pedal: An Infrastructure and Model for Automated Feedback Systems. In Proceedings of the 51th ACM Technical Symposium on Computer Science Education. ACM, 338--343.
[7]
Luke Gusukuma, Austin Cory Bart, Dennis Kafura, and Jeremy Ernst. 2018. Misconception-Driven Feedback: Results from an Experimental Study. In Proceedings of the 2018 ACM Conference on International Computing Education Research. ACM.
[8]
Luke Satoru Gusukuma. 2020. Misconception Driven Student Analysis Model: Applications of a Cognitive Model in Teaching Computing. Ph.D. Dissertation. Virginia Tech.
[9]
Georgiana Haldeman, Andrew Tjang, Monica Babecs -Vroman, Stephen Bartos, Jay Shah, Danielle Yucht, and Thu D Nguyen. 2018. Providing Meaningful Feedback for Autograding of Programming Assignments. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education. ACM, 278--283.
[10]
Andrew Head, Elena Glassman, Gustavo Soares, Ryo Suzuki, Lucas Figueredo, Loris D'Antoni, and Björn Hartmann. 2017. Writing reusable code feedback at scale with mixed-initiative program synthesis. In Proceedings of the Fourth (2017) ACM Conference on Learning@ Scale. 89--98.
[11]
Tony Jenkins. 2002. On the difficulty of learning to program. In Proceedings of the 3rd Annual Conference of the LTSN Centre for Information and Computer Sciences, Vol. 4. Citeseer, 53--58.
[12]
Hieke Keuning, Johan Jeuring, and Bastiaan Heeren. 2018. A systematic literature review of automated feedback generation for programming exercises. ACM Transactions on Computing Education (TOCE), Vol. 19, 1 (2018), 1--43.
[13]
Kenneth R Koedinger, Vincent Aleven, Neil Heffernan, Bruce McLaren, and Matthew Hockenberry. 2004. Opening the door to non-programmers: Authoring intelligent tutor behavior by demonstration. In International Conference on Intelligent Tutoring Systems. Springer, 162--174.
[14]
David Liu and Andrew Petersen. 2019. Static analyses in python programming courses. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education. 666--671.
[15]
Robert Moser. 1997. A fantasy adventure game as a learning environment: why learning to program is so difficult and what can be done about it. In ACM SIGCSE Bulletin, Vol. 29. ACM, 114--116.
[16]
Tom Murray. 1999. Authoring intelligent tutoring systems: An analysis of the state of the art. International Journal of Artificial Intelligence in Education (IJAIED), Vol. 10 (1999), 98--129.
[17]
Terence Parr. 2013. The definitive ANTLR 4 reference. Pragmatic Bookshelf.
[18]
Kelly Rivers and Kenneth R Koedinger. 2017. Data-driven hint generation in vast solution spaces: a self-improving python programming tutor. International Journal of Artificial Intelligence in Education, Vol. 27, 1 (2017), 37--64.
[19]
Anuj Ramesh Shah. 2003. Web-cat: A web-based center for automated testing. Thesis. Virginia Tech.
[20]
Valerie J Shute. 2008. Focus on formative feedback. Review of educational research, Vol. 78, 1 (2008), 153--189.
[21]
Rishabh Singh, Sumit Gulwani, and Armando Solar-Lezama. 2013. Automated feedback generation for introductory programming assignments. In Proceedings of the 34th ACM SIGPLAN conference on Programming language design and implementation. 15--26.
[22]
Jakub Swacha. 2017. Exercise solution check specification language for interactive programming learning environments. In 6th Symposium on Languages, Applications and Technologies (SLATE 2017). Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik.
[23]
Marieke Thurlings, Marjan Vermeulen, Theo Bastiaens, and Sjef Stijnen. 2013. Understanding feedback: A learning theory perspective. Educational Research Review, Vol. 9 (2013), 1--15.
[24]
Leo C Ureel II and Charles Wallace. 2019. Automated Critique of Early Programming Antipatterns. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education. 738--744.

Cited By

View all
  • (2024)Automated feedback for participants of hands-on cybersecurity trainingEducation and Information Technologies10.1007/s10639-023-12265-829:9(11555-11584)Online publication date: 1-Jun-2024
  • (2022)Code Quality Defects across Introductory Programming TopicsProceedings of the 53rd ACM Technical Symposium on Computer Science Education - Volume 110.1145/3478431.3499415(941-947)Online publication date: 22-Feb-2022

Index Terms

  1. A Specification Language for Matching Mistake Patterns with Feedback

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      SIGCSE '21: Proceedings of the 52nd ACM Technical Symposium on Computer Science Education
      March 2021
      1454 pages
      ISBN:9781450380621
      DOI:10.1145/3408877
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 05 March 2021

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. cs education
      2. feedback
      3. feedback specification language

      Qualifiers

      • Research-article

      Funding Sources

      Conference

      SIGCSE '21
      Sponsor:

      Acceptance Rates

      Overall Acceptance Rate 1,595 of 4,542 submissions, 35%

      Upcoming Conference

      SIGCSE TS 2025
      The 56th ACM Technical Symposium on Computer Science Education
      February 26 - March 1, 2025
      Pittsburgh , PA , USA

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)77
      • Downloads (Last 6 weeks)6
      Reflects downloads up to 13 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Automated feedback for participants of hands-on cybersecurity trainingEducation and Information Technologies10.1007/s10639-023-12265-829:9(11555-11584)Online publication date: 1-Jun-2024
      • (2022)Code Quality Defects across Introductory Programming TopicsProceedings of the 53rd ACM Technical Symposium on Computer Science Education - Volume 110.1145/3478431.3499415(941-947)Online publication date: 22-Feb-2022

      View Options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Login options

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media