Mind the Error Message: An Inverted Quiz Format to Direct Learner's Attention to Error Messages
Pages 382 - 388
Abstract
Novice learners of programming tend to neglect error messages, even though the messages have a lot of useful information for solving problems. While there exists research that aims to user-friendly error messages by changing the wording and by adding visual assistance, most of them do not focus on drawing learners' attention to error messages. We propose the enbugging quiz, a novel quiz format that requests the learner to craft a program that produces a specified error. This paper reports our design of enbugging quizzes and reports the results of our initial experiment, where we observed positive effects on the learners' attitudes towards error messages.
References
[1]
Umair Z Ahmed, Renuka Sindhgatta, Nisheeth Srivastava, and Amey Karkare. 2019. Targeted example generation for compilation errors. In 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 327--338.
[2]
Titus Barik. 2014. Improving error notification comprehension through visual overlays in IDEs. In 2014 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). IEEE, 177--178.
[3]
Titus Barik, Jim Witschey, Brittany Johnson, and Emerson Murphy-Hill. 2014. Compiler error notifications revisited: an interaction-first approach for helping developers more effectively comprehend and resolve error notifications. In Companion Proceedings of the 36th International Conference on Software Engineering. 536--539.
[4]
Brett A Becker. 2016. An effective approach to enhancing compiler error messages. In Proceedings of the 47th ACM Technical Symposium on Computing Science Education. 126--131.
[5]
Elizabeth Emily Carter. 2014. An intelligent debugging tutor for novice computer science students. (2014).
[6]
Der-Thanq Chen. 1997. Learning 10 computer programs in a month. (1997).
[7]
Ryan Chmiel and Michael C Loui. 2004. Debugging: from novice to expert. Acm Sigcse Bulletin, Vol. 36, 1 (2004), 17--21.
[8]
Paul Denny, Andrew Luxton-Reilly, and Dave Carpenter. 2014. Enhancing syntax error messages appears ineffectual. In Proceedings of the 2014 conference on Innovation & technology in computer science education. 273--278.
[9]
Paul Denny, James Prather, Brett A Becker, Catherine Mooney, John Homer, Zachary C Albrecht, and Garrett B Powell. 2021. On Designing Programming Error Messages for Novices: Readability and its Constituent Factors. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1--15.
[10]
Nobuo Funabiki, Yukiko Matsushima, Toru Nakanishi, Kan Watanabe, and Noriki Amano. 2013. A Java programming learning assistant system using test-driven development method. IAENG International Journal of Computer Science, Vol. 40, 1 (2013), 38--46.
[11]
Marleen Gilsing and Felienne Hermans. 2021. Gradual Programming in Hedy: A First User Study. In 2021 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). IEEE, 1--9.
[12]
Yoshinari Hachisu, Atsushi Yoshida, and Kiyoshi Agusa. 2017. A generator for Exercises of Program Error Correction and Answer Checker Programs (in japanese). IPSJ Transactions on Computers and Education (TCE), Vol. 3, 1 (2017), 64--78.
[13]
Björn Hartmann, Daniel MacDougall, Joel Brandt, and Scott R Klemmer. 2010. What would other programmers do: suggesting solutions to error messages. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 1019--1028.
[14]
Ryota Kamei, Masahiro Yosizuka, Isao Sasano, and Seiichi Komiya. 2021. Comprehension Check Problem Generation Method Using Perturbation to Compensate for Shortcomings of Tracing Learning--Evaluation of Effectiveness Based on Examples of Binary Operators(Syakyou gata gakusyu no kettenn wo oginau setsudou wo motiita rikaido kakunin mondai seisei syuhou--nikou ennzannsi no rei ni motoduku yuukousei no hyouka (in Japanese).). Computer software, Vol. 38, 1 (2021), 1_111--1_139.
[15]
Cagin Kazimoglu, Mary Kiernan, Liz Bacon, and Lachlan Mackinnon. 2012. A serious game for developing computational thinking and learning introductory computer programming. Procedia-Social and Behavioral Sciences, Vol. 47 (2012), 1991--1999.
[16]
Olli Kiljunen. 2021. Teaching Students to Fix Programming Errors with Tutorials Embedded in an IDE. In Proceedings of the 21st Koli Calling International Conference on Computing Education Research (Joensuu, Finland) (Koli Calling '21). Association for Computing Machinery, New York, NY, USA, Article 32, 3 pages. https://doi.org/10.1145/3488042.3489969
[17]
Essi Lahtinen, Kirsti Ala-Mutka, and Hannu-Matti J"arvinen. 2005. A study of the difficulties of novice programmers. Acm sigcse bulletin, Vol. 37, 3 (2005), 14--18.
[18]
Greg C Lee and Jackie C Wu. 1999. Debug It: A debugging practicing system. Computers & Education, Vol. 32, 2 (1999), 165--179.
[19]
Michael J Lee and Amy J Ko. 2011. Personifying programming tool feedback improves novice programmers' learning. In Proceedings of the seventh international workshop on Computing education research. 109--116.
[20]
Zhongxiu Liu, Rui Zhi, Andrew Hicks, and Tiffany Barnes. 2017. Understanding problem solving behavior of 6--8 graders in a debugging game. Computer Science Education, Vol. 27, 1 (2017), 1--29.
[21]
Andrew Luxton-Reilly, Emma McMillan, Elizabeth Stevenson, Ewan Tempero, and Paul Denny. 2018. Ladebug: An online tool to help novice programmers improve their debugging skills. In Proceedings of the 23rd annual acm conference on innovation and technology in computer science education. 159--164.
[22]
Guillaume Marceau, Kathi Fisler, and Shriram Krishnamurthi. 2011. Measuring the effectiveness of error messages designed for novice programmers. In Proceedings of the 42nd ACM technical symposium on Computer science education. 499--504.
[23]
Davin McCall and Michael Kölling. 2019. A new look at novice programmer errors. ACM Transactions on Computing Education (TOCE), Vol. 19, 4 (2019), 1--30.
[24]
Michael A Miljanovic and Jeremy S Bradbury. 2017. Robobug: a serious game for learning debugging techniques. In Proceedings of the 2017 acm conference on international computing education research. 93--100.
[25]
Marie-Hélène Nienaltowski, Michela Pedroni, and Bertrand Meyer. 2008. Compiler error messages: What can help novices?. In Proceedings of the 39th SIGCSE technical symposium on Computer science education. 168--172.
[26]
Raymond S Pettit, John Homer, and Roger Gee. 2017. Do Enhanced Compiler Error Messages Help Students? Results Inconclusive. In Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education. 465--470.
[27]
James Prather, Raymond Pettit, Kayla Holcomb McMurry, Alani Peters, John Homer, Nevan Simone, and Maxine Cohen. 2017. On novices' interaction with compiler error messages: A human factors approach. In Proceedings of the 2017 ACM Conference on International Computing Education Research. 74--82.
[28]
Katsuyuki Umezawa, Makoto Nakazawa, Masayuki Goto, and Shigeichi Hirasawa. 2019. Development of debugging exercise extraction system using learning history. In 2019 IEEE Tenth International Conference on Technology for Education (T4E). IEEE, 244--245.
[29]
Iris Vessey. 1985. Expertise in debugging computer programs: A process analysis. International Journal of Man-Machine Studies, Vol. 23, 5 (1985), 459--494.
[30]
John Wrenn and Shriram Krishnamurthi. 2017. Error messages are classifiers: a process to design and evaluate error messages. In Proceedings of the 2017 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software. 134--147.
[31]
Zihe Zhou, Shijuan Wang, and Yizhou Qian. 2021. Learning From Errors: Exploring the Effectiveness of Enhanced Error Messages in Learning to Program. Frontiers in Psychology, Vol. 12 (2021). https://doi.org/10.3389/fpsyg.2021.768962
Index Terms
- Mind the Error Message: An Inverted Quiz Format to Direct Learner's Attention to Error Messages
Recommendations
Compiler Error Messages Considered Unhelpful: The Landscape of Text-Based Programming Error Message Research
ITiCSE-WGR '19: Proceedings of the Working Group Reports on Innovation and Technology in Computer Science EducationDiagnostic messages generated by compilers and interpreters such as syntax error messages have been researched for over half of a century. Unfortunately, these messages which include error, warning, and run-time messages, present substantial difficulty ...
Teaching Programming Error Message Understanding
CompEd 2023: Working Group Reports on 2023 ACM Conference on Global Computing EducationAbout a decade ago there was a sharp increase in the number of research publications focusing on programming error messages. The majority of this work focused on improving or "enhancing'' compiler error messages to make them easier for humans, especially ...
Teaching Students To Use Programming Error Messages
CompEd 2023: Proceedings of the ACM Conference on Global Computing Education Vol 2Research shows many students struggle to use programming error and warning messages effectively. Instead of using these messages as aids to debug and fix their code, some students have negative emotional reactions to seeing 'angry red text'. Not ...
Comments
Information & Contributors
Information
Published In
June 2023
694 pages
ISBN:9798400701382
DOI:10.1145/3587102
- General Chairs:
- Mikko-Jussi Laakso,
- Mattia Monga,
- Program Chairs:
- Simon,
- Judithe Sheard
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 30 June 2023
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- JSPS KAKENHI
Conference
ITiCSE 2023
Sponsor:
Acceptance Rates
Overall Acceptance Rate 552 of 1,613 submissions, 34%
Upcoming Conference
ITiCSE '25
- Sponsor:
- sigcse
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 153Total Downloads
- Downloads (Last 12 months)57
- Downloads (Last 6 weeks)8
Reflects downloads up to 07 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in