skip to main content
10.1145/2048237.2048241acmconferencesArticle/Chapter ViewAbstractPublication PagessplashConference Proceedingsconference-collections
research-article

Mind your language: on novices' interactions with error messages

Published: 22 October 2011 Publication History

Abstract

Error messages are one of the most important tools that a language offers its programmers. For novices, this feed-back is especially critical. Error messages typically contain both a textual description of the problem and an indication of where in the code the error occurred. This paper reports on a series of studies that explore beginning students' inter-actions with the vocabulary and source-expression high-lighting in DrRacket. Our findings demonstrate that the error message significantly fail to convey information accurately to students, while also suggesting alternative designs that might address these problems.

References

[1]
G. Marceau, K. Fisler, and S. Krishnamurthi, "Measuring the Effectiveness of Error Messages Designed for Novice Programmers," Proceedings of the Symposium on Computer Science Education, 2011.
[2]
P.I. Pavlik, N. Presson, G. Dozzi, and B. Macwhinney, "The FaCT (fact and concept) system: A new tool linking cognitive science with educators," Proceedings of the Conference of the Cognitive Science Society, D. McNamara & G. Trafton, eds., 2007, pp. 397--402.
[3]
N. Wirth, "The Programming Language Pascal," Acta Informatica, vol. 1, 1971, pp. 35--63.
[4]
S. Bloch, Picturing Programs, College Publications, 2011.
[5]
V.J. Traver, "On compiler error messages: what they say and what they mean," Technical Report, Computer Languages and Systems Department, Jaume-I University, 2010.
[6]
B. Moskal, D. Lurie, and S. Cooper, "Evaluating the effectiveness of a new instructional approach," Proceedings of the Symposium on Computer Science Education, 2004, pp. 75--79.
[7]
D. Hagan and S. Markham, "Teaching Java with the BlueJ environment," Proceedings of Australasian Society for Computers in Learning in Tertiary Education Conference, 2000.
[8]
J.S. Rey, From Alice to BlueJ: a transition to Java, Master's thesis, School of Computing, Robert Gordon University, 2009.
[9]
M.M. Ben-Ari, "Compile and Runtime Errors in Java," http://stwww.weizmann.ac.il/g-cs/benari/oop/errors.pdf, accessed June 15, 2010.
[10]
K. Van Haaster and D. Hagan, "Teaching and Learning with BlueJ: an Evaluation of a Pedagogical Tool," Issues in Informing Science and Information Technology, vol. 1, 2004, pp. 455--470.
[11]
M. Crestani, "Experience report: growing programming languages for beginning students," Proceedings of the International Conference on Functional Programming, 2010.
[12]
A. Pears, S. Seidman, L. Malmi, L. Mannila, E. Adams, J. Bennedsen, M. Devlin, and J. Paterson, "A survey of literature on the teaching of introductory programming," ACM SIGCSE Bulletin, vol. 39, Dec. 2007, pp. 204--223.
[13]
P. Gross and K. Powers, "Evaluating assessments of novice programming environments," Proceedings of the International Workshop on Computing Education Research, New York, New York, USA: 2005, pp. 99--110.
[14]
B.S. Fagin and L. Merkle, "Quantitative analysis of the effects of robots on introductory Computer Science education," Journal on Educational Resources in Computing, vol. 2, 2002, pp. 1--18.
[15]
R.B.-B. Levy, M. Ben-Ari, and P.A. Uronen, "The Jeliot 2000 program animation system," Computers & Education, vol. 40, 2003, pp. 1--15.
[16]
T. Flowers, C. Carver, and J. Jackson, "Empowering students and building confidence in novice programmers through Gauntlet," Frontiers in Education, vol. 1, 2004, p. T3H/10 - T3H/13.
[17]
J. Jackson, M. Cobb, and C. Carver, "Identifying Top Java Errors for Novice Programmers," Proceedings of the Frontiers in Education Conference, 2005, p. T4C--24.
[18]
C. Burrell and M. Melchert, "Augmenting compiler error reporting in the Karel++ microworld," Proceedings of the Conference of the National Advisory Committee on Computing Qualifications, 2007, p. 41--46.
[19]
B.S. Lerner, M. Flower, D. Grossman, and C. Chambers, "Searching for type-error messages," Programming language design and implementation, 2007, p. 425.
[20]
L. Zolman, "STLFilt: An STL error message decryptor for C++," http://www.bdsoft.com/tools/stlfilt.html, accessed June 10, 2010, 2005.
[21]
J.M. Chabert and T.F. Higginbotham, "An Investigation of Novice Programmer Errors in IBM 370 (OS) Assembly Language," Proceedings of the ACM Southeast Regional Conference, 1976, pp. 319--323.
[22]
J.C. Spohrer and E. Soloway, "Novice mistakes: are the folk wisdoms correct?," Communications of the ACM, vol. 29, 1986.
[23]
N. Ragonis and M. Ben-Ari, "On understanding the statics and dynamics of object-oriented programs," ACM SIGCSE Bulletin, vol. 37, 2005, pp. 226--230.
[24]
M.C. Jadud, "A First Look at Novice Compilation Behaviour Using BlueJ," Computer Science Education, vol. 15, Mar. 2005, p. 25--40.
[25]
M.-H. Ng Cheong Vee, K. Mannock, and B. Meyer, "Empirical study of novice errors and error paths in object-oriented programming," Proceedings of the Conference of the Higher Education Academy, Subject Centre for Information and Computer Sciences, 2006, pp. 54--58.
[26]
J. Hage and P.V. Keeken, "Mining Helium programs with Neon," Technical Report, Department of Information and Computing Sciences, Utrecht University, 2007.
[27]
L. Murphy, G. Lewandowski, R. McCauley, B. Simon, L. Thomas, and C. Zander, "Debugging: the good, the bad, and the quirky - a qualitative analysis of novices' strategies," Proceedings of the Symposium on Computer Science Education, 2008, pp. 163--167.
[28]
M.C. Jadud, "Methods and tools for exploring novice compilation behaviour," Proceedings of the International Workshop on Computing Education Research, 2006, p. 73--84.
[29]
M.F. Köksal, R.E. Baar, and S. Üsküdarl1, "Screen-Replay: A Session Recording and Analysis Tool for DrScheme," Proceedings of the Scheme and Functional Programming Workshop, Technical Report, California Polytechnic State University, CPSLO-CSC-09-03, 2009, pp. 103--110.
[30]
N.J. Coull, SNOOPIE: development of a learning support tool for novice programmers within a conceptual framework, PhD Thesis, School of Computer Science, University Of St. Andrews, 2008.
[31]
H. Chad Lane and K. VanLehn, "Intention-based scoring: An approach to measuring success at solving the composition problem," Proceedings of the Symposium on Computer Science Education, New York, New York, USA: 2005, pp. 373--377.
[32]
M.-H. Nienaltowski, M. Pedroni, and B. Meyer, "Compiler Error Messages: What Can Help Novices?," Proceedings of the Technical Symposium on Computer Science Education, 2008, pp. 168--172.

Cited By

View all
  • (2024)About TeX's error messagesTUGboat10.47397/tb/45-3/tb141wermuth-errors45:3(388-398)Online publication date: 2024
  • (2024)The Effects of Generative AI on Computing Students’ Help-Seeking PreferencesProceedings of the 26th Australasian Computing Education Conference10.1145/3636243.3636248(39-48)Online publication date: 29-Jan-2024
  • (2024)"In the Beginning, I Couldn't Necessarily Do Anything With It": Links Between Compiler Error Messages and Sense of BelongingProceedings of the 2024 ACM Conference on International Computing Education Research - Volume 110.1145/3632620.3671105(14-26)Online publication date: 12-Aug-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
Onward! 2011: Proceedings of the 10th SIGPLAN symposium on New ideas, new paradigms, and reflections on programming and software
October 2011
134 pages
ISBN:9781450309417
DOI:10.1145/2048237
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 October 2011

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. beginner-friendly ides
  2. error message design
  3. novice programmers
  4. user-studies

Qualifiers

  • Research-article

Conference

SPLASH '11
Sponsor:

Acceptance Rates

Overall Acceptance Rate 40 of 105 submissions, 38%

Upcoming Conference

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)44
  • Downloads (Last 6 weeks)5
Reflects downloads up to 19 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)About TeX's error messagesTUGboat10.47397/tb/45-3/tb141wermuth-errors45:3(388-398)Online publication date: 2024
  • (2024)The Effects of Generative AI on Computing Students’ Help-Seeking PreferencesProceedings of the 26th Australasian Computing Education Conference10.1145/3636243.3636248(39-48)Online publication date: 29-Jan-2024
  • (2024)"In the Beginning, I Couldn't Necessarily Do Anything With It": Links Between Compiler Error Messages and Sense of BelongingProceedings of the 2024 ACM Conference on International Computing Education Research - Volume 110.1145/3632620.3671105(14-26)Online publication date: 12-Aug-2024
  • (2024)Teaching Programming Error Message UnderstandingWorking Group Reports on 2023 ACM Conference on Global Computing Education10.1145/3598579.3689377(1-30)Online publication date: 23-Sep-2024
  • (2024)Error Messages for Students Taught Using a Systematic Program Design CurriculumTrends in Functional Programming10.1007/978-3-031-74558-4_9(195-219)Online publication date: 8-Jan-2024
  • (2023)Regular Expressions in a CS Formal Languages CourseElectronic Proceedings in Theoretical Computer Science10.4204/EPTCS.382.3382(45-63)Online publication date: 14-Aug-2023
  • (2023)Getting into the Flow: Towards Better Type Error Messages for Constraint-Based Type InferenceProceedings of the ACM on Programming Languages10.1145/36228127:OOPSLA2(431-459)Online publication date: 16-Oct-2023
  • (2023)Framework for SQL Error Message Design: A Data-Driven ApproachACM Transactions on Software Engineering and Methodology10.1145/360718033:1(1-50)Online publication date: 23-Nov-2023
  • (2023)Always Provide Context: The Effects of Code Context on Programming Error Message EnhancementProceedings of the ACM Conference on Global Computing Education Vol 110.1145/3576882.3617909(147-153)Online publication date: 5-Dec-2023
  • (2023)A Study of Editor Features in a Creative Coding ClassroomProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580683(1-15)Online publication date: 19-Apr-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media