MatchingRef: Matching Variable Names in a Reference Page to Help Introductory CS Students Fix Compiler Errors

Debugging compiler errors is essential to programming and can be challenging for novice programmers. In introductory computer science courses, challenging errors can discourage students. One reason these errors are difficult to resolve is that most online help systems do not match a student's code. For example, online reference pages use different variable names, identifiers, and method names compared with a student's particular code. To utilize existing resources, students must wade through other people's code (which is often too advanced for novices to comprehend). This is time-consuming and does not provide novices with solutions.

To address this problem, we developed MatchingRef -a reference system that helps novices resolve compiler errors. It is a web-based reference guide that catalogs common Processing/Java compiler errors. MatchingRef integrates with the Processing programming environment to provide users with explanations and that match users' particular code. It includes a list of strategies to fix each error, accompanied by one or more concrete examples. Importantly, the key feature that distinguishes MatchingRef from prior reference systems (e.g., Decaf [1], HelpMeOut [4], CodeWrite [2]) is that MatchingRef examples are all dynamically generated using variable, method, and class names from users' programming environments.

There are four guiding design principles behind MatchingRef. First is readability, which is listed as a criterion that good error messages should exhibit [5]. The description from Java documentation is usually technical and causes confusion for novice programmers. Hence, we provide explanations that are less cryptic and more familiar in wording. Second is learning by examples. Examples have been shown to be effective way to learn and fix programming errors, such as in online forum like Stack Overflow [7]. Therefore, we include examples in each suggestion so that users can have a concrete idea on how to fix the problems. Third is familiarity, a common design principle across domains [3]. MatchingRef matches elements in the fixing examples with the original code of the users. Finally, our system is intended to reduce cognitive load by customizing the pages to users' code to show the relevant information. According to Cognitive Load Theory (CLT), humans have a finite ability to efficiently process input [6]. As MatchingRef provide users with matching error messages and matching names, their brain only needs to process useful information such as ways to fix the errors or roots of the errors. Users can avoid reading some information in the page multiple times such as variable names or class names because they are already familiar with them.

We conducted a within-subjects pilot study (n = 4) to evaluate whether MatchingRef improved novices' performance and comprehension while debugging compiler errors. Participants interacted with two designs of our system, one with matching variable names in the examples and one without this feature. Otherwise, the systems were identical. The participants were students in introductory computer science courses that use Processing. Due to the small sample size, we could not detect significant differences in task completion time. Although the conclusions we can draw are limited, we are encouraged to evaluate MatchingRef in a more formal study.