skip to main content
10.1145/3545945.3569741acmconferencesArticle/Chapter ViewAbstractPublication PagessigcseConference Proceedingsconference-collections
research-article
Public Access

On Students' Usage of Tracing for Understanding Code

Published: 03 March 2023 Publication History

Abstract

Explain in Plain English (EiPE) questions evaluate whether students can understand and explain the high-level purpose of code. We conducted a qualitative think-aloud study of introductory programming students solving EiPE questions. In this paper, we focus on how students use tracing (mental execution) to understand code in order to explain it.
We found that, in some cases, tracing can be an effective strategy for novices to understand and explain code. Furthermore, we observed three problems that prevented tracing from being helpful, which are 1) not employing tracing when it could be helpful (some struggling students explained correctly after the interviewer suggested tracing the code), 2) tracing incorrectly due to misunderstandings of the programming language, and 3) tracing with a set of inputs that did not sufficiently expose the code's behavior (upon interviewer suggesting inputs, students explained correctly). These results suggest that we should teach students to use tracing as a method for understanding code and teach them how to select appropriate inputs to trace.

References

[1]
Austin Cory Bart, Teomara Rutherford, and James Skripchuk. 2020. Evaluating an Instrumented Python CS1 Course. In CSEDM@ EDM.
[2]
John B. Biggs and K. F. Collis. 1982. Evaluating the quality of learning : the SOLO taxonomy (structure of the observed learning outcome) / John B. Biggs, Kevin F. Collis. Academic Press New York. xiii, 245 p. : pages.
[3]
Binglin Chen, Sushmita Azad, Rajarshi Haldar, Matthew West, and Craig Zilles. 2020. A Validated Scoring Rubric for Explain-in-Plain-English Questions. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education (SIGCSE).
[4]
Woei-Kae Chen, Chien-Hung Liu, and Hong-Ming Huang. 2017. Software Debugging Patterns for Novice Programmers. (2017).
[5]
Chih-Yueh Chou and Peng-Fei Sun. 2013. An educational tool for visualizing students' program tracing processes. Computer Applications in Engineering Education, Vol. 21, 3 (2013), 432--438.
[6]
Kathryn Cunningham, Sarah Blanchard, Barbara Ericson, and Mark Guzdial. 2017. Using tracing and sketching to solve programming problems: replicating and extending an analysis of what students draw. In Proceedings of the 2017 ACM Conference on International Computing Education Research. 164--172.
[7]
Kathryn Cunningham, Shannon Ke, Mark Guzdial, and Barbara Ericson. 2019. Novice rationales for sketching and tracing, and how they try to avoid it. In Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science Education. 37--43.
[8]
Paul Denny, James Prather, Brett A Becker, Zachary Albrecht, Dastyni Loksa, and Raymond Pettit. 2019. A closer look at metacognitive scaffolding: Solving test cases before programming. In Proceedings of the 19th Koli Calling international conference on computing education research. 1--10.
[9]
Francc oise Détienne and Elliot Soloway. 1990. An empirically-derived control structure for the process of program understanding. International Journal of Man-Machine Studies, Vol. 33, 3 (1990), 323--342.
[10]
Stephen H Edwards and Zalia Shams. 2014. Do student programmers all tend to write the same software tests?. In Proceedings of the 2014 conference on Innovation & technology in computer science education. 171--176.
[11]
K. Anders Ericsson and Herbert A. Simon. 1980. Verbal reports as data. Psychological Review, Vol. 87 (1980), 215--251.
[12]
Max Fowler, David H Smith IV, Mohammed Hassan, Seth Poulsen, Matthew West, and Craig Zilles. 2022. Reevaluating the relationship between explaining, tracing, and writing skills in CS1 in a replication study. Computer Science Education (2022), 1--29.
[13]
David Ginat. 2007. Hasty design, futile patching and the elaboration of rigor. In Proceedings of the 12th annual SIGCSE conference on Innovation and technology in computer science education. 161--165.
[14]
Raymond Lister, Elizabeth S Adams, Sue Fitzgerald, William Fone, John Hamer, Morten Lindholm, Robert McCartney, Jan Erik Moström, Kate Sanders, Otto Seppälä, Beth Simon, and Lynda Thomas. 2004. A multi-national study of reading and tracing skills in novice programmers. ACM SIGCSE Bulletin, Vol. 36, 4 (2004), 119--150.
[15]
Dastyni Loksa and Amy J Ko. 2016. The role of self-regulation in programming problem solving process and success. In Proceedings of the 2016 ACM conference on international computing education research. 83--91.
[16]
Mike Lopez, Jacqueline Whalley, Phil Robbins, and Raymond Lister. 2008. Relationships between reading, tracing and writing skills in introductory programming. In Proceedings of the Fourth International Workshop on Computing Education Research. ACM, 101--112.
[17]
Rifat Sabbir Mansur, Ayaan M Kazerouni, Stephen H Edwards, and Clifford A Shaffer. 2020. Exploring the Bug Investigation Techniques of Intermediate Student Programmers. In Koli Calling'20: Proceedings of the 20th Koli Calling International Conference on Computing Education Research. 1--10.
[18]
Laurie Murphy, Renée McCauley, and Sue Fitzgerald. 2012. 'Explain in plain English'questions: implications for teaching. In Proceedings of the 43rd ACM technical symposium on Computer Science Education. 385--390.
[19]
Greg L Nelson, Benjamin Xie, and Amy J Ko. 2017. Comprehension first: evaluating a novel pedagogy and tutoring system for program tracing in CS1. In Proceedings of the 2017 ACM conference on international computing education research. 2--11.
[20]
James Prather, Raymond Pettit, Brett A Becker, Paul Denny, Dastyni Loksa, Alani Peters, Zachary Albrecht, and Krista Masci. 2019. First things first: Providing metacognitive scaffolding for interpreting problem prompts. In Proceedings of the 50th ACM technical symposium on computer science education. 531--537.
[21]
Donna Teague, Malcolm Corney, Alireza Ahadi, and Raymond Lister. 2013. A qualitative think aloud study of the early neo-piagetian stages of reasoning in novice programmers. In Proceedings of the 15th Australasian Computing Education Conference [Conferences in Research and Practice in Information Technology, Volume 136]. Australian Computer Society, 87--95.
[22]
Donna Teague and Raymond Lister. 2014a. Blinded by their Plight: Tracing and the Preoperational Programmer. In PPIG. 8.
[23]
Donna Teague and Raymond Lister. 2014b. Longitudinal think aloud study of a novice programmer. In Conferences in Research and Practice in Information Technology Series.
[24]
Renske Weeda, Cruz Izu, Maria Kallia, and Erik Barendsen. 2020. Towards an Assessment Rubric for EiPE Tasks in Secondary Education: Identifying Quality Indicators and Descriptors. In Koli Calling'20: Proceedings of the 20th Koli Calling International Conference on Computing Education Research. 1--10.
[25]
Jacqueline Whalley, Raymond Lister, Errol Thompson, Tony Clear, Phil Robbins, P K Ajith Kumar, and Christine Prasad. 2006. An Australasian study of Reading and Comprehension Skills in Novice Programmers, using the Bloom and SOLO Taxonomies. Eighth Australasian Computing Education Conference (ACE2006) (2006).
[26]
Jack Wrenn and Shriram Krishnamurthi. 2021. Reading Between the Lines: Student Help-Seeking for (Un) Specified Behaviors. In 21st Koli Calling International Conference on Computing Education Research. 1--6.
[27]
John Wrenn, Tim Nelson, and Shriram Krishnamurthi. 2021. Using Relational Problems to Teach Property-Based Testing. The art science and engineering of programming, Vol. 5, 2 (2021).
[28]
Benjamin Xie, Dastyni Loksa, Greg L Nelson, Matthew J Davidson, Dongsheng Dong, Harrison Kwik, Alex Hui Tan, Leanne Hwa, Min Li, and Andrew J Ko. 2019. A theory of instruction for introductory programming skills. Computer Science Education, Vol. 29, 2--3 (2019), 205--253.

Cited By

View all
  • (2024)Evaluating How Novices Utilize Debuggers and Code Execution to Understand CodeProceedings of the 2024 ACM Conference on International Computing Education Research - Volume 110.1145/3632620.3671126(65-83)Online publication date: 12-Aug-2024
  • (2023)Evaluating Beacons, the Role of Variables, Tracing, and Abstract Tracing for Teaching Novices to Understand Program IntentProceedings of the 2023 ACM Conference on International Computing Education Research - Volume 110.1145/3568813.3600140(329-343)Online publication date: 7-Aug-2023

Index Terms

  1. On Students' Usage of Tracing for Understanding Code

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    SIGCSE 2023: Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1
    March 2023
    1481 pages
    ISBN:9781450394314
    DOI:10.1145/3545945
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 03 March 2023

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. explain in plain english
    2. think-aloud
    3. tracing

    Qualifiers

    • Research-article

    Funding Sources

    Conference

    SIGCSE 2023
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 1,595 of 4,542 submissions, 35%

    Upcoming Conference

    SIGCSE TS 2025
    The 56th ACM Technical Symposium on Computer Science Education
    February 26 - March 1, 2025
    Pittsburgh , PA , USA

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)124
    • Downloads (Last 6 weeks)12
    Reflects downloads up to 17 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Evaluating How Novices Utilize Debuggers and Code Execution to Understand CodeProceedings of the 2024 ACM Conference on International Computing Education Research - Volume 110.1145/3632620.3671126(65-83)Online publication date: 12-Aug-2024
    • (2023)Evaluating Beacons, the Role of Variables, Tracing, and Abstract Tracing for Teaching Novices to Understand Program IntentProceedings of the 2023 ACM Conference on International Computing Education Research - Volume 110.1145/3568813.3600140(329-343)Online publication date: 7-Aug-2023

    View Options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Login options

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media