skip to main content
research-article

A Review of Worked Examples in Programming Activities

Published: 29 December 2022 Publication History

Abstract

This article reviews literature on worked examples in the context of programming activities. We focus on two types of examples, namely, code-tracing and code-generation, because there is sufficient research on these to warrant a review. We synthesize key results according to themes that emerged from the review. This synthesis aims to provide practical guidance for educators and shed light on future research opportunities. While there is established work in some areas (e.g., dynamic code-tracing examples in the form of program visualization tools, utility of subgoals in code-generation examples, and incomplete examples in the form of Parsons puzzles), there are also gaps. Thus, the article concludes with directions for future work on examples in computer science education.

References

[1]
Nouf M. Al-Barakati and Arwa Y. Al-Aama. 2009. The effect of visualizing roles of variables on student performance in an introductory programming course. In Proceedings of the 14th Annual ACM SIGCSE Conference on Innovation and Technology in Computer Science Education. ACM, New York, NY, 228–232.
[2]
John R. Anderson. 1993. Rules of the Mind. Psychology Press.
[3]
Robert K. Atkinson, Sharon J. Derry, Alexander Renkl, and Donald Wortham. 2000. Learning from examples: Instructional principles from the worked examples research. Rev. Edu. Res. 70, 2 (2000), 181–214.
[4]
Dipali D. Awasekar. 2013. Effect of program visualization to teach computer programming in a resource constrained classroom. In Proceedings of the IEEE 5th International Conference on Technology for Education. IEEE, 93–100.
[5]
Gargi Banerjee, Sahana Murthy, and Sridhar Iyer. 2015. Effect of active learning using program visualization in technology-constrained college classrooms. Res. Pract. Technol. Enhanced Learn. 10, 1 (2015), 1–25.
[6]
Piraye Bayman and Richard E. Mayer. 1988. Using conceptual models to teach BASIC computer programming. J. Edu. Psychol. 80, 3 (1988), 291.
[7]
Maik Beege, Sascha Schneider, Steve Nebel, Justus Zimm, Sarah Windisch, and Günter D. Rey. 2021. Learning programming from erroneous worked-examples. Which type of error is beneficial for learning? Learn. Instruct. 75 (2021), 101497.
[8]
Mordechai Ben-Ari, Roman Bednarik, Ronit B. B. Levy, Gil Ebel, Andrés Moreno, Niko Myller, and Erkki Sutinen. 2011. A decade of research and development on program animation: The Jeliot experience. J. Visual Lang. Comput. 22, 5 (2011), 375–384.
[9]
Kiran Bisra, Qing Liu, John C. Nesbit, Farimah Salimi, and Philip H. Winne. 2018. Inducing Self-Explanation: A Meta-Analysis. Edu. Psychol. Rev. 30, 3 (2018), 703–725.
[10]
Benedict Du Boulay. 1986. Some difficulties of learning to program. J. Edu. Comput. Res. 2, 1 (1986), 57–73. arXiv:https://doi.org/10.2190/3LFX-9RRF-67T8-UVK9
[11]
Peter Brusilovsky, Lauri Malmi, Roya Hosseini, Julio Guerra, Teemu Sirkiä, and Kerttu Pollari-Malmi. 2018. An integrated practice system for learning programming in Python: Design and evaluation. Res. Pract. Technol. Enhanced Learn. 13, 1 (2018), 1–40.
[12]
Richard Catrambone. 1998. The subgoal learning model: Creating better examples so that students can solve novel problems. J. Exper. Psychol.: Gen. 127, 4 (1998), 355.
[13]
Ibrahim Cetin. 2020. Teaching loops concept through visualization construction. Inform. Edu. Int. J. 19, 4 (2020), 589–609.
[14]
Kuo-En Chang, Bea-Chu Chiao, Sei-Wang Chen, and Rong-Shue Hsiao. 2000. A programming learning system for beginners-A completion strategy approach. IEEE Trans. Edu. 43, 2 (2000), 211–220.
[15]
Xingliang Chen, Antonija Mitrovic, and Moffat Mathews. 2016. Do erroneous examples improve learning in addition to problem solving and worked examples? In Proceedings of the Intelligent Tutoring Systems Conference, Alessandro Micarelli, John Stamper, and Kitty Panourgia (Eds.). Springer International Publishing, Cham, 13–22.
[16]
Michelene T. H. Chi, Miriam Bassok, Matthew W. Lewis, Peter Reimann, and Robert Glaser. 1989. Self-explanations: How students study and use examples in learning to solve problems. Cogn. Sci. 13, 2 (1989), 145–182.
[17]
Michelene T. H. Chi. 2009. Active-constructive-interactive: A conceptual framework for differentiating learning activities. Top. Cogn. Sci. 1, 1 (2009), 73–105.
[18]
Cristina Conati and Kurt Vanlehn. 2000. Toward computer-based support of meta-cognitive skills: A computational framework to coach self-explanation. Int. J. Artific. Intell. Edu. 11 (2000), 398–415.
[19]
Graham Cooper and John Sweller. 1987. Effects of schema acquisition and rule automation on mathematical problem-solving transfer. J. Edu. Psychol. 79, 4 (1987), 347.
[20]
Kathryn Cunningham, Sarah Blanchard, Barbara Ericson, and Mark Guzdial. 2017. Using tracing and sketching to solve programming problems: Replicating and extending an analysis of what students draw. In Proceedings of the ACM Conference on International Computing Education Research (ICER’17). ACM, New York, NY, 164–172.
[21]
Kathryn Cunningham, Shannon Ke, Mark Guzdial, and Barbara Ericson. 2019. Novice rationales for sketching and tracing, and how they try to avoid it. In Proceedings of the ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE’19). ACM, New York, NY, 37–43.
[22]
Aleksandar Davidovic, Jim Warren, and Elena Trichina. 2003. Learning benefits of structural example-based adaptive tutoring systems. IEEE Trans. Edu. 46, 2 (2003), 241–251.
[23]
Yuemeng Du, Andrew Luxton-Reilly, and Paul Denny. 2020. A review of research on parsons problems. In Proceedings of the 22nd Australasian Computing Education Conference (ACE’20). ACM, New York, NY, 195–202.
[24]
Kelley Durkin and Bethany Rittle-Johnson. 2012. The effectiveness of using incorrect examples to support learning about decimal magnitude. Learn. Instruct. 22, 3 (2012), 206–214.
[25]
Gil Ebel and Mordechai Ben-Ari. 2006. Affective effects of program visualization. In Proceedings of the 2nd International Workshop on Computing Education Research (ICER’06). ACM, New York, NY, 1–5.
[26]
Barbara J. Ericson, James D. Foley, and Jochen Rick. 2018. Evaluating the efficiency and effectiveness of adaptive parsons problems. In Proceedings of the ACM Conference on International Computing Education Research (ICER’18). ACM, New York, NY, 60–68.
[27]
Barbara J. Ericson, Lauren E. Margulieux, and Jochen Rick. 2017. Solving parsons problems versus fixing and writing code. In Proceedings of the 17th Koli Calling International Conference on Computing Education Research (KoliCalling’17). ACM, New York, NY, 20–29.
[28]
Geela V. F. Fabic, Antonija Mitrovic, and Kourosh Neshatian. 2019. Evaluation of parsons problems with menu-based self-explanation prompts in a mobile python tutor. Int. J. Artific. Intell. Edu. 29, 4 (2019), 507–535.
[29]
Sue Fitzgerald, Beth Simon, and Lynda Thomas. 2005. Strategies that students use to trace code: An analysis based in grounded theory. In Proceedings of the 1st International Workshop on Computing Education Research (ICER’05). ACM, New York, NY, 69–80.
[30]
Soniya Gadgil, Timothy J. Nokes-Malach, and Michelene T. H. Chi. 2012. Effectiveness of holistic mental model confrontation in driving conceptual change. Learn. Instruct. 22, 1 (2012), 47–61.
[31]
Dedre Gentner. 1983. Structure-mapping: A theoretical framework for analogy. Cogn. Sci. 7, 2 (1983), 155–170.
[32]
Nasser Giacaman. 2012. Teaching by example: Using analogies and live coding demonstrations to teach parallel computing concepts to undergraduate students. In Proceedings of the IEEE 26th International Parallel and Distributed Processing Symposium Workshops and PhD Forum. IEEE, 1295–1298.
[33]
Mary L. Gick and Keith J. Holyoak. 1980. Analogical problem solving. Cogn. Psychol. 12, 3 (1980), 306–355.
[34]
Jean M. Griffin. 2019. Designing intentional bugs for learning. In Proceedings of the 1st UK and Ireland Computing Education Research Conference. ACM, New York, NY, Article 5, 7 pages.
[35]
Cornelia S. Große and Alexander Renkl. 2007. Finding and fixing errors in worked examples: Can this foster learning outcomes? Learn. Instruct. 17, 6 (2007), 612–634.
[36]
Sally Hamouda, Stephen H. Edwards, Hicham G. Elmongui, Jeremy V. Ernst, and Clifford A. Shaffer. 2018. RecurTutor: An interactive tutorial for learning recursion. ACM Trans. Comput. Edu. 19, 1, Article 1 (Nov. 2018), 25 pages.
[37]
Qiang Hao, David H. Smith IV, Naitra Iriumi, Michail Tsikerdekis, and Amy J. Ko. 2019. A systematic investigation of replications in computing education research. ACM Trans. Comput. Edu. 19, 4, Article 42 (Aug2019), 18 pages.
[38]
Kyle J. Harms, Jason Chen, and Caitlin L. Kelleher. 2016. Distractors in parsons problems decrease learning efficiency for young novice programmers. In Proceedings of the ACM Conference on International Computing Education Research (ICER’16). ACM, New York, NY, 241–250.
[39]
Rachel Harsley, Nick Green, Mehrdad Alizadeh, Sabita Acharya, Davide Fossati, Barbara Di Eugenio, and Omar AlZoubi. 2016. Incorporating analogies and worked out examples as pedagogical strategies in a computer science tutoring system. In Proceedings of the 47th ACM Technical Symposium on Computing Science Education (SIGCSE’16). ACM, New York, NY, 675–680.
[40]
Carl C. Haynes and Barbara J. Ericson. 2021. Problem-solving efficiency and cognitive load for adaptive parsons problems vs. writing the equivalent code. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI’21). ACM, New York, NY, Article 60, 15 pages.
[41]
Juha Helminen, Petri Ihantola, Ville Karavirta, and Satu Alaoutinen. 2013. How do students solve parsons programming problems?–Execution-based vs. line-based feedback. In Proceedings of the Learning and Teaching in Computing and Engineering. IEEE, 55–61.
[42]
Matthew Hertz and Maria Jump. 2013. Trace-based teaching in early programming courses. In Proceedings of the 44th ACM Technical Symposium on Computer Science Education (SIGCSE’13). ACM, New York, NY, 561–566.
[43]
Jane Hoffswell, Arvind Satyanarayan, and Jeffrey Heer. 2018. Augmenting code with in situ visualizations to aid program understanding. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI’18). ACM, New York, NY, 1–12.
[44]
Roya Hosseini, Kamil Akhuseyinoglu, Peter Brusilovsky, Lauri Malmi, Kerttu Pollari-Malmi, Christian Schunn, and Teemu Sirkiä. 2020. Improving engagement in program construction examples for learning python programming. Int. J. Artific. Intell. Edu. 30, 2 (2020), 299–336.
[45]
Roya Hosseini, Kamil Akhuseyinoglu, Andrew Petersen, Christian D. Schunn, and Peter Brusilovsky. 2018. PCEX: Interactive program construction examples for learning programming. In Proceedings of the 18th Koli Calling International Conference on Computing Education Research (KoliCalling’18). ACM, New York, NY, Article 5, 9 pages.
[46]
Roya Hosseini, Teemu Sirkiä, Julio Guerra, Peter Brusilovsky, and Lauri Malmi. 2016. Animated examples as practice content in a Java programming course. In Proceedings of the 47th ACM Technical Symposium on Computing Science Education (SIGCSE’16). ACM, New York, NY, 540–545.
[47]
Ruanqianqian Huang, Kasra Ferdowsi, Ana Selvaraj, Adalbert Gerald Soosai Raj, and Sorin Lerner. 2022. Investigating the impact of using a live programming environment in a CS1 course. In Proceedings of the 53rd ACM Technical Symposium on Computer Science Education (SIGCSE’22). ACM, New York, NY, 495–501.
[48]
Derek Hwang, Vardhan Agarwal, Yuzi Lyu, Divyam Rana, Satya Ganesh Susarla, and Adalbert Gerald Soosai Raj. 2021. A qualitative analysis of lecture videos and student feedback on static code examples and live coding: A case study. In Proceedings of the Australasian Computing Education Conference (ACE’21). ACM, New York, NY, 147–157.
[49]
Michelle Ichinco, Kyle J. Harms, and Caitlin Kelleher. 2017. Towards understanding successful novice example use in blocks-based programming. J. Visual Lang. Sent. Syst. 3 (2017), 101–118.
[50]
Michelle Ichinco, Wint Yee Hnin, and Caitlin L. Kelleher. 2017. Suggesting API usage to novice programmers with the example guru. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI’17). ACM, New York, NY, 1105–1117.
[51]
Andrew F. Jarosz and Jennifer Wiley. 2014. What are the odds? A practical guide to computing and reporting Bayes factors. J. Problem Solv. 7, 1 (2014), 2.
[52]
Jay Jennings and Kasia Muldner. 2021. Investigating students’ reasoning in a code-tracing tutor. In Proceedings of the 22nd International Conference on Artificial Intelligence in Education (AIED’21). Springer-Verlag, Berlin, 203–214.
[53]
Jay Jennings and Kasia Muldner. 2021. When does scaffolding provide too much assistance? A code-tracing tutor investigation. Int. J. Artific. Intell. Edu. 31, 4 (2021), 784–819.
[54]
Johanna Joentausta and Arto Hellas. 2018. Subgoal labeled worked examples in K-3 education. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education (SIGCSE’18). ACM, New York, NY, 616–621.
[55]
Erkki Kaila, Teemu Rajala, Mikko-Jussi Laakso, and Tapio Salakoski. 2010. Effects of course-long use of a program visualization tool. In Proceedings of the 12th Australasian Conference on Computing Education, Vol. 103. ACM, 97–106.
[56]
Slava Kalyuga, Paul Ayres, Paul Chandler, and John Sweller. 2003. The expertise reversal effect. Edu. Psychol. 38, 1 (2003), 23–31. arXiv:https://doi.org/10.1207/S15326985EP3801_4
[57]
Manu Kapur. 2014. Productive failure in learning math. Cogn. Sci. 38, 5 (2014), 1008–1022.
[58]
Amruth N. Kumar. 2014. An evaluation of self-explanation in a programming tutor. In Proceedings of the International Conference on Intelligent Tutoring Systems. Springer International Publishing, 248–253.
[59]
Amruth N. Kumar. 2019. Helping students solve parsons puzzles better. In Proceedings of the ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE’19). ACM, New York, NY, 65–70.
[60]
Bridjet Lee and Kasia Muldner. 2020. Instructional video design: Investigating the impact of monologue-and dialogue-style presentations. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI’20). ACM, New York, NY, 1–12.
[61]
Alex Lishinski, Aman Yadav, Jon Good, and Richard Enbody. 2016. Learning to program: Gender differences and interactive effects of students’ motivation, goals, and self-efficacy on performance. In Proceedings of the ACM Conference on International Computing Education Research (ICER’16). ACM, New York, NY, 211–220.
[62]
Raymond Lister, Elizabeth S. Adams, Sue Fitzgerald, William Fone, John Hamer, Morten Lindholm, Robert McCartney, Jan Erik Moström, Kate Sanders, Otto Seppälä, Beth Simon, and Lynda Thomas. 2004. A multi-national study of reading and tracing skills in novice programmers. In Working Group Reports from ITiCSE on Innovation and Technology in Computer Science Education (ITiCSE-WGR’04). ACM, New York, NY, 119–150.
[63]
Tomasz D. Loboda and Peter Brusilovsky. 2010. User-adaptive explanatory program visualization: Evaluation and insights from eye movements. User Model. User-Adapt. Interact. 20, 3 (2010), 191–226.
[64]
Lauren E. Margulieux and Richard Catrambone. 2016. Improving problem solving with subgoal labels in expository text and worked examples. Learn. Instruct. 42 (2016), 58–71.
[65]
Lauren E. Margulieux and Richard Catrambone. 2017. Using learners’ self-explanations of subgoals to guide initial problem solving in app inventor. In Proceedings of the ACM Conference on International Computing Education Research (ICER’17). ACM, New York, NY, 21–29.
[66]
Lauren E. Margulieux and Richard Catrambone. 2019. Finding the best types of guidance for constructing self-explanations of subgoals in programming. J. Learn. Sci. 28, 1 (2019), 108–151.
[67]
Lauren E. Margulieux, Richard Catrambone, and Mark Guzdial. 2016. Employing subgoals in computer programming education. Comput. Sci. Edu. 26, 1 (2016), 44–67.
[68]
Lauren E. Margulieux, Richard Catrambone, and Laura M. Schaeffer. 2018. Varying effects of subgoal labeled expository text in programming, chemistry, and statistics. Instruct. Sci. 46, 5 (2018), 707–722.
[69]
Lauren E. Margulieux, Briana B. Morrison, and Adrienne Decker. 2020. Reducing withdrawal and failure rates in introductory programming with subgoal labeled worked examples. Int. J. STEM Edu. 7, 1 (2020), 1–16.
[70]
Lauren E. Margulieux, Briana B. Morrison, Baker Franke, and Harivololona Ramilison. 2020. Effect of implementing subgoals in Code.org’s intro to programming unit in computer science principles. ACM Trans. Comput. Edu. 20, 4, Article 26 (Oct.2020), 24 pages.
[71]
Lauren E. Margulieux, Briana B. Morrison, Mark Guzdial, and Richard Catrambone. 2016. Training learners to self-explain: Designing instructions and examples to improve problem solving. In Proceedings of the International Conference of the Learning Sciences (ICLS’16), Vol. 2. International Society of the Learning Sciences, 98–105. Retrieved from https://www.isls.org/icls/2016/docs/ICLS2016_Volume_1_30June2016.pdf.
[72]
Richard E. Mayer. 1975. Different problem-solving competencies established in learning computer programming with and without meaningful models. J. Edu. Psychol. 67, 6 (1975), 725.
[73]
Richard E. Mayer. 1976. Some conditions of meaningful learning for computer programming: Advance organizers and subject control of frame order. J. Edu. Psychol. 68, 2 (1976), 143.
[74]
Richard E. Mayer. 1989. Models for understanding. Rev. Edu. Res. 59, 1 (1989), 43–64.
[75]
Richard E. Mayer and Bruce K. Bromage. 1980. Difference recall protocols for technical texts due to advance organizers. J. Edu. Psychol. 72, 2 (1980), 209.
[76]
Monika Mladenović, Žana Žanko, and Marin A. Cuvic. 2021. The impact of using program visualization techniques on learning basic programming concepts at the K-12 level. Comput. Appl. Eng. Edu. 29, 1 (2021), 145–159.
[77]
Andrés Moreno, Erkki Sutinen, and Mike Joy. 2014. Defining and evaluating conflictive animations for programming education: The case of Jeliot ConAn. In Proceedings of the 45th ACM Technical Symposium on Computer Science Education (SIGCSE’14). ACM, New York, NY, 629–634.
[78]
Briana B. Morrison. 2017. Dual modality code explanations for novices: Unexpected results. In Proceedings of the ACM Conference on International Computing Education Research (ICER’17). ACM, New York, NY, 226–235.
[79]
Briana B. Morrison, Lauren E. Margulieux, and Adrienne Decker. 2020. The curious case of loops. Comput. Sci. Edu. 30, 2 (2020), 127–154.
[80]
Briana B. Morrison, Lauren E. Margulieux, Barbara Ericson, and Mark Guzdial. 2016. Subgoals help students solve Parsons problems. In Proceedings of the 47th ACM Technical Symposium on Computing Science Education (SIGCSE’16). ACM, New York, NY, 42–47.
[81]
Briana B. Morrison, Lauren E. Margulieux, and Mark Guzdial. 2015. Subgoals, context, and worked examples in learning computing problem solving. In Proceedings of the 11th Annual International Conference on Computing Education Research (ICER’15). ACM, New York, NY, 21–29.
[82]
Seyedreza Mousavi, Renae Low, and John Sweller. 1995. Reducing cognitive load by mixing auditory and visual presentation modes. J. Edu. Psychol. 87, 2 (1995), 319–334.
[83]
Kasia Muldner, Winslow Burleson, and Michelene Chi. 2014. Learning from self-explaining emergent phenomena. In Proceedings of the International Conference of the Learning Sciences (ICLS’14), Vol. 2. International Society of the Learning Sciences, 847–854.
[84]
Kasia Muldner and Cristina Conati. 2010. Scaffolding meta-cognitive skills for effective analogical problem solving via tailored example selection. Int. J. Artific. Intell. Edu. 20, 2 (2010), 99–136.
[85]
Amir Shareghi Najar, Antonija Mitrovic, and Bruce M. McLaren. 2016. Learning with intelligent tutors and worked examples: selecting learning activities adaptively leads to better learning outcomes than a fixed curriculum. User Model. User-Adapt. Interact. 26, 5 (Dec.2016), 459–491.
[86]
Greg L. Nelson, Benjamin Xie, and Amy J. Ko. 2017. Comprehension first: Evaluating a novel pedagogy and tutoring system for program tracing in CS1. In Proceedings of the ACM Conference on International Computing Education Research (ICER’17). ACM, New York, NY, 42–51.
[87]
Seppo Nevalainen and Jorma Sajaniemi. 2006. An experiment on short-term effects of animated versus static visualization of operations on program perception. In Proceedings of the 2nd International Workshop on Computing Education Research (ICER’06). ACM, New York, NY, 7–16.
[88]
Dale Parsons and Patricia Haden. 2006. Parson’s programming puzzles: A fun and effective learning tool for first programming courses. In Proceedings of the 8th Australasian Conference on Computing Education (ACE’06). Australian Computer Society, AUS, 157–163.
[89]
Elizabeth Patitsas, Michelle Craig, and Steve Easterbrook. 2013. Comparing and contrasting different algorithms leads to increased student learning. In Proceedings of the 9th Annual International ACM Conference on International Computing Education Research (ICER’13). ACM, New York, NY, 145–152.
[90]
José Baltasar García Pérez-Schofield, Matías García-Rivera, Francisco Ortin, and María J. Lado. 2019. Learning memory management with C-Sim: A C-based visual tool. Comput. Appl. Eng. Edu. 27, 5 (2019), 1217–1235.
[91]
D. N. Perkins, Chris Hancock, Renee Hobbs, Fay Martin, and Rebecca Simmons. 1986. Conditions of learning in novice programmers. J. Edu. Comput. Res. 2, 1 (1986), 37–55.
[92]
Thomas W. Price, Joseph Jay Williams, Jaemarie Solyst, and Samiha Marwan. 2020. Engaging students with instructor solutions in online programming homework. In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 1–7.
[93]
Yizhou Qian and James Lehman. 2017. Students’ misconceptions and other difficulties in introductory programming: A literature review. ACM Trans. Comput. Edu. 18, 1, Article 1 (Oct.2017), 24 pages.
[94]
Jill L. Quilici and R. Mayer. 1996. Role of examples in how students learn to categorize statistics word problems. J. Edu. Psychol. 88, 1 (1996), 144–161.
[95]
Adalbert Gerald Soosai Raj, Pan Gu, Eda Zhang, Arokia Xavier Annie R., Jim Williams, Richard Halverson, and Jignesh M. Patel. 2020. Live-coding vs static code examples: Which is better with respect to student learning and cognitive load? In Proceedings of the 22nd Australasian Computing Education Conference (ACE’20). ACM, New York, NY, 152–159.
[96]
Adalbert Gerald Soosai Raj, Jignesh M. Patel, Richard Halverson, and Erica Rosenfeld Halverson. 2018. Role of live-coding in learning introductory programming. In Proceedings of the 18th Koli Calling International Conference on Computing Education Research (KoliCalling’18). ACM, New York, NY, Article 13, 8 pages.
[97]
Teemu Rajala, Erkki Kaila, Johannes Holvitie, Riku Haavisto, Mikko-Jussi Laakso, and Tapio Salakoski. 2011. Comparing the collaborative and independent viewing of program visualizations. In Proceedings of the Frontiers in Education Conference (FIE’11). IEEE Computer Society, F3G–1-1–F3G–7.
[98]
Stephen K. Reed, Alexandra Dempster, and Michael Ettinger. 1985. Usefulness of analogous solutions for solving algebra word problems. J. Exper. Psychol.: Learn., Mem. Cogn. 11, 1 (Jan.1985), 106–125.
[99]
Alexander Renkl. 1997. Learning from worked-out examples: A study on individual differences. Cogn. Sci. 21, 1 (1997), 1–29.
[100]
Alexander Renkl. 2014. Toward an instructionally oriented theory of example-based learning. Cogn. Sci. 38 (2014), 1–37.
[101]
Bethany Rittle-Johnson, Abbey M. Loehr, and Kelley Durkin. 2017. Promoting self-explanation to improve mathematics learning: A meta-analysis and instructional design principles. ZDM Math. Edu. 49 (2017), 599–611.
[102]
Bethany Rittle-Johnson, Jon R. Star, and Kelley Durkin. 2012. Developing procedural flexibility: Are novices prepared to learn from comparing procedures? Brit. J. Edu. Psychol. 82, Pt 3 (2012), 436–55.
[103]
A. Robins, J. Rountree, and N. Rountree. 2003. Learning and teaching programming: A review and discussion. Comput. Sci. Edu. 13, 2 (2003), 137–172.
[104]
Brian H. Ross. 1987. This is like that: The use of earlier problems and the separation of similarity effects. J. Exper. Psychol.: Learn., Mem. Cogn. 13, 4 (Oct.1987), 629–639.
[105]
Marc J. Rubin. 2013. The effectiveness of live-coding to teach introductory programming. In Proceedings of the 44th ACM Technical Symposium on Computer Science Education (SIGCSE’13). ACM, New York, NY, 651–656.
[106]
Jorma Sajaniemi, Marja Kuittinen, and Taina Tikansalo. 2008. A study of the development of students’ visualizations of program state during an elementary object-oriented programming course. J. Edu. Res. Comput. 7, 4, Article 3 (Jan.2008), 31 pages.
[107]
Dale Schunk, Antoinette R. Hanson, and Paula D. Cox. 1987. Peer-model attributes and children’s achievement behaviors. J. Edu. Psychol. 79, 1 (1987), 54–61.
[108]
Ana Selvaraj, Eda Zhang, Leo Porter, and Adalbert Gerald Soosai Raj. 2021. Live coding: A review of the literature. In Proceedings of the 26th ACM Conference on Innovation and Technology in Computer Science Education. ACM, New York, NY, 164–170.
[109]
Amir Shareghi Najar and Antonija Mitrovic. 2013. Examples and tutored problems: How can self-explanation make a difference to learning? In Artificial Intelligence in Education, H. Chad Lane, Kalina Yacef, Jack Mostow, and Philip Pavlik (Eds.). Springer, Berlin, 339–348.
[110]
Nianfeng Shi, Zhiyu Min, and Ping Zhang. 2017. Effects of visualizing roles of variables with animation and IDE in novice program construction. Telemat. Informat. 34, 5 (2017), 743–754.
[111]
Teemu Sirkiä and Juha Sorva. 2015. How do students use program visualizations within an interactive ebook? In Proceedings of the 11th Annual International Conference on International Computing Education Research (ICER’15). ACM, New York, NY, 179–188.
[112]
Elliot Soloway and Kate Ehrlich. 1984. Empirical studies of programming knowledge. IEEE Trans. Softw. Eng. SE-10, 5 (1984), 595–609.
[113]
Juha Sorva. 2013. Notional machines and introductory programming education. ACM Trans. Comput. Edu. 31, 2, Article 8 (July2013), 31 pages.
[114]
Juha Sorva, Ville Karavirta, and Lauri Malmi. 2013. A review of generic program visualization systems for introductory programming education. ACM Trans. Comput. Edu. 13, 4, Article 15 (Nov.2013), 64 pages.
[115]
Ben Stephenson. 2019. Coding demonstration videos for CS1. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education (SIGCSE’19). ACM, New York, NY, 105–111.
[116]
John Sweller. 2011. Cognitive load theory, in Psychology of Learning and Motivation, Vol. 55. Academic Press, 37–76.
[117]
John Sweller, Paul Ayres, and Slava Kalyuga. 2011. The split-attention effect. In Cognitive Load Theory. Springer, New York, NY, 111–128.
[118]
John Sweller and Graham A. Cooper. 1985. The use of worked examples as a substitute for problem solving in learning algebra. Cogn. Instruct. 2, 1 (1985), 59–89.
[119]
John G. Trafton and Brian J. Reiser. 1993. Studying examples and solving problems: Contributions to skill acquisition. In Proceedings of the 15th Annual Conference of the Cognitive Science Society. Lawrence Erlbaum Associates, 1017–1022.
[120]
Jaime Urquiza-Fuentes and J. Ángel Velázquez-Iturbide. 2013. Toward the effective use of educational program animations: The roles of student’s engagement and topic complexity. Comput. Edu. 67 (2013), 178–192.
[121]
Vesa Vainio and Jorma Sajaniemi. 2007. Factors in novice programmers’ poor tracing skills. In Proceedings of the 12th Annual SIGCSE Conference on Innovation and Technology in Computer Science Education (ITiCSE’07). ACM, New York, NY, 236–240.
[122]
Tamara van Gog, Liesbeth Kester, and Fred Paas. 2011. Effects of worked examples, example-problem, and problem-example pairs on novices’ learning. Contemp. Edu. Psychol. 36, 3 (2011), 212–218.
[123]
Tamara van Gog and Nikol Rummel. 2010. Example-Based Learning: Integrating Cognitive and Social-Cognitive Research Perspectives. Edu. Psychol. Rev. 22, 2 (June2010), 155–174.
[124]
Tamara van Gog, Nikol Rummel, and Alexander Renkl. 2019. Learning how to solve problems by studying examples. In The Cambridge Handbook of Cognition and Education, John Dunlosky and Katherine A.Editors Rawson (Eds.). Cambridge University Press, 183–208.
[125]
Jeroen J. G. van Merriënboer, Jan Gerrit Schuurman, Marcel B. M. de Croock, and Fred Paas. 2002. Redirecting learners attention during training: Effects on cognitive load, transfer test performance and training efficiency. Learn. Instruct. 12, 1 (2002), 11–37.
[126]
Jeroen J. G. Van Merriënboer. 1990. Strategies for programming instruction in high school program: Program completion vs. program generation. J. Edu. Comput. Res. 6, 3 (1990), 265–285.
[127]
Jeroen J. G. van Merriënboer and Marcel B. M. de Croock. 1992. Strategies for computer-based programming instruction: Program completion vs. program generation. J. Edu. Comput. Res. 8, 3 (1992), 365–394.
[128]
Kurt VanLehn. 1996. Cognitive skill acquisition. Annu. Rev. Psychol. 47, 1 (1996), 513–539. arXiv:https://doi.org/10.1146/annurev.psych.47.1.513PMID: 15012487.
[129]
Kurt VanLehn. 1998. Analogy events: How examples are used during problem solving. Cogn. Sci. 22, 3 (1998), 347–388.
[130]
Kurt VanLehn. 1999. Rule-learning events in the acquisition of a complex skill: An evaluation of cascade. J. Learn. Sci. 8, 1 (1999), 71–125.
[131]
Peng Wang, Roman Bednarik, and Andrés Moreno. 2012. During automatic program animation explanations after animations have greater impact than before animations. In Proceedings of the 12th Koli Calling International Conference on Computing Education Research (KoliCalling’12). ACM, New York, NY, 100–109.
[132]
Wengran Wang, Archit Kwatra, James Skripchuk, Neeloy Gomes, Alexandra Milliken, Chris Martens, Tiffany Barnes, and Thomas Price. 2021. Novices’ learning barriers when using code examples in open-ended programming. In Proceedings of the 26th ACM Conference on Innovation and Technology in Computer Science Education V. 1. ACM, New York, NY, 394–400.
[133]
Wengran Wang, Yudong Rao, Rui Zhi, Samiha Marwan, Ge Gao, and Thomas W. Price. 2020. Step Tutor: Supporting students through step-by-step example-based feedback. In Proceedings of the ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE’20). ACM, New York, NY, 391–397.
[134]
Gerhard Weber. 1996. Individual selection of examples in an intelligent learning environment. Int. J. Artific. Intell. Edu. 7, 1 (1996), 3–31.
[135]
Nathaniel Weinman, Armando Fox, and Marti A. Hearst. 2021. Improving instruction of programming patterns with faded Parsons problems. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI’21). ACM, New York, NY, Article 53, 4 pages.
[136]
Ruth Wylie and Michelene T. H. Chi. 2014. The self-explanation principle in multimedia learning. In The Cambridge Handbook of Multimedia Learning (2nd ed.), Richard E. Mayer (Ed.). Cambridge University Press, 413–432.
[137]
Benjamin Xie, Dastyni Loksa, Greg L. Nelson, Matthew J. Davidson, Dongsheng Dong, Harrison Kwik, Alex Hui Tan, Leanne Hwa, Min Li, and Amy J. Ko. 2019. A theory of instruction for introductory programming skills. Comput. Sci. Edu. 29, 2–3 (2019), 205–253.
[138]
Benjamin Xie, Greg L. Nelson, and Amy J. Ko. 2018. An explicit strategy to scaffold novice program tracing. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education (SIGCSE’18). ACM, New York, NY, 344–349.
[139]
Jeong Yang, Young Lee, and Kai-Hsiung Chang. 2018. Evaluations of JaguarCode: A web-based object-oriented programming environment with static and dynamic visualization. J. Syst. Softw. 145 (2018), 147–163.
[140]
Albina Zavgorodniaia, Arto Hellas, Otto Seppälä, and Juha Sorva. 2020. Should explanations of program code use audio, text, or both? A replication study. In Proceedings of the 20th Koli Calling International Conference on Computing Education Research (KoliCalling’20). ACM, New York, NY, Article 5, 10 pages.
[141]
Albina Zavgorodniaia, Artturi Tilanterä, Ari Korhonen, Otto Seppälä, Arto Hellas, and Juha Sorva. 2021. Algorithm visualization and the elusive modality effect. In Proceedings of the 17th ACM Conference on International Computing Education Research (ICER’21). ACM, New York, NY, 368–378.
[142]
Rui Zhi, Min Chi, Tiffany Barnes, and Thomas W. Price. 2019. Evaluating the effectiveness of Parsons problems for block-based programming. In Proceedings of the ACM Conference on International Computing Education Research (ICER’19). ACM, New York, NY, 51–59.

Cited By

View all
  • (2025)Mapping the Anatomy of Novice-Generated Mistakes in Learning ProgrammingEffective Computer Science Education in K-12 Classrooms10.4018/979-8-3693-4542-9.ch007(171-192)Online publication date: 24-Jan-2025
  • (2024)Evaluating the Efficacy of Peer-Created Worked-Example Videos in a Computer Systems CourseJournal of Computing Sciences in Colleges10.5555/3665609.366561539:8(83-97)Online publication date: 17-May-2024
  • (2024)Examples out of Thin Air: AI-Generated Dynamic Context to Assist Program Comprehension by ExampleCompanion Proceedings of the 8th International Conference on the Art, Science, and Engineering of Programming10.1145/3660829.3660845(99-107)Online publication date: 11-Mar-2024
  • Show More Cited By

Index Terms

  1. A Review of Worked Examples in Programming Activities

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Computing Education
    ACM Transactions on Computing Education  Volume 23, Issue 1
    March 2023
    396 pages
    EISSN:1946-6226
    DOI:10.1145/3578368
    • Editor:
    • Amy J. Ko
    Issue’s Table of Contents

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 29 December 2022
    Online AM: 02 September 2022
    Accepted: 04 August 2022
    Revised: 20 May 2022
    Received: 31 July 2021
    Published in TOCE Volume 23, Issue 1

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Worked examples
    2. review
    3. code-tracing examples
    4. code-generation examples

    Qualifiers

    • Research-article
    • Refereed

    Funding Sources

    • NSERC Discovery Grant

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)435
    • Downloads (Last 6 weeks)25
    Reflects downloads up to 13 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)Mapping the Anatomy of Novice-Generated Mistakes in Learning ProgrammingEffective Computer Science Education in K-12 Classrooms10.4018/979-8-3693-4542-9.ch007(171-192)Online publication date: 24-Jan-2025
    • (2024)Evaluating the Efficacy of Peer-Created Worked-Example Videos in a Computer Systems CourseJournal of Computing Sciences in Colleges10.5555/3665609.366561539:8(83-97)Online publication date: 17-May-2024
    • (2024)Examples out of Thin Air: AI-Generated Dynamic Context to Assist Program Comprehension by ExampleCompanion Proceedings of the 8th International Conference on the Art, Science, and Engineering of Programming10.1145/3660829.3660845(99-107)Online publication date: 11-Mar-2024
    • (2024)Identifying K-12 Students' Approaches to Using Worked Examples for Epistemic ProgrammingProceedings of the 2024 Symposium on Eye Tracking Research and Applications10.1145/3649902.3655094(1-7)Online publication date: 4-Jun-2024
    • (2024)Evaluating LLM-generated Worked Examples in an Introductory Programming CourseProceedings of the 26th Australasian Computing Education Conference10.1145/3636243.3636252(77-86)Online publication date: 29-Jan-2024
    • (2023)JuGaze: A Cell-based Eye Tracking and Logging Tool for Jupyter NotebooksProceedings of the 23rd Koli Calling International Conference on Computing Education Research10.1145/3631802.3631824(1-11)Online publication date: 13-Nov-2023
    • (2023)Multi-Institutional Multi-National Studies of Parsons ProblemsProceedings of the 2023 Working Group Reports on Innovation and Technology in Computer Science Education10.1145/3623762.3633498(57-107)Online publication date: 22-Dec-2023
    • (2023)Toward Studying Example-Based Live Programming in CS/SE EducationProceedings of the 2nd ACM SIGPLAN International Workshop on Programming Abstractions and Interactive Notations, Tools, and Environments10.1145/3623504.3623568(17-24)Online publication date: 18-Oct-2023
    • (2023)Evaluating the Utility of Notional Machine Representations to Help Novices Learn to Code TraceProceedings of the 2023 ACM Conference on International Computing Education Research - Volume 110.1145/3568813.3600119(314-328)Online publication date: 7-Aug-2023
    • (2023)Learning programming practice and programming theory in the computer laboratoryEuropean Journal of Engineering Education10.1080/03043797.2023.229495349:2(330-347)Online publication date: 26-Dec-2023

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    Full Text

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media