Skip to main content

Advertisement

SpringerLink
Log in

Improving Engagement in Program Construction Examples for Learning Python Programming

  • ARTICLE
  • Published:
International Journal of Artificial Intelligence in Education Aims and scope Submit manuscript

Abstract

This research is focused on how to support students’ acquisition of program construction skills through worked examples. Although examples have been consistently proven to be valuable for student’s learning, the learning technology for computer science education lacks program construction examples with interactive elements that could engage students. The goal of this work is to investigate the value of the “engaging” features in programming examples. We introduce PCEX, an online tool developed to present program construction examples in an engaging fashion. We also present the results of a controlled study with a between-subject design that was conducted in a large introductory Python programming class to compare PCEX with non-interactive worked examples focused on program construction. The results of our study show the positive impact of interactive program construction examples on student’s engagement, problem-solving performance, and learning.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. We chose start date over due date as the time reference because some students submitted assignments after the due date was passed. Therefore, using the difference between the due date and the submission date would have made the interpretation of results more difficult, as some differences would be negative.

  2. Using all students’ data showed the same pattern of results for this analysis.

References

  • Aiken, L.S, West, S.G, & Reno, R.R. (1991). Multiple regression: Testing and interpreting interactions. Sage.

  • Atkinson, R.K, & Renkl, A. (2007). Interactive example-based learning environments: Using interactive elements to encourage effective processing of worked examples. Educational Psychology Review, 19(3), 375–386.

    Google Scholar 

  • Atkinson, R.K, Derry, S.J, Renkl, A., & Wortham, D. (2000). Learning from examples: Instructional principles from the worked examples research. Review of educational research, 70(2), 181– 214.

    Google Scholar 

  • Auvinen, T., Hakulinen, L., & Malmi, L. (2018). Increasing students’ awareness of their behavior in online learning environments with visualizations and achievement badges. IEEE Transactions on Learning Technologies, 8(3), 261–273.

    Google Scholar 

  • Bathgate, M., & Schunn, C. (2017). The psychological characteristics of experiences that influence science motivation and content knowledge. International Journal of Science Education, 39(17), 2402– 2432.

    Google Scholar 

  • Bloom, S.B. (1956). Taxonomy of educational objectives, handbook I: The cognitive domain. New York: David McKay Co Inc.

    Google Scholar 

  • Brna, P. (1998). Searching for examples with a programming techniques editor. Journal of Computing and Information Technology, 6(1), 13–26.

    Google Scholar 

  • Brusilovsky, P. (1994). Explanatory visualization in an educational programming environment: connecting examples with general knowledge. In: 4th international conference on human-computer interaction, EWHCI’94, vol. 876 of Lecture Notes in Computer Science, pp. 202–212. Springer-Verlag.

  • Brusilovsky, P, & Yudelson, M.V. (2008). From webex to navex: Interactive access to annotated program examples. Proceedings of the IEEE, 96(6), 990–999.

    Google Scholar 

  • Brusilovsky, P., Yudelson, M., & Hsiao, I-H. (2009). Problem solving examples as first class objects in educational digital libraries: Three obstacles to overcome. Journal of Educational Multimedia and Hypermedia, 18(3), 267–288.

    Google Scholar 

  • Byrne, M. D, Catrambone, R., & Stasko, J.T. (1999). Evaluating animations as student aids in learning computer algorithms. Computers & education, 33(4), 253–278.

    Google Scholar 

  • Catrambone, R. (1998). The subgoal learning model: Creating better examples so that students can solve novel problems. Journal of Experimental Psychology: General, 127(4), 355–376.

    Google Scholar 

  • Chen, X., Mitrovic, A., & Matthews, M. (2019a). Learning from worked examples, erroneous examples and problem solving: Towards adaptive selection of learning activities. IEEE Transactions on Learning Technologies, pages 1–1.

  • Chen, X., Mitrovic, A., & Matthews, M. (2019b). Investigating the effect of agency on learning from worked examples, erroneous examples and problem solving. International Journal of Artificial Intelligence in Education.

  • Chi, M.T.H., Bassok, M., Lewis, M.W., Reimann, P., & Glaser, R. (1989). Self-explanations: How students study and use examples in learning to solve problems. Cognitive Science, 13(2), 145–182.

    Google Scholar 

  • Christenson, S.L., Reschly, A.L., & Wylie, C. (2012). Handbook of research on student engagement. Springer Science & Business Media.

  • Cooper, S., Dann, W., & Pausch, R. (2003). Teaching objects-first in introductory computer science. In: ACM SIGCSE Bulletin, vol. 35, pp. 191–195. ACM.

  • Dann, W., Cosgrove, D., Slater, D., Culyba, D., & Cooper, S. (2012). Mediated transfer: Alice 3 to java. In: Proceedings of the 43rd ACM technical symposium on Computer Science Education, pp. 141–146. ACM.

  • Davidovic, A., Warren, J., & Trichina, E. (2003). Learning benefits of structural example-based adaptive tutoring systems. IEEE Transactions on Education, 46(2), 241–251.

    Google Scholar 

  • Ericson, B.J., Guzdial, M.J., & Morrison, B.B. (2015). Analysis of interactive features designed to enhance learning in an ebook. In: Proceedings of the 11th annual international conference on international computing education research, pp. 169–178 ACM.

  • Ericson, B.J, Margulieux, L.E., & Rick, J. (2017). Solving parsons problems versus fixing and writing code. In: Proceedings of the 17th Koli calling conference on computing education research, pp. 20–29. ACM.

  • Esteves, M., & Mendes, A. (2003). Oop-anim, a system to support learning of basic object oriented programming concepts. In: Proceedings of compsystech’2003-international conference on computer systems and technologies, Sofia. Bulgaria.

  • Evans, C., & Gibbons, N.J. (2007). The interactivity effect in multimedia learning. Computers & Education, 49(4), 1147–1160.

    Google Scholar 

  • Fabic, G.V.F., Mitrovic, A., & Neshatian, K. (2019). Evaluation of parsons problems with menu-based self-explanation prompts in a mobile python tutor. International Journal of Artificial Intelligence in Education. ISSN 1560-4306.

  • Getao, K.W. (1990). An environment to support the use of program examples while learning to program in lisp. In: Proceedings of the IFIP TC13 3rd international conference on human-computer interaction, pp. 1015–1016. North-Holland Publishing Co.

  • Hansen, S.R., Narayanan, N.H., & Schrimpsher, D. (2000). Helping learners visualize and comprehend algorithms. Interactive Multimedia Electronic Journal of Computer-Enhanced Learning, 2(1).

  • Harvey, B., & Mönig, J. (2010). Bringing “no ceiling” to scratch: Can one language serve kids and computer scientists. Proc Constructionism.

  • Hundhausen, C.D., Douglas, S.A., & Stasko, J.T. (2002). A meta-study of algorithm visualization effectiveness. Journal of Visual Languages & Computing, 13 (3), 259–290.

    Google Scholar 

  • Ihantola, P., & Karavirta, V. (2011). Two-Dimensional Parson’s Puzzles: The Concept, Tools, and First Observations. Journal of Information Technology Education: Innovations in Practice, 10, 1–14.

    Google Scholar 

  • Kalyuga, S., Chandler, P., & Sweller, J. (2000). Incorporating learner experience into the design of multimedia instruction. Journal of Educational Psychology, 92(1), 126–136.

    Google Scholar 

  • Kalyuga, S., Chandler, P., Tuovinen, J., & Sweller, J. (2001). When problem solving is superior to studying worked examples. Journal of Educational Psychology, 93(3), 579–588.

    Google Scholar 

  • Kalyuga, S., Ayres, P., Chandler, P., & Sweller, J. (2003). The expertise reversal effect. Educational Psychologist, 38(1), 23–31.

    Google Scholar 

  • Kay, R.H., & Knaack, L. (2009). Assessing learning, quality and engagement in learning objects: The learning object evaluation scale for students (loes-s). Educational Technology Research and Development, 57(2), 147–168.

    Google Scholar 

  • Khandwala, K., & Guo, P.J. (2018). Codemotion: expanding the design space of learner interactions with computer programming tutorial videos. In: Proceedings of the Fifth Annual ACM Conference on Learning at Scale, pp. 57, 1–57, vol. 10.

  • Lawrence, A.W. (1993). Empirical studies of the value of algorithm animation in algorithm understanding. PhD thesis: Georgia Institute of Technology.

    Google Scholar 

  • Lieberman, H. (1987). An example-based environment for beginning programmers. In Artificial intelligence and education (pp. 135–151). Norwood: Ablex Publishing.

  • Linn, MC., & Clancey, M. J. (1992). The case for case studies of programming problems. Communications of the ACM, 35(3), 121–132.

    Google Scholar 

  • Loboda, T., Guerra, J., Hosseini, R., & Brusilovsky, P. (2014). Mastery grids: An open source social educational progress visualization. In de Freitas, S, Rensing, C, Muñoz Merino, P.J., & Ley, T (Eds.) 9th European conference on technology enhanced learning (EC-TEL 2014), vol. 8719 of lecture notes in computer science (pp. 235–248).

  • Loboda, T.D., & Brusilovsky, P. (2010). User-adaptive explanatory program visualization: Evaluation and insights from eye movements. User Modeling and User-Adapted Interaction, 20(3), 191–226.

    Google Scholar 

  • McLaren, BM., Adams, D., Durkin, K., Goguadze, G., Mayer, RE., Rittle-Johnson, B., Sosnovsky, S., Isotani, S., & van Velsen, M. (2012). To err is human, to explain and correct is divine: A study of interactive erroneous examples with middle school math students. In: 7th European conference on technology enhanced learning (EC-TEL 2012), vol. 7563 of lecture notes in computer science, pp. 222–235.

  • Miller, B.N., & Ranum, D.L. (2012). Beyond pdf and epub: toward an interactive textbook. In: Proceedings of the 17th ACM annual conference on innovation and technology in computer science education, pp. 150–155. ACM.

  • Miyadera, Y., Kurasawa, K., Nakamura, S., Yonezawa, N., & Yokoyama, S. (2007). A real-time monitoring system for programming education using a generator of program animation systems. JCP, 2(3), 12–20.

    Google Scholar 

  • Morrison, B.B., Margulieux, L.E., Ericson, B., & Guzdial, M. (2016). Subgoals help students solve parsons problems. In: Proceedings of the 47th ACM Technical Symposium on Computing Science Education, pages 42–47 ACM.

  • Myller, N. (2006). Automatic prediction question generation during program visualization. In: Proceedings of the 4th program visualization workshop.

  • Najar, A.S., Mitrovic, A., & McLaren, B.M. (2016). Learning with intelligent tutors and worked examples: Selecting learning activities adaptively leads to better learning outcomes than a fixed curriculum. User Modeling and User-Adapted Interaction, 26(5), 459–491. ISSN 1573-1391.

    Google Scholar 

  • Naps, T.L. (2005). Jhavé: Supporting algorithm visualization. IEEE Computer Graphics and Applications, 25(5), 49–55.

    Google Scholar 

  • Naps, T.L, Eagan, J.R, & Norton, L.L. (2000). Jhavé – an environment to actively engage students in web-based algorithm visualizations. In: ACM SIGCSE bulletin, vol. 32, pp. 109–113. ACM.

  • Naps, TL., Rößling, G., Almstrum, V., Dann, W., Fleischer, R., Hundhausen, C., Korhonen, A., Malmi, L., McNally, M., Rodger, S., & Velázquez-Iturbide, J.Á. (2002). Exploring the role of visualization and engagement in computer science education. ACM SIGCSE bulletin, 35, 131–152.

    Google Scholar 

  • Nokes-Malach, T.J, VanLehn, K., Belenky, D.M., Lichtenstein, M., & Cox, G. (2013). Coordinating principles and examples through analogy and self-explanation. European Journal of Psychology of Education, 28(4), 1237–1263.

    Google Scholar 

  • Nunnally, J.C. (1978). Psychometric Ttheory: 2d Ed. McGraw-Hill.

  • Paas, F.G.W.C., & Van Merriënboer, J.J.G. (1994). Variability of worked examples and transfer of geometrical problem-solving skills: A cognitive-load approach. Journal of educational psychology, 86(1), 122–133.

    Google Scholar 

  • Park, J., Park, Y.H., Kim, J., Cha, J., Kim, S., & Alice, O.H. (2018). Elicast: embedding interactive exercises in instructional programming screencasts. In Proceedings of the 5th annual ACM conference on learning at scale, pp. 58, (Vol. 10 pp. 1–58).

  • Parsons, D., & Haden, P. (2006). Parson’s programming puzzles: a fun and effective learning tool for first programming courses. In: Proceedings of the 8th Australasian conference on computing Education-Volume 52, pp. 157–163. Australian Computer Society Inc.

  • Pirolli, P.L., & Anderson, J.R. (1985). The role of learning from examples in the acquisition of recursive programming skills. Canadian Journal of Psychology/Revue canadienne de psychologie, 39(2), 240–272.

    Google Scholar 

  • Reeve, J. (2013). How students create motivationally supportive learning environments for themselves: The concept of agentic engagement. Journal of Educational Psychology, 105(3), 579–595.

    MathSciNet  Google Scholar 

  • Reeve, J., & Tseng, C.-M. (2011). Agency as a fourth aspect of students’ engagement during learning activities. Contemporary Educational Psychology, 36(4), 257–267.

    Google Scholar 

  • Renkl, A. (1997). Learning from worked-out examples: A study on individual differences. Cognitive science, 21(1), 1–29.

    Google Scholar 

  • Renkl, A., & Atkinson, R. (2007). An example order for cognitive skill acquisition. Oxford University Press.

  • Resnick, M., Maloney, J., Monroy-Hernández, A., Rusk, N., Eastmond, E., Brennan, K., Millner, A., Rosenbaum, E., Silver, J., Silverman, B., & et al. (2009). Scratch: programming for all. Communications of the ACM, 52(11), 60–67.

    Google Scholar 

  • Rivers, K. (2017). Automated data-driven hint generation for learning programming. PhD thesis: Carnegie Mellon University.

    Google Scholar 

  • Sajaniemi, J., & Kuittinen, M. (2003). Program animation based on the roles of variables. In: Proceedings of the ACM symposium on Software visualization, pp. 7–ff. ACM.

  • Salden, R.J. C. M., Aleven, V., Schwonke, R., & Renkl, A. (2010a). The expertise reversal effect and worked examples in tutored problem solving. Instructional Science, 38(3), 289–307. ISSN 1573- 1952.

    Google Scholar 

  • Salden, R.J.C.M., Koedinger, K.R., Renkl, A., Aleven, V., & McLaren, B.M. (2010b). Accounting for beneficial effects of worked examples in tutored problem solving. Educational Psychology Review, 22(4), 379–392.

    Google Scholar 

  • Sears, A., & Wolfe, R. (1995). Visual analysis: Adding breadth to a computer graphics course. In: ACM SIGCSE Bulletin, vol. 27, pp. 195–198. ACM.

  • Sharrock, R., Hamonic, E., Hiron, M., & Carlier, S. (2017). Codecast: An innovative technology to facilitate teaching and learning computer programming in a c language online course. In: Proceedings of the 4th ACM conference on learning at scale, pp. 147–148. ACM.

  • Sirkiä, T. (2013). A javascript library for visualizing program execution. In: Proceedings of the 13th Koli calling international conference on computing education research, pp. 189–190. ACM.

  • Skinner, E.A., Kindermann, T.A., & Furrer, C.J. (2009). A motivational perspective on engagement and disaffection: Conceptualization and assessment of children’s behavioral and emotional participation in academic activities in the classroom. Educational and Psychological Measurement, 69(3), 493–525.

    MathSciNet  Google Scholar 

  • Sorva, J., Karavirta, V., & Malmi, L. (2013). A review of generic program visualization systems for introductory programming education. ACM Transactions on Computing Education (TOCE), 13(4), 15:1–15:64.

    Google Scholar 

  • Sweller, J., & Cooper, G.A. (1985). The use of worked examples as a substitute for problem solving in learning algebra. Cognition and instruction, 2(1), 59–89.

    Google Scholar 

  • Sweller, J., Van Merrienboer, J.J.G., & Paas, F.G.W.C. (1998). Cognitive architecture and instructional design. Educational psychology review, 10(3), 251–296.

    Google Scholar 

  • Trafton, J.G., & Reiser, B.J. (1993). The contributions of studying examples and solving problems to skill acquisition. In: Proceedings of the 15th annual conference of the cognitive science society, pp. 1017–1022 ACM.

  • Walker, C.O., Greene, B.A., & Mansell, R.A. (2006). Identification with academics, intrinsic/extrinsic motivation, and self-efficacy as predictors of cognitive engagement. Learning and individual differences, 16(1), 1–12.

    Google Scholar 

  • Ward, M., & Sweller, J. (1990). Structuring effective worked examples. Cognition and instruction, 7(1), 1–39.

    Google Scholar 

  • Weber, G. (1996). Individual selection of examples in an intelligent learning environment. Journal of Interactive Learning Research, 7(1), 3–31.

    Google Scholar 

  • Weber, G., & Brusilovsky, P. (2001). Elm-art: An adaptive versatile system for web-based instruction. International Journal of Artificial Intelligence in Education (IJAIED), 12, 351–384.

    Google Scholar 

  • Weber, G., & Mollenberg, A. (1994). Elm-pe: A knowledge-based programming environment for learning lisp. In: Proceedings of ED-MEDIA 1994, pp. 557–562 ERIC.

  • Yudelson, M., Brusilovsky, P., & Zadorozhny, V. (2007). A user modeling server for contemporary adaptive hypermedia: An evaluation of push approach to evidence propagation. In Conati, C., McCoy, K., & Paliouras, G. (Eds.) 11th International Conference on User Modeling, UM 2007, vol. 4511 of Lecture Notes in Computer Science, pp/ 27–36. Springer Verlag.

  • Zhi, R., Price, T.W., Marwan, S., Milliken, A., Barnes, T., & Chi, M. (2019). Exploring the impact of worked examples in a novice programming environment. In: Proceedings of the 50th ACM technical symposium on computer science education, SIGCSE ’19, pp. 98–104. ACM.

Download references

Author information

Authors and Affiliations

  1. University of Pittsburgh, Pittsburgh, PA, USA

    Roya Hosseini, Kamil Akhuseyinoglu, Peter Brusilovsky & Christian Schunn

  2. Aalto University, Espoo, Finland

    Lauri Malmi, Kerttu Pollari-Malmi & Teemu Sirkiä

Authors
  1. Roya Hosseini
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kamil Akhuseyinoglu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Peter Brusilovsky
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lauri Malmi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kerttu Pollari-Malmi
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Christian Schunn
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Teemu Sirkiä
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Roya Hosseini.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hosseini, R., Akhuseyinoglu, K., Brusilovsky, P. et al. Improving Engagement in Program Construction Examples for Learning Python Programming. Int J Artif Intell Educ 30, 299–336 (2020). https://doi.org/10.1007/s40593-020-00197-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40593-020-00197-0

Keywords

Search

Navigation