Wengran Wang
Ge Gao
Thomas W. Price
ABSTRACT
Students often get stuck when programming independently, and need help to progress. Existing, automated feedback can help students progress, but it is unclear whether it ultimately leads to learning. We present Step Tutor, which helps struggling students during programming by presenting them with relevant, step-by-step examples. The goal of Step Tutor is to help students progress, and engage them in comparison, reflection, and learning. When a student requests help, Step Tutor adaptively selects an example to demonstrate the next meaningful step in the solution. It engages the student in comparing "before" and "after" code snapshots, and their corresponding visual output, and guides them to reflect on the changes. Step Tutor is a novel form of help that combines effective aspects of existing support features, such as hints and Worked Examples, to help students both progress and learn. To understand how students use Step Tutor, we asked nine undergraduate students to complete two programming tasks, with its help, and interviewed them about their experience. We present our qualitative analysis of students' experience, which shows us why and how they seek help from Step Tutor, and Step Tutor's affordances. These initial results suggest that students perceived that Step Tutor accomplished its goals of helping them to progress and learn.
- Vincent Aleven. 2013. Help seeking and intelligent tutoring systems: Theoretical perspectives and a step towards theoretical integration. In International handbook of metacognition and learning technologies. Springer, 311--335.Google Scholar
- Vincent Aleven, Ido Roll, Bruce M McLaren, and Kenneth R Koedinger. 2016. Help helps, but only so much: Research on help seeking with intelligent tutoring systems. International Journal of Artificial Intelligence in Education, Vol. 26, 1 (2016), 205--223.Google ScholarCross Ref
- Vincent Aleven, Elmar Stahl, Silke Schworm, Frank Fischer, and Raven Wallace. 2003. Help seeking and help design in interactive learning environments. Review of educational research, Vol. 73, 3, 277--320.Google Scholar
- Computing Research Association et al. 2017. Generation CS: Computer science undergraduate enrollments surge since 2006. Retrieved March, Vol. 20 (2017), 2017.Google Scholar
- Robert K Atkinson, Alexander Renkl, and Mary Margaret Merrill. 2003. Transitioning from studying examples to solving problems: Effects of self-explanation prompts and fading worked-out steps. Journal of educational psychology, Vol. 95, 4 (2003), 774.Google ScholarCross Ref
- Albert Bandura. 1977. Self-efficacy: toward a unifying theory of behavioral change. Psychological review, Vol. 84, 2 (1977), 191.Google Scholar
- Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology, Vol. 3, 2 (2006), 77--101.Google Scholar
- Peter Brusilovsky. 2001. WebEx: Learning from Examples in a Programming Course.. In WebNet, Vol. 1. 124--129.Google Scholar
- Ruth Butler. 1998. Determinants of help seeking: Relations between perceived reasons for classroom help-avoidance and help-seeking behaviors in an experimental context. Journal of Educational Psychology, Vol. 90, 4 (1998), 630.Google ScholarCross Ref
- Michelene TH Chi and Ruth Wylie. 2014. The ICAP framework: Linking cognitive engagement to active learning outcomes. Educational psychologist, Vol. 49, 4 (2014), 219--243.Google Scholar
- Ruth C Clark, Frank Nguyen, and John Sweller. 2011. Efficiency in learning: Evidence-based guidelines to manage cognitive load .John Wiley & Sons.Google Scholar
- Jarno Coenen, Sebastian Gross, and Niels Pinkwart. 2017. Comparison of Feedback Strategies for Supporting Programming Learning in Integrated Development Environments (IDEs). In International Conference on Computer Science, Applied Mathematics and Applications. Springer, 72--83.Google Scholar
- Allan Collins, John Seely Brown, and Susan E Newman. 1988. Cognitive apprenticeship: Teaching the craft of reading, writing and mathematics. Thinking: The Journal of Philosophy for Children, Vol. 8, 1 (1988), 2--10.Google ScholarCross Ref
- Albert T Corbett and John R Anderson. 2001. Locus of feedback control in computer-based tutoring: Impact on learning rate, achievement and attitudes. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 245--252.Google ScholarDigital Library
- Bob Edmison, Stephen H. Edwards, and Manuel A. Pérez-Qui nones. 2017. Using Spectrum-Based Fault Location and Heatmaps to Express Debugging Suggestions to Student Programmers (ACE '17). Association for Computing Machinery, New York, NY, USA, 48--54.Google Scholar
- Dan Garcia, Brian Harvey, and Tiffany Barnes. 2015. The beauty and joy of computing. ACM Inroads, Vol. 6, 4 (2015), 71--79.Google ScholarDigital Library
- Dedre Gentner, Jeffrey Loewenstein, and Leigh Thompson. 2003. Learning and transfer: A general role for analogical encoding. Journal of Educational Psychology, Vol. 95, 2 (2003), 393.Google ScholarCross Ref
- Peter Gerjets, Katharina Scheiter, and Richard Catrambone. 2004. Designing instructional examples to reduce intrinsic cognitive load: Molar versus modular presentation of solution procedures. Instructional Science, Vol. 32, 1--2 (2004), 33--58.Google ScholarCross Ref
- Mary L Gick and Keith J Holyoak. 1983. Schema induction and analogical transfer. Cognitive psychology, Vol. 15, 1 (1983), 1--38.Google Scholar
- Sebastian Gross, Bassam Mokbel, Benjamin Paassen, Barbara Hammer, and Niels Pinkwart. 2014. Example-based feedback provision using structured solution spaces. International Journal of Learning Technology 10, Vol. 9, 3 (2014), 248--280.Google Scholar
- Sebastian Gross and Niels Pinkwart. 2015. Towards an integrative learning environment for java programming. In 2015 IEEE 15th International Conference on Advanced Learning Technologies. IEEE, 24--28.Google ScholarDigital Library
- Luke Gusukuma, Austin Cory Bart, Dennis Kafura, and Jeremy Ernst. 2018. Misconception-driven feedback: Results from an experimental study. In Proceedings of the 2018 ACM Conference on International Computing Education Research. ACM, 160--168.Google ScholarDigital Library
- Stuart Hansen and Erica Eddy. 2007. Engagement and frustration in programming projects. ACM SIGCSE Bulletin, Vol. 39, 1 (2007), 271--275.Google ScholarDigital Library
- Gregory Hume, Joel Michael, Allen Rovick, and Martha Evens. 1996. Hinting as a tactic in one-on-one tutoring. The Journal of the Learning Sciences, Vol. 5, 1 (1996), 23--47.Google ScholarCross Ref
- Michelle Ichinco, Kyle J Harms, and Caitlin Kelleher. 2017. Towards Understanding Successful Novice Example User in Blocks-Based Programming. Journal of Visual Languages and Sentient Systems, Vol. 3 (2017), 101--118.Google ScholarCross Ref
- Michelle Ichinco and Caitlin Kelleher. 2015. Exploring novice programmer example use. In 2015 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). IEEE, 63--71.Google ScholarCross Ref
- Slava Kalyuga. 2009. The expertise reversal effect. IGI Global.Google Scholar
- Stuart A Karabenick. 2004. Perceived achievement goal structure and college student help seeking. Journal of educational psychology, Vol. 96, 3 (2004), 569.Google ScholarCross Ref
- Andrew J Ko, Brad A Myers, and Htet Htet Aung. 2004. Six learning barriers in end-user programming systems. In 2004 IEEE Symposium on Visual Languages-Human Centric Computing. IEEE, 199--206.Google ScholarDigital Library
- Kenneth R Koedinger and Vincent Aleven. 2007. Exploring the assistance dilemma in experiments with cognitive tutors. Educational Psychology Review, Vol. 19, 3 (2007), 239--264.Google ScholarCross Ref
- Kenneth R Koedinger, Albert T Corbett, and Charles Perfetti. 2012. The Knowledge-Learning-Instruction framework: Bridging the science-practice chasm to enhance robust student learning. Cognitive science, Vol. 36, 5 (2012), 757--798.Google Scholar
- Hannu Kuusela and Paul Pallab. 2000. A comparison of concurrent and retrospective verbal protocol analysis. The American journal of psychology, Vol. 113, 3 (2000), 387.Google Scholar
- Samiha Marwan, Joseph Jay Williams, and Thomas Price. 2019 a. An Evaluation of the Impact of Automated Programming Hints on Performance and Learning. In Proceedings of the 2019 ACM Conference on International Computing Education Research. 61--70.Google ScholarDigital Library
- Samiha Marwan, Nicholas Lytle, Joseph Jay Williams, and Thomas Price. 2019 b. The Impact of Adding Textual Explanations to Next-step Hints in a Novice Programming Environment. In Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science Education. 520--526.Google ScholarDigital Library
- Elizabeth Patitsas, Michelle Craig, and Steve Easterbrook. 2013. Comparing and contrasting different algorithms leads to increased student learning. In Proceedings of the ninth annual international ACM conference on International computing education research. ACM, 145--152.Google ScholarDigital Library
- Thomas Price, Rui Zhi, and Tiffany Barnes. 2017c. Evaluation of a Data-driven Feedback Algorithm for Open-ended Programming. International Educational Data Mining Society (2017).Google Scholar
- Thomas W Price, Yihuan Dong, and Dragan Lipovac. 2017a. iSnap: towards intelligent tutoring in novice programming environments. In Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education (SIGCSE '17). ACM, 483--488.Google ScholarDigital Library
- Thomas W Price, Zhongxiu Liu, Veronica Cateté, and Tiffany Barnes. 2017b. Factors Influencing Students' Help-Seeking Behavior while Programming with Human and Computer Tutors. In Proceedings of the 2017 ACM Conference on International Computing Education Research. ACM, 127--135.Google ScholarDigital Library
- Thomas W Price, Rui Zhi, and Tiffany Barnes. 2017 d. Hint generation under uncertainty: The effect of hint quality on help-seeking behavior. In International Conference on Artificial Intelligence in Education. Springer, 311--322.Google ScholarCross Ref
- Bethany Rittle-Johnson and Jon R Star. 2007. Does comparing solution methods facilitate conceptual and procedural knowledge? An experimental study on learning to solve equations. Journal of Educational Psychology, Vol. 99, 3 (2007), 561.Google ScholarCross Ref
- Kelly Rivers. 2017. Automated Data-Driven Hint Generation for Learning Programming. (2017).Google Scholar
- Ido Roll, Ryan SJ d Baker, Vincent Aleven, and Kenneth R Koedinger. 2014. On the benefits of seeking (and avoiding) help in online problem-solving environments. Journal of the Learning Sciences, Vol. 23, 4 (2014), 537--560.Google ScholarCross Ref
- Donald A Schön. 1987. Teaching artistry through reflection in action: Educating the reflective practitioner. (1987).Google Scholar
- Benjamin Shih, Kenneth R Koedinger, and Richard Scheines. 2011. A response time model for bottom-out hints as worked examples. Handbook of educational data mining (2011), 201--212.Google Scholar
- John Sweller. 1988. Cognitive load during problem solving: Effects on learning. Cognitive science, Vol. 12, 2 (1988), 257--285.Google Scholar
- John Sweller. 2006. The worked example effect and human cognition. Learning and instruction (2006).Google Scholar
- W. Price Thomas, Jay Williams Joseph, Solyst Jaemarie, and Marwan Samiha. 2020. Engaging Students with Instructor Solutions in Online Programming Homework. In To be published in the 2020 Association for Computing Machinery's Special Interest Group on Computer Human Interaction (ACM SIGCHI ?20) .Google Scholar
- John Gregory Trafton and Brian J Reiser. 1994. The contributions of studying examples and solving problems to skill acquisition. Ph.D. Dissertation. Citeseer.Google Scholar
- Lev Vygotsky. 1978. Interaction between learning and development. Readings on the development of children, Vol. 23, 3 (1978), 34--41.Google Scholar
- Wengran Wang, Rui Zhi, Alexandra Milliken, Nicholas Lytle, and Thomas W. Price. 2020. Crescendo: Engaging Students to Self-Paced Programming Practices (SIGCSE '20). ACM, 859--865.Google Scholar
- Rui Zhi, Samiha Marwan, Yihuan Dong, Nicholas Lytle, Thomas W Price, and Tiffany Barnes. 2019 a. Toward Data-Driven Example Feedback for Novice Programming. Proceedings of the International Conference on Educational Data Mining (2019), 218--227.Google Scholar
- Rui Zhi, Thomas W Price, Nicholas Lytle, Yihuan Dong, and Tiffany Barnes. 2018. Reducing the State Space of Programming Problems through Data-Driven Feature Detection. In Educational Data Mining in Computer Science Education (CSEDM) Workshop@ EDM .Google Scholar
- Rui Zhi, Thomas W Price, Samiha Marwan, Alexandra Milliken, Tiffany Barnes, and Min Chi. 2019 b. Exploring the Impact of Worked Examples in a Novice Programming Environment. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education. ACM, 98--104.Google ScholarDigital Library
Index Terms
- Step Tutor: Supporting Students through Step-by-Step Example-Based Feedback
Recommendations
Investigating the Impact of Unsolicited Next-Step and Subgoal Hints on Dropout in a Logic Proof Tutor (Abstract Only)
SIGCSE '17: Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science EducationWe have been incrementally adding data-driven methods into the Deep Thought logic tutor for the purpose of creating a fully data-driven intelligent tutoring system. Our previous research has shown that the addition of data-driven hints, worked examples, ...
Next-Step Hint Generation for Introductory Programming Using Large Language Models
ACE '24: Proceedings of the 26th Australasian Computing Education ConferenceLarge Language Models possess skills such as answering questions, writing essays or solving programming exercises. Since these models are easily accessible, researchers have investigated their capabilities and risks for programming education. This work ...
When Is Tutorial Dialogue More Effective Than Step-Based Tutoring?
ITS 2014: 12th International Conference on Intelligent Tutoring Systems - Volume 8474It is often assumed that one-on-one dialogue with a tutor, which involves micro-steps, is more effective than conventional step-based tutoring. Although earlier research often has not supported this hypothesis, it may be because tutors often are not ...
Comments