skip to main content
10.1145/3341525.3387394acmconferencesArticle/Chapter ViewAbstractPublication PagesiticseConference Proceedingsconference-collections
research-article

Unproductive Help-seeking in Programming: What it is and How to Address it

Published: 15 June 2020 Publication History

Abstract

While programming, novices often lack the ability to effectively seek help, such as when to ask for a hint or feedback. Students may avoid help when they need it, or abuse help to avoid putting in effort, and both behaviors can impede learning. In this paper we present two main contributions. First, we investigated log data from students working in a programming environment that offers automated hints, and we propose a taxonomy of unproductive help-seeking behaviors in programming. Second, we used these findings to design a novel user interface for hints that subtly encourages students to seek help with the right frequency, estimated with a data-driven algorithm. We conducted a pilot study to evaluate our data-driven (DD) hint display, compared to a traditional interface, where students request hints on-demand as desired. We found students with the DD display were less than half as likely to engage in unproductive help-seeking, and we found suggestive evidence that this may improve their learning.

References

[1]
Vincent Aleven and Kenneth R Koedinger. 2001. Investigations into help seeking and learning with a cognitive tutor. In Papers of the AIED-2001 workshop on help provision and help seeking in interactive learning environments. 47--58.
[2]
Vincent Aleven, Bruce Mclaren, Ido Roll, and Kenneth Koedinger. 2006. Toward meta-cognitive tutoring: A model of help seeking with a Cognitive Tutor. International Journal of Artificial Intelligence in Education 16, 2 (2006), 101--128.
[3]
Vincent Aleven, Ido Roll, Bruce M McLaren, and Kenneth R Koedinger. 2016. Help helps, but only so much: Research on help seeking with intelligent tutoring systems. International Journal of Artificial Intelligence in Education 26, 1 (2016),205--223.
[4]
Vincent Aleven, Elmar Stahl, Silke Schworm, Frank Fischer, and Raven Wallace. 2003. Help seeking and help design in interactive learning environments. Review of educational research 73, 3 (2003), 277--320.
[5]
Computing Research Association et al. 2017. Generation CS: Computer Science Undergraduate Enrollments Surge Since 2006.(2017).
[6]
Ryan Shaun Baker, Albert T Corbett, and Kenneth R Koedinger. 2004. Detecting student misuse of intelligent tutoring systems. In International conference on intelligent tutoring systems. Springer, 531--540.
[7]
Michael Ball. 2018.Lambda: An Autograder forsnap. Technical Report. Electrical Engineering and Computer Sciences University of California at Berkeley.
[8]
Brett A Becker, Graham Glanville, Ricardo Iwashima, Claire McDonnell, Kyle Goslin, and Catherine Mooney. 2016. Effective compiler error message enhancement for novice programming students. Computer Science Education 26, 2--3(2016), 148--175.
[9]
Susan Bergin, Ronan Reilly, and Desmond Traynor. 2005. Examining the role of self-regulated learning on introductory programming performance. In Proceedings of the first international workshop on Computing education research. ACM, 81--86.
[10]
Piroska Biró and Maria Csernoch. 2014. Deep and surface metacognitive processes in non-traditional programming tasks. In2014 5th IEEE Conference on Cognitive Info communications (CogInfoCom). IEEE, 49--54.
[11]
Albert T Corbett and John R Anderson. 2001. Locus of feedback control in computer-based tutoring: Impact on learning rate, achievement and attitudes. In Proceedings of the SIGCHI conference on Human factors in computing systems.ACM, 245--252.
[12]
National Research Council et al. 2000. How people learn: Brain, mind, experience,and school: Expanded edition. National Academies Press.
[13]
Ryan SJ d Baker, Sujith M Gowda, and Albert T Corbett. 2011. Towards predicting future transfer of learning. In International Conference on Artificial Intelligence in Education. Springer, 23--30.
[14]
Yihuan Dong, Samiha Marwan, Veronica Catete, Thomas W. Price, and Tiffany Barnes. 2019. Defining Tinkering Behavior in Open-ended Block-based Programming Assignments. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education. ACM, 1204--1210.
[15]
Davide Fossati, Barbara Di Eugenio, STELLAN Ohlsson, Christopher Brown, and Lin Chen. 2015. Data driven automatic feedback generation in the iList intelligent tutoring system. Technology, Instruction, Cognition and Learning 10, 1 (2015),5--26.
[16]
Dan Garcia, Brian Harvey, and Tiffany Barnes. 2015. The beauty and joy of computing. ACM Inroads 6, 4 (2015), 71--79.
[17]
Luke Gusukuma, Austin Cory Bart, Dennis Kafura, and Jeremy Ernst. 2018. Misconception-Driven Feedback. Proceedings of the 2018 ACM Conference on International Computing Education Research - ICER '181 (2018), 160--168.
[18]
David E Johnson. 2016. ITCH: Individual Testing of Computer Homework for Scratch Assignments. InProceedings of the 47th ACM Technical Symposium onComputing Science Education. ACM, 223--227.
[19]
Stuart A Karabenick and Richard S Newman. 2009. Seeking help: Generalizableself-regulatory process and social-cultural barometer. (2009).
[20]
Samad Kardan and Cristina Conati. 2015. Providing adaptive support in aninteractive simulation for learning: An experimental evaluation. InProceedings ofthe 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM,3671--3680.
[21]
Andrew J Ko, Brad A Myers, and Htet Htet Aung. 2004. Six learning barriers inend-user programming systems. In 2004 IEEE Symposium on Visual Languages-Human Centric Computing. IEEE, 199--206.
[22]
Samiha Marwan, Joseph Jay Williams, and Thomas W. Price. 2019. An Evaluationof the Impact of Automated Programming Hints on Performance and Learning. In Proceedings of the 2019 ACM Conference on International Computing Education Research. ACM, 61--70.
[23]
Samiha Marwan, Nicholas Lytle, Joseph Jay Williams, and Thomas W. Price. 2019. The Impact of Adding Textual Explanations to Next-step Hints in a Novice Programming Environment. In Proceedings of the 2019 ACM Conference on Innovationand Technology in Computer Science Education. ACM, 520--526.
[24]
Antonija Mitrovic, Stellan Ohlsson, and Devon K Barrow. 2013. The effect of positive feedback in a constraint-based intelligent tutoring system. Computers & Education 60, 1 (2013), 264--272.
[25]
Kazuhisa Miwa, Hitoshi Terai, Nana Kanzaki, and Ryuichi Nakaike. 2013. Stoic Behavior in Hint Seeking when Learning using an Intelligent Tutoring System. In Proceedings of the Annual Meeting of the Cognitive Science Society, Vol. 35.
[26]
R Charles Murray and Kurt Van Lehn. 2005. Effects of Dissuading Unnecessary Help Requests While Providing Proactive Help. In AIED. Citeseer, 887--889.
[27]
R Charles Murray and Kurt Van Lehn. 2006. A comparison of decision-theoretic, fixed-policy and random tutorial action selection. In International Conference on Intelligent Tutoring Systems. Springer, 114--123.
[28]
Mitchell J Nathan, Kenneth R Koedinger, Martha W Alibali, et al. 2001. Expert blind spot: When content knowledge eclipses pedagogical content knowledge. In Proceedings of the third international conference on cognitive science, Vol. 644648.
[29]
Sharon Nelson-Le Gall. 1981. Help-seeking: An understudied problem-solving skill in children. Developmental Review 1, 3 (1981), 224--246.
[30]
Sharon Nelson-Le Gall. 1986. Help-seeking behavior in learning. ERIC Clearing-house.
[31]
Richard S Newman. 1994. Adaptive help seeking: A strategy of self-regulated learning. (1994).
[32]
Stellan Ohlosson, Barbara Di Eugenio, Bettina Chow, Davide Fossati, Xin Lu,and Trina C. Kershaw. 2007. Beyond the code-and-count analysis of tutoring dialogues. Artificial intelligence in education: Building technology rich learning contexts that work, R. Luckin, KR Koedinger, and J. Greer, Eds. IOS Press(2007),349--356.
[33]
Thomas W. Price, Yihuan Dong, and Dragan Lipovac. 2017. iSnap: Towards Intelligent Tutoring in Novice Programming Environments. In Proceedings of the ACM Technical Symposium on Computer Science Education.
[34]
Thomas W. Price, Zhongxiu Liu, Veronica Cateté, and Tiffany Barnes. 2017. Factors Influencing Students' Help-Seeking Behavior while Programming with Human and Computer Tutors. In Proceedings of the 2017 ACM Conference on International Computing Education Research. ACM, 127--135.
[35]
Minna Puustinen. 1998. Help-seeking behavior in a problem-solving situation: Development of self-regulation. European Journal of Psychology of education 13,2 (1998), 271.
[36]
Leena Razzaq and Neil T Heffernan. 2010. Hints: is it better to give or wait tobe asked?. In International Conference on Intelligent Tutoring Systems. Springer, 349--358.
[37]
Leena Razzaq, Neil T Heffernan, and RW Lindeman. 2007. What level of tutor feedback is best. In Proceedings of the 13th Conference on Artificial Intelligence in Education, IOS Press.
[38]
A Renkl. 2002. Learning from Worked-out Examples: Instructional Explanations Supplement Self Explanations. Learning and Instruction 12 (2002), 149--176.
[39]
Kelly Rivers and Kenneth R Koedinger. 2017. Data-driven hint generation in vast solution spaces: a self-improving python programming tutor. International Journal of Artificial Intelligence in Education 27, 1 (2017), 37--64.
[40]
Ido Roll, Ryan SJ d Baker, Vincent Aleven, and Kenneth R Koedinger. 2014. On the benefits of seeking (and avoiding) help in online problem-solving environments. Journal of the Learning Sciences 23, 4 (2014), 537--560.
[41]
Marieke Thurlings, Marjan Vermeulen, Theo Bastiaens, and Sjef Stijnen. 2013. Understanding feedback: A learning theory perspective. Educational Research Review 9 (2013), 1--15.
[42]
Bram E Vaessen, Frans J Prins, and Johan Jeuring. 2014. University students' achievement goals and help-seeking strategies in an intelligent tutoring system. Computers & Education 72 (2014), 196--208.
[43]
Kurt Vanlehn, Collin Lynch, Kay Schulze, Joel A Shapiro, Robert Shelby, Linwood Taylor, Don Treacy, Anders Weinstein, and Mary Wintersgill. 2005. The Andesphysics tutoring system: Lessons learned. International Journal of Artificial Intelligence in Education 15, 3 (2005), 147--204.
[44]
Wengran Wang, Rui Zhi, Alexandra Milliken, Nicholas Lytle, and Thomas W.Price. 2020. Crescendo: Engaging Students to Self-Paced Programming Practices. In To be published in the 51st ACM Technical Symposium on Computer Science Education (SIGCSE '20).
[45]
H Wood and D Wood. 1999. Help seeking, learning and contingent tutoring. Computers & Education 33, 2--3 (1999), 153--169.

Cited By

View all
  • (2024)Exploring Debugging Challenges and Strategies Using Structural Topic Model: A Comparative Analysis of High and Low-Performing StudentsJournal of Educational Computing Research10.1177/0735633124129117462:8(2104-2126)Online publication date: 13-Oct-2024
  • (2024)Exploring Novices' Problem-Solving Strategies in Computing and Math DomainsProceedings of the 24th Koli Calling International Conference on Computing Education Research10.1145/3699538.3699557(1-8)Online publication date: 12-Nov-2024
  • (2024)Debugging with an AI Tutor: Investigating Novice Help-seeking Behaviors and Perceived LearningProceedings of the 2024 ACM Conference on International Computing Education Research - Volume 110.1145/3632620.3671092(84-94)Online publication date: 12-Aug-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ITiCSE '20: Proceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science Education
June 2020
615 pages
ISBN:9781450368742
DOI:10.1145/3341525
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 June 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. adaptive hints
  2. block-based programming
  3. help-seeking

Qualifiers

  • Research-article

Conference

ITiCSE '20
Sponsor:

Acceptance Rates

Overall Acceptance Rate 552 of 1,613 submissions, 34%

Upcoming Conference

ITiCSE '25
Innovation and Technology in Computer Science Education
June 27 - July 2, 2025
Nijmegen , Netherlands

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)89
  • Downloads (Last 6 weeks)15
Reflects downloads up to 20 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Exploring Debugging Challenges and Strategies Using Structural Topic Model: A Comparative Analysis of High and Low-Performing StudentsJournal of Educational Computing Research10.1177/0735633124129117462:8(2104-2126)Online publication date: 13-Oct-2024
  • (2024)Exploring Novices' Problem-Solving Strategies in Computing and Math DomainsProceedings of the 24th Koli Calling International Conference on Computing Education Research10.1145/3699538.3699557(1-8)Online publication date: 12-Nov-2024
  • (2024)Debugging with an AI Tutor: Investigating Novice Help-seeking Behaviors and Perceived LearningProceedings of the 2024 ACM Conference on International Computing Education Research - Volume 110.1145/3632620.3671092(84-94)Online publication date: 12-Aug-2024
  • (2024)Learning Agent-based Modeling with LLM Companions: Experiences of Novices and Experts Using ChatGPT & NetLogo ChatProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642377(1-18)Online publication date: 11-May-2024
  • (2024)Identifying critical self-regulated learning skills: a Delphi process studyComputer Science Education10.1080/08993408.2024.2382631(1-28)Online publication date: 27-Oct-2024
  • (2024)Automated Detection and Analysis of Gaming the System in Novice ProgrammersArtificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium and Blue Sky10.1007/978-3-031-64315-6_30(338-346)Online publication date: 2-Jul-2024
  • (2023)Factors Influencing the Social Help-seeking Behavior of Introductory Programming Students in a Competitive University EnvironmentACM Transactions on Computing Education10.1145/363905924:1(1-27)Online publication date: 30-Dec-2023
  • (2023)Understanding the Effects of Using Parsons Problems to Scaffold Code Writing for Students with Varying CS Self-Efficacy LevelsProceedings of the 23rd Koli Calling International Conference on Computing Education Research10.1145/3631802.3631832(1-12)Online publication date: 13-Nov-2023
  • (2023)How Novices Use LLM-based Code Generators to Solve CS1 Coding Tasks in a Self-Paced Learning EnvironmentProceedings of the 23rd Koli Calling International Conference on Computing Education Research10.1145/3631802.3631806(1-12)Online publication date: 13-Nov-2023
  • (2023)Impact of Hint Content on Performance and Learning: A Study with Primary School Children in a Scratch CourseProceedings of the 18th WiPSCE Conference on Primary and Secondary Computing Education Research10.1145/3605468.3605498(1-10)Online publication date: 27-Sep-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media