Abstract
In this paper we investigate whether providing feedback to learners within an Integrated Development Environment (IDE) helps them write correct programs. Here, we use two approaches: feedback based on stack trace analysis, and feedback based on structural comparisons of a learner program and appropriate sample programs. In order to investigate both approaches, we developed two prototypical extensions for the Eclipse IDE. In a laboratory study, we empirically evaluated the impact of the extensions on learners’ performance while they solved programming tasks. The statistical analyses did not reveal any statistically significant effects of the prototype extensions on the performance of the learners, however, the results of a qualitative analysis imply that the provided feedback had at least a marginal impact on the performance of some learners. Also, feedback from the participants confirmed the benefit of providing feedback directly within IDEs.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Processing is both a Java based IDE, as well as a kind of simplified Java programming language for educational purposes. The project website is available at http://www.processing.org/.
- 2.
NetBeans is an open source IDE supporting, amongst others, Java (see https://netbeans.org/).
- 3.
IntelliJ IDEA is an Apache 2 licensed IDE for non-commercial use supporting, amongst others, Java (see https://www.jetbrains.com/idea/).
- 4.
In this survey in 2016, Eclipse is ranked second to IntelliJ for the first time in their survey, however only by a marginal 5%.
- 5.
For more details, see the homepage of the toolbox, available here: https://openresearch.cit-ec.de/projects/tcs.
References
European Commission: Coding - the 21st century skill 6 July 2016. https://ec.europa.eu/digital-single-market/coding-21st-century-skill. Accessed 4 Nov 2016
Balanskat, A., Engelhardt, K.: Computing Our Future - Computer Programming and Coding - Priorities, School Curricula and Initiatives Across Europe. European Schoolnet, Brussels (2015)
The White House - Office of the Press Secretary: Remarks of President Barack Obama – State of the Union Address as Delivered, 13 January 2016. https://www.whitehouse.gov/the-press-office/2016/01/12/remarks-president-barack-obama-%E2%80%93-prepared-delivery-state-union-address. Accessed 04 Nov 2016
Zweben, S., Bizot, B.: 2015 taulbee survey: continued booming undergraduate CS enrollment; doctoral degree production dips slightly. Comput. Res. News 28(5), 2–60 (2016)
Self, J.: The defining characteristics of intelligent tutoring systems research: ITS care, precisely. Int. J. Artif. Intell. Educ. (IJAIED) 10, 350–364 (1998)
Nienaltowski, M., Pedroni, M., Meyer, B.: Compiler error messages: what can help novices? In: Proceedings of the 39th SIGCSE Technical Symposium on Computer Science Education, pp. 168–172 (2008)
Marceau, G., Fisler, K., Krishnamurthi, S.: Mind your language: on novices’ interactions with error messages. In: Proceedings of the 10th SIGPLAN Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software (Onward! 2011), pp. 3–18 (2011)
Hristova, M., Misra, A., Rutter, M., Mercuri, R.: Identifying and correcting Java programming errors for introductory computer science students. In: Proceedings of the 34th SIGCSE Technical Symposium on Computer Science Education (SIGCSE 2003), pp. 153–156 (2003)
Murphy, C., Kim, E., Kaiser, G., Cannon, A.: Backstop: a tool for debugging runtime errors. In: Proceedings of the 39th SIGCSE Technical Symposium on Computer Science Education (SIGCSE 2008), pp. 173–177 (2008)
Hartmann, B., MacDougall, D., Brandt, J., Klemmer, S.: What would other programmers do: suggesting solutions to error messages. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2010), pp. 1019–1028 (2010)
Flowers, T., Carver, C.A., Jackson, J.: Empowering students and building confidence in novice programmers through Gauntlet. In: 34th Annual Frontiers in Education, FIE 2004, pp. T3H10–T3H13 (2004)
Holland, J., Mitrovic, A., Martin, B.: J-LATTE: a constraint-based tutor for Java. In: Proceedings of the 17th International Conference on Computers in Education, pp. 142–146 (2009)
Sykes, E.: Design, development and evaluation of the Java intelligent tutoring system. Tech. Inst. Cogn. Learn. 8, 25–65 (2010)
Abu-Naser, S., Ahmed, A., Al-Masri, N., Deeb, A., Moshtaha, E., Abu-Lamdy, M.: An intelligent tutoring system for learning Java objects. Int. J. Artif. Intell. Appl. (IJAIA) 2(2), 68–77 (2011)
Codeanywhere Inc.: Most Popular Desktop IDEs & Code Editors in 2014, 13 January 2015. https://blog.codeanywhere.com/most-popular-ides-code-editors/. Accessed 04 Nov 2016
Maple, S.: Java Tools and Technologies Landscape Report 2016, 14 July 2016. http://zeroturnaround.com/rebellabs/java-tools-and-technologies-landscape-2016/. Accessed 4 Nov 2016
Biradar, M.: Popularity of Programming Languages, 28 July 2015. https://maheshbiradar.wordpress.com/2015/07/28/popularity-of-programming-language/. Accessed 4 Nov 2016
Stack Exchange Inc.: Developer Survey Results 2016, March 2016. http://stackoverflow.com/research/developer-survey-2016#technology-development-environments. Accessed 5 Nov 2016
Jackson, J., Cobb, M., Carver, C.: Identifying top Java errors for novice programmers. In: Proceedings - Frontiers in Education Conference, p. T4C (2005)
Jadud, J.: A first look at novice compilation behaviour using BlueJ. Comput. Sci. Educ. 15, 25–40 (2005)
Altadmri, A., Brown, N.: 37 million compilations: investigating novice programming mistakes in large-scale student data. In: Proceedings of the 46th ACM Technical Symposium on Computer Science Education, pp. 522–527 (2015)
Tabanao, E., Rodrigo, M., Jadud, M.: Predicting at-risk novice Java programmers through the analysis of online protocols. In: Proceedings of the Seventh International Workshop on Computing Education Research (ICER 2011), pp. 85–92 (2011)
McCall, D., Kölling, M.: Meaningful categorisation of novice programmer errors. In: Frontiers in Education Conference, pp. 2589–2596 (2014)
Becker, B.: An effective approach to enhancing compiler error messages. In: Proceedings of the 47th ACM Technical Symposium on Computing Science Education (SIGCSE 2016), pp. 126–131 (2016)
Denny, P., Luxton-Reilly, A., Tempero, E.: All syntax errors are not equal. In: Proceedings of the 17th ACM Annual Conference on Innovation and Technology in Computer Science Education (ITiCSE 2012), pp. 75–80 (2012)
Gross, S., Mokbel, B., Hammer, B., Pinkwart, N.: Learning feedback in intelligent tutoring systems. KI - Künstliche Intell. 29(4), 413–418 (2015)
Gross, S., Pinkwart, N.: How do learners behave in help-seeking when given a choice? Int. J. Artif. Intell. Educ. 9112, 600–603 (2015)
Mokbel, B., Gross, S., Paassen, B., Pinkwart, N., Hammer, B.: Domain-independent proximity measures in intelligent tutoring systems. In: Proceedings of the 6th International Conference on Educational Data Mining (EDM), pp. 334–335 (2013)
Acknowledgement
This work was supported by the Deutsche Forschungsgemeinschaft (DFG) under the grant “DynaFIT – Learning Dynamic Feedback in Intelligent Tutoring Systems” (PI 764/6-2).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this paper
Cite this paper
Coenen, J., Gross, S., Pinkwart, N. (2018). Comparison of Feedback Strategies for Supporting Programming Learning in Integrated Development Environments (IDEs). In: Le, NT., van Do, T., Nguyen, N., Thi, H. (eds) Advanced Computational Methods for Knowledge Engineering. ICCSAMA 2017. Advances in Intelligent Systems and Computing, vol 629. Springer, Cham. https://doi.org/10.1007/978-3-319-61911-8_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-61911-8_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-61910-1
Online ISBN: 978-3-319-61911-8
eBook Packages: EngineeringEngineering (R0)