Pierre Ortegat
ABSTRACT
Automated feedback and grading platforms can require substantial effort when encoding new programming exercises for first-year students. Such exercises are usually simple but require defining several test cases to ensure their functional correctness. This paper describes our initial effort to leverage automated test case generation for simple programming exercises. We rely on grey-box fuzzing and random combinations of method calls to test the students' solutions and compare their execution to the results produced by a reference implementation. We implemented our approach in a prototype, called SimPyTest, openly available on GitHub. We discuss its usage and possible future extensions.
- Mauricio Aniche. 2022. Effective Software Testing: A Developer’s Guide. Simon and Schuster.
Google Scholar
- Roberto Baldoni, Emilio Coppa, Daniele Cono D’elia, Camil Demetrescu, and Irene Finocchi. 2019. A Survey of Symbolic Execution Techniques. Comput. Surveys, 51, 3 (2019), May, 1–39. https://doi.org/10.1145/3182657
Google ScholarDigital Library
- O. Bonaventure, Q. De Coninck, F. Duchêne, A. Gégo, M. Jadin, F. Michel, M. Piraux, C. Poncin, and O. Tilmans. 2020. Open educational resources for computer networking. ACM SIGCOMM Computer Communication Review, 50, 3 (2020), July, 38–45. https://doi.org/10.1145/3411740.3411746
Google ScholarDigital Library
- Sébastien Combéfis and Vianney le Clément de Saint-Marcq. 2012. Teaching Programming and Algorithm Design with Pythia, a Web-Based Learning Platform. Olympiads in Informatics, 6 (2012), 31–43.
Google Scholar
- Thomas H Cormen, Charles E Leiserson, Ronald L Rivest, and Clifford Stein. 2022. Introduction to algorithms. MIT press.
Google Scholar
- Guillaume Derval, Anthony Gego, Pierre Reinbold, Benjamin Frantzen, and Peter Van Roy. 2015. Automatic grading of programming exercises in a MOOC using the INGInious platform. In EMOOCS’15. Mons, Belgium. 86–91.
Google Scholar
- Guillaume Derval, Anthony Gégo, Ludovic Taffin, and al. 2022. INGInious. https://github.com/UCL-INGI/INGInious
Google Scholar
- Piotr Duch and Tomasz Jaworski. 2018. Dante - Automated Assessments Tool for Students’ Programming Assignments. In HSI’18. IEEE, Gdansk, Poland. 162–168. https://doi.org/10.1109/HSI.2018.8431146
Google ScholarCross Ref
- Nobuo Funabiki, Ryota Kusaka, Nobuya Ishihara, and Wen-Chung Kao. 2017. A Proposal of Test Code Generation Tool for Java Programming Learning Assistant System. In AINA’17. IEEE, Taiwan. 51–56. https://doi.org/10.1109/AINA.2017.60
Google ScholarCross Ref
- Stephan Lukasczyk, Florian Kroiß, and Gordon Fraser. 2020. Automated Unit Test Generation for Python. In SSBSE’20. Springer International Publishing, 9–24. https://doi.org/10.1007/978-3-030-59762-7_2
Google ScholarDigital Library
- Ruan Reis, Gustavo Soares, Melina Mongiovi, and Wilkerson L. Andrade. 2019. Evaluating Feedback Tools in Introductory Programming Classes. In FIE’19). IEEE, Covington, KY, USA. 1–7. https://doi.org/10.1109/FIE43999.2019.9028418
Google ScholarDigital Library
- Felipe Restrepo-Calle, Jhon Jairo Ramírez-Echeverry, and Fabio A. Gonzalez. 2018. Uncode: Interactive System for Learning and Automatic Evaluation of Computer Programming Skills. In EduLearn’18. Palma, Spain. 6888–6898. https://doi.org/10.21125/edulearn.2018.1632
Google ScholarCross Ref
- Thomas Staubitz, Ralf Teusner, and Christoph Meinel. 2017. Towards a repository for open auto-gradable programming exercises. In TALE’17. IEEE, Hong Kong. 66–73. https://doi.org/10.1109/TALE.2017.8252306
Google ScholarCross Ref
- Andreas Zeller, Rahul Gopinath, Marcel Böhme, Gordon Fraser, and Christian Holler. 2021. The Fuzzing Book. CISPA Helmholtz Center for Information Security. https://www.fuzzingbook.org/ Retrieved 2021-10-26.
Google Scholar
Index Terms
- Towards automated testing for simple programming exercises
Recommendations
Mapping hardness of automated software testing
ISSTA 2017: Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and AnalysisAutomated Test Case Generation (ATCG) is an important topic in Software Testing, with a wide range of techniques and tools being used in academia and industry. While their usefulness is widely recognized, due to the labor-intensive nature of the task, ...
Grey-box concolic testing on binary code
ICSE '19: Proceedings of the 41st International Conference on Software EngineeringWe present grey-box concolic testing, a novel path-based test case generation method that combines the best of both white-box and grey-box fuzzing. At a high level, our technique systematically explores execution paths of a program under test as in ...
Comments