skip to main content
10.1145/2889160.2889203acmconferencesArticle/Chapter ViewAbstractPublication PagesicseConference Proceedingsconference-collections
research-article

STAGE: a software tool for automatic grading of testing exercises: case study paper

Published: 14 May 2016 Publication History

Abstract

We report on an approach and associated tool-support for automatically evaluating and grading exercises in Software Engineering courses, by connecting various third-party tools to the online learning platform Moodle. In the case study presented here, the tool was used in several instances of a lecture course to automatically measure the test coverage criteria wrt. the test cases defined by the students for given Java code. We report on empirical evidence gathered using this case-study (involving more than 250 students), including the results of a survey conducted after the exercises (which yielded positive feedback from the students), as well as a performance evaluation of our tool implementation.

References

[1]
A. P. F. F. Lopes, "Teaching with Moodle in higher education," tech. rep., Institute of Accounting and Administration (ISCAP), Polytechnic Institute of Oporto (IPP), 2011.
[2]
A. Fox, "From MOOCs to SPOCs," Communications of the ACM, vol. 56, no. 12, pp. 38--40, 2013.
[3]
J. Burge, A. Fox, D. Grossman, G. Roth, and J. Warren, "SPOCs: what, why, and how," in 46th ACM Technical Symposium on Computer Science Education, SIGCSE '15, pp. 595--596, ACM, 2015.
[4]
J. Hollingsworth, "Automatic graders for programming classes," Commun. ACM, vol. 3, pp. 528--529, Oct. 1960.
[5]
G. E. Forsythe and N. Wirth, "Automatic grading programs," Commun. ACM, vol. 8, no. 5, pp. 275--278, 1965.
[6]
C. Douce, D. Livingstone, and J. Orwell, "Automatic test-based assessment of programming: A review," J. Educ. Resour. Comput., vol. 5, Sept. 2005.
[7]
P. Ihantola, T. Ahoniemi, V. Karavirta, and O. Seppälä, "Review of recent systems for automatic assessment of programming assignments," in Proceedings of the 10th Koli Calling International Conference on Computing Education Research, Koli Calling '10, (New York, NY, USA), pp. 86--93, ACM, 2010.
[8]
J. C. Caiza and J. M. D. Alamo, "Programming assignments automatic grading: Review of tools and implementations," in Proceedings of 7th International Technology, Education and Development Conference, (Valencia (Spain)), January 2013.
[9]
S. H. Edwards and Z. Shams, "Comparing test quality measures for assessing student-written tests," in Companion Proceedings of the 36th International Conference on Software Engineering, pp. 354--363, ACM, 2014.
[10]
A. Barana, M. Marchisio, and S. Rabellino, "Automated assessment in mathematics," in Proceedings of COMPSAC, 2015.
[11]
S. Zhigang, S. Xiaohong, Z. Ning, and C. Yanyu, "Moodle plugins for highly efficient programmin courses," in Moodle Research Conference, vol. 1, pp. 157--163, 2012.
[12]
C. Sanchez, O. Ramos, P. Márquez, E. Martí, J. Rocarias, and D. Gil, "Automatic evaluation of practices in Moodle for self learning in engineering," Journal of Technology and Science Education, vol. 5, no. 2, 2015.
[13]
S. T. Deane, Further development on the Moodle CodeHandIn Package. PhD thesis, Flinders University-Adelaide, Australia, 2014.
[14]
R. Lobb, "Coderunner documentation (v2.4.2)." http://coderunner.org.nz/mod/book/tool/print/index.php?id=50, October 2015.
[15]
R. Likert, "A technique for the measurement of attitudes.," Archives of psychology, 1932.
[16]
M. B. Miles, A. M. Huberman, and J. Saldaña, Qualitative data analysis: A methods sourcebook. SAGE Publications, Incorporated, 2013.
[17]
S. Robertson, "Moodle on the move." WebEx session, https://onlinevideo.napier.ac.uk/Play/5081, 2015.

Cited By

View all
  • (2024)Learning with Style: Improving Student Code-Style Through Better Automated FeedbackProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630889(1175-1181)Online publication date: 7-Mar-2024
  • (2024)Automated Student Assignment Evaluation Based on Information Retrieval and Statistical TechniquesProceedings of the Second Congress on Control, Robotics, and Mechatronics10.1007/978-981-97-7094-6_12(157-167)Online publication date: 14-Nov-2024
  • (2023)Automated Assessment in Computer Science: A Bibliometric Analysis of the LiteratureLearning Technologies and Systems10.1007/978-3-031-33023-0_11(122-134)Online publication date: 26-May-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ICSE '16: Proceedings of the 38th International Conference on Software Engineering Companion
May 2016
946 pages
ISBN:9781450342056
DOI:10.1145/2889160
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 May 2016

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. automatic grading
  2. case study
  3. education and training
  4. software engineering
  5. software testing

Qualifiers

  • Research-article

Funding Sources

  • EU
  • DFG

Conference

ICSE '16
Sponsor:

Acceptance Rates

Overall Acceptance Rate 276 of 1,856 submissions, 15%

Upcoming Conference

ICSE 2025

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)9
  • Downloads (Last 6 weeks)1
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Learning with Style: Improving Student Code-Style Through Better Automated FeedbackProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630889(1175-1181)Online publication date: 7-Mar-2024
  • (2024)Automated Student Assignment Evaluation Based on Information Retrieval and Statistical TechniquesProceedings of the Second Congress on Control, Robotics, and Mechatronics10.1007/978-981-97-7094-6_12(157-167)Online publication date: 14-Nov-2024
  • (2023)Automated Assessment in Computer Science: A Bibliometric Analysis of the LiteratureLearning Technologies and Systems10.1007/978-3-031-33023-0_11(122-134)Online publication date: 26-May-2023
  • (2022)Automated Code Assessment for Education: Review, Classification and Perspectives on Techniques and ToolsSoftware10.3390/software10100021:1(3-30)Online publication date: 8-Feb-2022
  • (2022)Simplifying the creation and maintenance of automated assessments of programming tasks via Test Specific LanguageProceedings of the 2022 6th International Conference on Education and E-Learning10.1145/3578837.3578840(14-20)Online publication date: 21-Nov-2022
  • (2022)Automated Assessment in Computer Science Education: A State-of-the-Art ReviewACM Transactions on Computing Education10.1145/3513140Online publication date: 24-Feb-2022
  • (2021)A Systematic Review of the Effects of Automatic Scoring and Automatic Feedback in Educational SettingsIEEE Access10.1109/ACCESS.2021.31008909(108190-108198)Online publication date: 2021
  • (2019)Executable Examples for Programming Problem ComprehensionProceedings of the 2019 ACM Conference on International Computing Education Research10.1145/3291279.3339416(131-139)Online publication date: 30-Jul-2019
  • (2018)Who Tests the Testers?Proceedings of the 2018 ACM Conference on International Computing Education Research10.1145/3230977.3230999(51-59)Online publication date: 8-Aug-2018
  • (2017)Investigating code quality tools in the context of software engineering educationComputer Applications in Engineering Education10.1002/cae.2179325:2(230-241)Online publication date: 1-Mar-2017

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media