article

Automated assessment and experiences of teaching programming

Authors:

Colin A. Higgins,

Pavlos Symeonidis,

Athanasios TsintsifasAuthors Info & Claims

Journal on Educational Resources in Computing (JERIC), Volume 5, Issue 3

Pages 5 - es

https://doi.org/10.1145/1163405.1163410

Published: 01 September 2005 Publication History

Abstract

This article reports on the design, implementation, and usage of the CourseMarker (formerly known as CourseMaster) courseware Computer Based Assessment (CBA) system at the University of Nottingham. Students use CourseMarker to solve (programming) exercises and to submit their solutions. CourseMarker returns immediate results and feedback to the students. Educators author a variety of exercises that benefit the students while offering practical benefits. To date, both educators and students have been hampered by CBA software that has been constructed to assess text-based or multiple-choice answers only. Although there exist a few CBA systems with some capability to automatically assess programming coursework, none assess Java programs and none are as flexible, architecture-neutral, robust, or secure as the CourseMarker CBA system.

References

[1]

Arnow, D. and Barshay, O. 1999a. On-line programming examinations using WebToTeach. In Proceedings of the 4th Annual SIGCSE/SIGCUE Conference on Innovation and Technology in Computer Science Education (Krakow, Poland, June 27--30). 21-24.

[2]

Arnow, D. and Barshey, O. 1999b. WebToTeach: An interactive focused programming exercise system. In Proceedings of the 29th ASEE/IEEE Frontiers in Education Conference (Puerto Rico, Nov.).

[3]

Ahmadzadeh, M., Elliman, D., and Higgins, C. 2005. An analysis of patterns of debugging among novice computer science students. In Proceedings of the ITiCSE 2005 Conference (Lisbon).

[4]

Al-Yahya, M., Higgins, C., Brailsford, T., and Ashman, H. 2005. Question Master: An open model for reusing dialogue in learning environments. In Proceedings of the CAL'05 Conference on Virtual Learning.

[5]

Benford, S. D., Burke, E. K., Foxley, E., Gutteridge, N. H., and Mohd Zin, A. 1993. Experiences with the Ceilidh system. In Proceedings of the 1st International Conference on Computer Based Learning in Science (Vienna).

[6]

Benford, S. D., Burke, E. K., Foxley, E., Gutteridge, N. H., Higgins, C., and Mohd Zin, A. 1994. Software support for automated assessment and administration. J. Res. Comput. Edu. (1994).

[7]

Charman, D. and Elmes, A. 1998. Computer Based Assessment: A Guide to Good Practice. Vol. I. University of Plymouth, 1998.

[8]

Culwin, F. and Lancaster, T. 2001. Plagiarism issues for higher education. Inf. Security 21, 2, 36--41.

[9]

Daly, C. 1999. RoboProf and an introductory computer programming course. In Proceedings of the 4th Annual SIGCSE/SIGCUE Conference on Innovation and Technology in Computer Science Education (Krakow, June 27--30). 155--158.

[10]

Foxley, E., Higgins, C., Hegazy, T., Symeonidis, P., and Tsintsifas, A. 2001a. The CourseMaster CBA system: Improvements over Ceilid. In Proceedings of the Fifth International Computer Assisted Assessment Conference (Loughborough University, UK, July 2--4). 189--201.

[11]

Foxley, E., Higgins, C., Symeonidis, P., and Tsintsifas, A. 2001b. The CourseMaster automated assessment system---A next generation Ceilidh. In Proceedings of the Workshop on Computer Assisted Assessment to Support the ICS Disciplines (University of Warwick, April 5--6).

[12]

Foxley, E., Higgins, C., Tsintsifas, A., and Symeonidis, P. 2000. The Ceilidh-CourseMaster system, An introduction. In Proceedings of the. 4th Java in the Curriculum Conference (South Bank University, UK, Jan. 24).

[13]

Foxley, E., Tsintsifas, A., Higgins, C., and Symeonidis P. 1999. Ceilidh, A system for the automatic evaluation of students' programming work. In Proceedings of the CBLISS 99 Conference (University of Twente, Holland, July 2--7).

[14]

Foxley, E., Higgins, C., Symeonidis, P., and Tsintsifas, A. 1998. Security issues under Ceilidh's WWW interface. In Proceedings of the ICCE'98 Conference (Beijing, Oct. 14--17). 235--240.

[15]

Higgins, C., Symeonidis, P., and Tsintsifas, A. 2002a. Diagram-based CBA using DATsys and CourseMaster. In Proceedings of the International Conference on Computers in Education (ICCE2002, Auckland, New Zealand, Dec. 3--6).

[16]

Higgins, C., Tsintsifas, A., and Symeonidis, P. 2002b. CourseMaster marking programs and diagrams. In Proceedings of the Dealing with Plagiarism in ICS Education Conference (Warwick, April 11--12).

[17]

Hyvönen, J. and Malmi, L. 1993. TRAKLA---A system for teaching algorithms using email and a graphical editor. In Proceedings of the HYPERMEDIA Conference (Vaasa). 141--147.

[18]

Jackson, D. and Usher, M. 1997. Grading student programs using ASSYST, In Proceedings of the 28th SIGCSE Technical Symposium on Computer Science Education (San Jose, CA, Feb. 27--March 1). 335--339.

[19]

Luck, M. and Joy, M. S. 1999. A secure on-line submission system. Softw. --- Pract. Exper. 29, 8, 721--740.

[20]

Mason, D. and Woit, D. 1998. Integrating technology into computer science examinations. In Proceedings of the 29th SIGCSE Technical Symposium on Computer Science Education (Atlanta, GA, Feb. 26--March 1). 1401--1404.

[21]

Sun Microsystems. 1999. The JAVA HotSpot performance engine architecture. White paper. http://java.sun.com/products/hotspot/whitepaper.html, 1999.

[22]

Symeonidis, P. 2001. An in-depth review of CourseMaster marking subsystem. Tech. Rep., LTR Group, University of Nottingham, UK.

[23]

Symeonidis, P. 1998. Creating an exercise using CourseMarker (formerly JavaCeilidh). Tech. Rep., LTR Group, University of Nottingham, UK.

[24]

Yong, C. and Higgins, C. 2003. Automatically creating personalised exercises based on student profiles. In Proceedings of the ITiCSE 2003 Conference (Thessaloniki, Greece, June 30--July 2).

Cited By

Asgari MTsai FMannila LStrömbäck FSadique K(2024)Students’ perspectives on using digital tools in programming coursesDiscover Education10.1007/s44217-024-00144-43:1Online publication date: 22-May-2024
https://doi.org/10.1007/s44217-024-00144-4
Lagakis PDemetriadis S(2024)EvaAI: A Multi-agent Framework Leveraging Large Language Models for Enhanced Automated GradingGenerative Intelligence and Intelligent Tutoring Systems10.1007/978-3-031-63028-6_32(378-385)Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1007/978-3-031-63028-6_32
Lagakis PDemetriadis SPsathas G(2024)Automated Grading in Coding Exercises Using Large Language ModelsSmart Mobile Communication & Artificial Intelligence10.1007/978-3-031-54327-2_37(363-373)Online publication date: 27-Feb-2024
https://doi.org/10.1007/978-3-031-54327-2_37
Show More Cited By

Index Terms

Automated assessment and experiences of teaching programming
1. Applied computing
  1. Education

Recommendations

A Phenomenological Study of Online Assessment Perceptions of Macao Higher Learning Institutions During the COVID-19 Pandemic: :Dual Perspectives of Students and Instructors
ICEDS '22: Proceedings of the 2022 3rd International Conference on Education Development and Studies

Online assessment is the key issue of studying online learning during the COVID-19 pandemic. Often, online assessments are understood as the mirror to face-to-face learning assessments. In fact, online assessment should start from the premise that all ...
Automated Assessment of Programming Assignments
CSERC '13: Proceedings of the 3rd Computer Science Education Research Conference on Computer Science Education Research

This is a position paper in which I argue that massive open online programming courses can benefit by the application of automated assessment of programming assignments.

I gathered success factors and identified concerns related to automatic assessment ...
Beyond Programming: A Computer-Based Assessment of Computational Thinking Competency
Computational Thinking (CT), entailing both domain-general and domain-specific skills, is a competency fundamental to computing education and beyond. However, as a cross-domain competency, appropriate assessment design and method remain equivocal. Indeed, ...

Comments

Information & Contributors

Information

Published In

cover image Journal on Educational Resources in Computing

Journal on Educational Resources in Computing Volume 5, Issue 3

September 2005

135 pages

ISSN:1531-4278

EISSN:1531-4278

DOI:10.1145/1163405

Issue’s Table of Contents

Copyright © 2005 ACM.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 September 2005

Published in JERIC Volume 5, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

98
Total Citations
View Citations
2,412
Total Downloads

Downloads (Last 12 months)71
Downloads (Last 6 weeks)7

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Asgari MTsai FMannila LStrömbäck FSadique K(2024)Students’ perspectives on using digital tools in programming coursesDiscover Education10.1007/s44217-024-00144-43:1Online publication date: 22-May-2024
https://doi.org/10.1007/s44217-024-00144-4
Lagakis PDemetriadis S(2024)EvaAI: A Multi-agent Framework Leveraging Large Language Models for Enhanced Automated GradingGenerative Intelligence and Intelligent Tutoring Systems10.1007/978-3-031-63028-6_32(378-385)Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1007/978-3-031-63028-6_32
Lagakis PDemetriadis SPsathas G(2024)Automated Grading in Coding Exercises Using Large Language ModelsSmart Mobile Communication & Artificial Intelligence10.1007/978-3-031-54327-2_37(363-373)Online publication date: 27-Feb-2024
https://doi.org/10.1007/978-3-031-54327-2_37
Amin MRahman MShirafuji AWatanobe Y(2023)Analysis of the Programming Languages Preferred by Novice Programmers for Solving Programming ProblemProceedings of the 2023 4th Asia Service Sciences and Software Engineering Conference10.1145/3634814.3634818(23-28)Online publication date: 27-Oct-2023
https://dl.acm.org/doi/10.1145/3634814.3634818
Rico-Juan JSánchez-Cartagena VValero-Mas JGallego A(2023)Identifying Student Profiles Within Online Judge Systems Using Explainable Artificial IntelligenceIEEE Transactions on Learning Technologies10.1109/TLT.2023.323911016:6(955-969)Online publication date: Dec-2023
https://doi.org/10.1109/TLT.2023.3239110
Ramesha VShankar SThalanki SKurpad SAuradkar P(2023)BigHOST: Automatic Grading System for Big Data Assignments2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Internet Computing Workshops (CCGridW)10.1109/CCGridW59191.2023.00051(256-262)Online publication date: May-2023
https://doi.org/10.1109/CCGridW59191.2023.00051
Sudakova NSavina TMasalimova AMikhaylovsky MKarandeeva LZhdanov S(2022)Online Formative Assessment in Higher Education: Bibliometric AnalysisEducation Sciences10.3390/educsci1203020912:3(209)Online publication date: 14-Mar-2022
https://doi.org/10.3390/educsci12030209
Vesin BMangaroska KAkhuseyinoglu KGiannakos M(2022)Adaptive Assessment and Content Recommendation in Online Programming Courses: On the Use of Elo-ratingACM Transactions on Computing Education10.1145/351188622:3(1-27)Online publication date: 9-Jun-2022
https://dl.acm.org/doi/10.1145/3511886
Sychev OPéraire CSerebrenik A(2022)Write a lineProceedings of the ACM/IEEE 44th International Conference on Software Engineering: Software Engineering Education and Training10.1145/3510456.3514159(265-276)Online publication date: 21-May-2022
https://dl.acm.org/doi/10.1145/3510456.3514159
Luukkainen RKasurinen JNikula ULenarduzzi VPéraire CSerebrenik A(2022)ASPAProceedings of the ACM/IEEE 44th International Conference on Software Engineering: Software Engineering Education and Training10.1145/3510456.3514149(29-39)Online publication date: 21-May-2022
https://dl.acm.org/doi/10.1145/3510456.3514149
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents