Authors:
Fábio De Rezende Souza
;
Francisco de Assis Zampirolli
and
Guiou Kobayashi
Affiliation:
Centro de Matemática, Computação e Cognição, Universidade Federal do ABC (UFABC), 9.210-580, Santo André, São Paulo and Brazil
Keyword(s):
Artificial Intelligence, Automatic Grading, Text Classification, Deep Learning.
Related
Ontology
Subjects/Areas/Topics:
Blended Learning
;
Computer-Supported Education
;
Learning/Teaching Methodologies and Assessment
Abstract:
Thousands of students have their assignments evaluated by their teachers every day around the world while developing their studies in any branch of science. A fair evaluation of their schoolwork is a very challenging task. Here we present a method for validating the grades attributed by professors to students programming exercises in an undergraduate introductory course in computer programming. We collected 938 final exam exercises in Java Language developed during this course, evaluated by different professors, and trained a convolutional neural network over those assignments. First, we submit their codes to a cleaning process (by removing comments and anonymizing variables). Next, we generated an embedding representation of each source code produced by students. Finally, this representation is taken as the input of the neural network which classifies each label (corresponding to the possible grades A, B, C, D or F). An independent neural network is trained with source code solution
s corresponding to each assignment. We obtained an average accuracy of 74.9% in a 10−fold cross validation for each grade. We believe that this method can be used to validate the grading process made by professors in order to detect errors that might happen during this process.
(More)