An Intelligent Online Judge System for Programming Training

Abstract

Online judge (OJ) systems are becoming increasingly popular in various applications such as programming training, competitive programming contests and even employee recruitment, mainly due to their ability of automatic evaluation of code submissions. In higher education, OJ systems have been extensively used in programming courses because the automatic evaluation feature can drastically reduce the grading workload of instructors and teaching assistants and thereby makes the class size scalable. However, in our teaching we feel that existing OJ systems should improve their ability on giving feedback to students and teachers, especially on code errors and knowledge states. The lack of such automatic feedback increases teachers’ involvement and thus prevents college programming training from being more scalable. To tackle this challenge, we leverage historical student data obtained from our OJ system and implement two automated functions, namely, code error prediction and student knowledge tracing, using machine learning models. We demonstrate how students and teachers may benefit from the adoption of these two functions during programming training.