Modeling Learners’ Programming Skills and Question Levels Through Machine Learning

Abstract

Many universities have started to adopt online programming tools to support students’ programming practice, yet the services currently offered by the existing tools are somewhat passive with respect to considering a student’s programming skill level and providing appropriate code questions. To enhance students’ learning experience and improve their programming skills, it would be helpful to examine students’ programming abilities and provide them with the most suitable code questions and guidelines. Machine learning can play a role in modeling the level of students’ programming skills as well as the difficulty of questions by taking the students’ programming experience and code submissions into account. This paper presents a study on the development of machine learning models to classify the levels of students’ programming skills and those of programming questions, based on the data of students’ code submissions. We extracted a total of 197 features of code quality, code readability and system time. We used those features to build classification models. The model for the student level (four classes) and the question level (five classes) yielded 0.60 and 0.82 F1-scores, respectively, showing reasonable classification performance. We discuss our study highlights and their implications, such as group and question matching based on code submissions and user experience improvement.