Predicting User Dropout from Their Online Learning Behavior

Abstract

The Covid-19 pandemic, which required more people to work and learn remotely, emphasized the benefits of online learning. However, these online learning environments, which are typically used on an individual basis, can make it difficult for many to finish courses effectively. At the same time, online learning allows for the monitoring of users, which may help to identify learners who are struggling. In this article, we present the results of a set of experiments focusing on the early prediction of user drop out, based on data from the New Heroes Academy, a learning center providing online courses.

For measuring the impact of user behavior over time with respect to user drop out, we build a range of random forest classifiers. Each classifier uses all features, but the feature values are calculated from the day a user starts a course up to a particular day. The target describes whether the user will finish the course or not. Our experimental results (using 10-fold cross-validation) show that the classifiers provide good results (over 90% accuracy from day three with somewhat lower results for the classifiers for day one and two). In particular, the time-based and action-based features have a major impact on the performance, whereas the start-based feature is only important early on (i. e., during day one).