Student behavior in a web-based educational system: Exit intent prediction

Highlights

• We proposed a classifier predicting if the student ends the session in next action.

• Proposed classifier processes the dynamically changing data streams of logs.

• List of most important attributes for the session end prediction is identified.

• Analysis of students׳ behavior in a web-based educational system is provided.

Abstract

The behavior of users over the web is one of the most relevant and research topic nowadays. Not only mining the user׳s behavior in order to provide better content is popular, but the prediction of the user׳s behavior is interesting and can increase user experience. Moreover, the business clearly desires such information to improve their services. In this paper we focus to the education domain as it belongs to the most dynamically transforming areas. Web based e-learning systems are nowadays reaching still greater popularity, because of possibilities they offer to students. We analyze various sources of “e-students” feedback and discuss today׳s challenges from the logging and feedback collecting point of view. Next, we focus on the prediction of student׳s next action within an e-learning application (in the mean of “stay or leave?” question). Such information can improve students׳ attrition rate by introducing various personalized approaches. We proposed the classifier based on polynomial regression and stochastic gradient descent to learn the attributes importance. In this way we are able to process a stream of data in one single iteration and thus we are able to reflect dynamic users׳ behavior changes. Our experiments are based on the log data collected from our web-based education system ALEF during three-year period. We found that there is an extensive heterogeneity in the users׳ (student) behavior which we were able to handle by using individual weights calculated for every user.

Introduction

Every one of us is a unique person who responds differently to perceptions obtained from the environment. The task of interaction with a software and various systems can be problematic for this reason, since the systems are mainly designed to operate in a one strictly defined way, regardless of user who interact with them. Nowadays an increasing amount of web-based systems use personalization, because it allows to match the content to specific user׳s needs and preferences. This process may take many forms – it can be an adaptation of a content presented to the user, a change of a search results order, an arrangement or a change of system interface components appearance etc.

In order to provide adaptive, personalized or specifically adjusted content or service, the user needs, preferences and often attitudes have to be known and visible to the system. From this point of view, the user׳s feedback plays crucial role. As the both – users and business as well gains benefits from such a tailored content or service (user access relevant information or products in shorter time, business lowers the adverts cost and raises profits), the “obsession” on collecting user feedback and using it to improve the web increases day by day.

In business sphere, the task of the prediction whether a customer will stay or exit using particular service (e.g., do not prolong the contract) is referred as the attrition or conversion rate. Such a prediction can be computed based on the customer behavior and the feedback he/she leaves during the contract (which is usually long term). However, in this paper, we focus on the task of the learning session end prediction for specific student in the e-learning web-based system (prediction of student exit intent in the session). This represents a novel application of standard long term attrition rate task to the short term behavior, which brings new challenges and also possibilities for user behavior prediction expressed by his/her next action(s).

E-learning systems are currently very popular and millions of students learn using those (Jegatha Deborah et al., 2014). Moreover, similarly to e-learning systems the MOOCs (Massive Open Online Course) are often used to promote the universities and allows them to sell certificates to graduates, which brings a huge business potential. E-learning system typically contains various courses containing learning materials divided into logical units called learning objects (LOs), e.g., explanations, questions or the practical exercises to solve presented in various forms combing text and multimedia. This rich information source can be used to improve the e-learning based on specific students׳ characteristics and behavior.

There are many advantages of e-learning in comparison to traditional education. One of the most important is, that “e-students” can adjust the learning process to their own needs and speed, which fit them the most. Jovanovic et al. (2012) proposed the clustering method for grouping students based on their cognitive learning style. This way are users able to spend their time in e-learning effectively, because system is able to automatically adapt their learning materials with respect to their learning styles.

Another advantage of e-learning systems is the possibility to adapt the course structure, navigation or its content exactly to the needs of every student individually. The concept of the adaptation and personalization of web-based systems for the domain of e-learning was introduced by Brusilovsky (1996) and is still intensively researched nowadays. There were proposed the methods of personalized recommendation, such as hybrid approach from Klasnja-Milevic et al. (2011) which is similarly to previous approach based on students׳ learning styles, but in this case also on frequent sequences of in content learned.

The task of the session end prediction represents an interesting challenge of e-learning. Students sometimes decide to stop learning while they did not understand fully the materials. If the e-learning application would be able to predict that student will probably leave soon, it could motivate him/her to stay longer, remind him/her the learning object he/she has not studied yet or offer him/her some questions to test his/her real knowledge. In this way the system will be able to help the student to learn effectively, e.g., not miss any of topics to learn in order to better prepare for his/her exam.

Our contributions presented in this papers are:

• An analysis of e-learning students׳ behavior and feedback types and sources.

• Novel approach for student exit intent prediction for actual session designed for highly dynamic data in the form of data stream.

In comparison to the state-of-the-art approaches and challenges in the attrition rate prediction including e-learning domain, our proposed approach focuses on short-term behavior prediction (in the mean of one session). Proposed approach fully takes an advantage of all available user characteristics- including students׳ performance, their personalities or learning styles. Thanks to the predictor architecture (polynomial binary classifier, using the stochastic gradient descent algorithm), we are able to process students׳ actions within the system as the data stream and dynamically make predictions for actual sessions. Such a short-term prediction is not used in today׳s web-based systems, including the e-learning domain.

The rest of the paper is structured as follows. The related work and the state-of-the-art is presented in the next section. The section “E-Students” Challenges describes the current trends of e-learning and its advantages in comparison to the traditional learning approaches focusing on the student feedback. We demonstrate the most important features and the ways of collecting the feedback from students׳ actions considering our e-learning system ALEF (Adaptive LEarning Framework). In the following section we focus on one task of user behavior prediction. We describe proposed method for prediction the next user action in web-based educational system in mean if he/she stay or will leave. The section Evaluation shows the results of the proposed method used with various settings. Finally, in the section Conclusions we summarize the achieved results and discuss future work.