Examining Student Engagement in Online Learning Platforms for Promoting Exam Readiness and Success in Undergraduate Nursing Education

Promoting students' readiness and first-time success on the National Council Licensure Examination for Registered Nurses (NCLEX-RN) is an important driver of investigations and interventions in undergraduate nursing education. However, few studies have linked nursing students' engagement in online learning platforms to their exam performance. We address this gap in the field by applying feature engineering and prediction modeling to engagement and outcome data available for approximately 600 students enrolled in the Bachelor of Science in Nursing program for registered nurses (BSN-RN) from 2015-2022 at a mid-size private university in the Southern U.S.

We derived a total of 185 features from datasets capturing student engagement in two online learning platforms; namely, Elsevier's Adaptive Quizzing (EAQ) and Sherpath. These features reflected patterns of student activity on and performance in assignments and quizzes (e.g., ratio of total completed assignments to distinct completed, longest answer streak). We then created prediction models to identify the EAQ and Sherpath features that were associated with strong performance on (a) Health and Environmental Sciences Institute (HESI) Exit Exams (E2), a high-stakes standardized assessment, and success on (b) the NCLEX-RN.

The best performing classification models are able to correctly distinguish 80% of the time between students who attained a E2 score of at least 850 and students who did not (AUC ROC=0.80). The model for predicting whether a student passes the NCLEX-RN on their first attempt was able to distinguish those students from students who did not pass, 64% of the time (AUC ROC=0.64). The regression models predicting average E2 score and number of NCLEX-RN attempts had a root mean squared error (RMSE) of about 0.81 and 0.87 standard deviations, respectively.

This examination has enabled us to recommend ways in which nursing educators can enhance their use of Sherpath and EAQ, particularly towards identifying and supporting students that require additional curricular intervention to improve their performance on E2 and the NCLEX-RN. We conclude this paper with future directions for research and practice for nursing educators interested in facilitating learning at scale.