Abstract
Massive open online courses (MOOCs) hold the promise of democratizing the learning process. However, providing effective feedback has proven hard to offer at scale since most methods require a teacher or tutor. Leveraging big data in MOOCs offers a mechanism to develop predictive models that can inform computer-based pedagogical tutors. We review research on grade prediction and examine the predictive power of a model based on user video-watching behavior. In a MOOC organized around weekly assignments, we find that frequency of video viewing per week is a better predictor than individual viewing features such as plays, pauses, seeking, and rate changes. This finding is useful for MOOCs that use assignments for course evaluations in addition or to the exclusion of in-video quizzes for formative assessment. Engaging, well-crafted assignments in MOOCs have the potential of boosting student retention and course completion by fostering a deeper understanding through application and practice.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Agarwal, P., Karpicke, J., Kang, S., Roediger, H., & McDermott, K. (2008). Examining the testing effect with open- and closed-book tests. Applied Cognitive Psychology, 22(7), 861–876. https://doi.org/10.1002/acp.1391.
Ashenafi, M. M., Riccardi, G., & Ronchetti, M. (2015). Predicting students’ final exam scores from their course activities. In 2015 IEEE Frontiers in Education Conference (FIE) (pp. 1–9). IEEE. https://doi.org/10.1109/FIE.2015.7344081.
Azevedo, R., & Hadwin, A. F. (2005). Scaffolding Self-regulated Learning and Metacognition – Implications for the Design of Computer-based Scaffolds. Instructional Science, 33(5–6), 367–379. https://doi.org/10.1007/s11251-005-1272-9.
Bazelais, P., Lemay, D. J., & Doleck, T. (2018). Examining the Link Between Prior Achievement in Secondary Education and Performance in College: Using Data from Pre-University Physics Courses. Journal of Formative Design in Learning, 2(2), 114–120. https://doi.org/10.1007/s41686-018-0020-x.
Bangert-Drowns, R., Kulik, J., & Kulik, C. (1991). Effects of Frequent Classroom Testing. The Journal of Educational Research, 85(2), 89–99. https://doi.org/10.1080/00220671.1991.10702818.
Bozkurt, A., Akgün-Özbek, E., & Zawacki-Richter, O. (2017). Trends and Patterns in Massive Open Online Courses: Review and Content Analysis of Research on MOOCs (2008-2015). The International Review Of Research In Open And Distributed Learning, 18(5). https://doi.org/10.19173/irrodl.v18i5.3080.
Brahimi, T., & Sarirete, A. (2015). Learning outside the classroom through MOOCs. Computers in Human Behavior, 51, 604–609. https://doi.org/10.1016/j.chb.2015.03.013.
Brinton, C. G., & Chiang, M. (2015). MOOC Performance Prediction via Clickstream Data and Social Learning Networks. IEEE Conference on Computer Communications (INFOCOM), 2299–2307.
DeBoer, J., Ho, A. D., Stump, G. S., & Breslow, L. (2014). Changing “Course”. Educational Researcher, 43(2), 74–84. https://doi.org/10.3102/0013189X14523038.
Dutt, A., Ismail, M., & Herawan, T. (2017). A Systematic Review on Educational Data Mining. IEEE Access, 5, 15991–16005. https://doi.org/10.1109/access.2017.2654247.
Ebben, M., & Murphy, J. (2014). Unpacking MOOC scholarly discourse: a review of nascent MOOC scholarship. Learning, Media and Technology, 39(3), 328–345.
Formanek, M., Wenger, M., Buxner, S., Impey, C., & Sonam, T. (2017). Insights about large-scale online peer assessment from an analysis of an astronomy MOOC. Computers & Education, 113, 243–262. https://doi.org/10.1016/j.compedu.2017.05.019.
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. (2009). The WEKA data mining software. ACM SIGKDD Explorations Newsletter, 11(1), 10. https://doi.org/10.1145/1656274.1656278.
Henseler, J., Hubona, G., & Ray, P. (2016). Using PLS path modeling in new technology research: updated guidelines. Industrial Management & Data Systems, 116(1), 2–20. https://doi.org/10.1108/imds-09-2015-0382.
Hong, B., Wei, Z., & Yang, Y. (2017). Online education performance prediction via time-related features. In 2017 IEEE/ACIS 16th International Conference on Computer and Information Science (ICIS) (pp. 95–100). IEEE. https://doi.org/10.1109/ICIS.2017.7959976
Hulse, T., Harrison, A., Ostrow, K. S., Botelho, A. F., & Heffernan, N. T. (2018). Starters and Finishers: Predicting Next Assignment Completion from Student Behavior During Math Problem Solving. In K. E. Boyer & M. Yudelson (Eds.), Proceedings of the 11th International Conference on Educational Data Mining (pp. 525–528). Buffalo, NY: International Educational Data Mining Society.
Jansen, D., & Schuwer, R. (2015). Institutional MOOC strategies in Europe (pp. 1–34). EADTU. Retrieved from https://www.surfspace.nl/media/bijlagen/artikel-1763-22974efd1d43f52aa98e0ba04f14c9f3.pdf
Jiang, Y., Bosch, N., Baker, R., Paquette, L., Ocumpaugh, J., Andres, J.M.A.L., Moore, A.L., Biswas, G. (2018) Expert Feature-Engineering vs. Deep Neural Networks: Which is Better for Sensor-Free Affect Detection? In C. P. Rosé, R. Martínez-Maldonado, U. Hoppe, R. Luckin, M. Mavrikis, K. Porayska-Pomsta, B. McLaren, & B. du Boulay (Eds.), Proceedings of the 19th International Conference on Artificial Intelligence in Education (pp. 198–211). London, UK: Springer International Publishing. https://doi.org/10.1007/978-3-319-93843-1_15.
Kock, N. (2018a). WarpPLS. Retrieved from http://www.scriptwarp.com/warppls/
Kock, N. (2018b). WarpPLS 6.0 user manual. ScripWarp Systems. Retrieved from http://cits.tamiu.edu/WarpPLS/UserManual_v_6_0.pdf
Kovanović, V., Joksimović, S., Gašević, D., Siemens, G., & Hatala, M. (2015). What public media reveals about MOOCs: A systematic analysis of news reports. British Journal of Educational Technology, 46(3), 510–527. https://doi.org/10.1111/bjet.12277.
Li, X., Xie, L., & Wang, H. (2016). Grade Prediction in MOOCs. In 2016 IEEE Intl Conference on Computational Science and Engineering (CSE) and IEEE Intl Conference on Embedded and Ubiquitous Computing (EUC) and 15th Intl Symposium on Distributed Computing and Applications for Business Engineering (DCABES) (pp. 386–392). IEEE. https://doi.org/10.1109/CSE-EUC-DCABES.2016.213
Meier, Y., Xu, J., Atan, O., & Schaar, M. van der. (2015). Personalized Grade Prediction: A Data Mining Approach. In 2015 IEEE International Conference on Data Mining (pp. 907–912). IEEE. https://doi.org/10.1109/ICDM.2015.54.
Mothukuri, U. K., Reddy, B. V., Reddy, P. N., Gutti, S., Mandula, K., Parupalli, R., … Magesh, E. (2017). Improvisation of learning experience using learning analytics in eLearning. In 2017 5th National Conference on E-Learning & E-Learning Technologies (ELELTECH) (pp. 1–6). IEEE. https://doi.org/10.1109/ELELTECH.2017.8074995.
Qiu, J., Tang, J., Liu, T. X., Gong, J., Zhang, C., Zhang, Q., & Xue, Y. (2016). Modeling and Predicting Learning Behavior in MOOCs. In Proceedings of the Ninth ACM International Conference on Web Search and Data Mining - WSDM ‘16, 93–102. https://doi.org/10.1145/2835776.2835842.
Reich, J. (2015). Rebooting MOOC Research. Science, 347(6217), 34–35. https://doi.org/10.1126/science.1261627.
Roediger, H., & Karpicke, J. (2006a). Test-Enhanced Learning: Taking Memory Tests Improves Long-Term Retention. Psychological Science, 17(3), 249–255. https://doi.org/10.1111/j.1467-9280.2006.01693.x.
Roediger, H., & Karpicke, J. (2006b). The Power of Testing Memory: Basic Research and Implications for Educational Practice. Perspectives on Psychological Science, 1(3), 181–210. https://doi.org/10.1111/j.1745-6916.2006.00012.x.
Romero, C., & Ventura, S. (2010). Educational Data Mining: A Review of the State of the Art. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 40(6), 601–618. https://doi.org/10.1109/TSMCC.2010.2053532.
Shmueli, G. (2010). To Explain or to Predict? Statistical Science, 25(3), 289–310. https://doi.org/10.1214/10-STS330.
Sinha, T., Jermann, P., Li, N., & Dillenbourg, P. (2014). Your click decides your fate: Inferring Information Processing and Attrition Behavior from MOOC Video Clickstream Interactions. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3–14, October 25–29, 2014, Doha, Qatar. Retrieved from http://arxiv.org/abs/1407.7131
Slater, S., Joksimović, S., Kovanovic, V., Baker, R., & Gasevic, D. (2016). Tools for Educational Data Mining. Journal of Educational and Behavioral Statistics, 42(1), 85–106. https://doi.org/10.3102/1076998616666808.
Toven-Lindsey, B., Rhoads, R., & Lozano, J. (2015). Virtually unlimited classrooms: Pedagogical practices in massive open online courses. The Internet and Higher Education, 24, 1–12. https://doi.org/10.1016/j.iheduc.2014.07.001.
Xie, T., Zheng, Q., Zhang, W., & Qu, H. (2017). Modeling and Predicting the Active Video-Viewing Time in a Large-Scale E-Learning System. IEEE Access, 5, 11490–11504. https://doi.org/10.1109/ACCESS.2017.2717858.
Yang, T.-Y., Brinton, C. G., Joe-Wong, C., & Chiang, M. (2017). Behavior-Based Grade Prediction for MOOCs via Time Series Neural Networks. IEEE Journal of Selected Topics in Signal Processing, 4553(c), 1–1. https://doi.org/10.1109/JSTSP.2017.2700227.
Zhu, M., Sari, A., & Lee, M. (2018). A systematic review of research methods and topics of the empirical MOOC literature (2014–2016). The Internet and Higher Education, 37, 31–39. https://doi.org/10.1016/j.iheduc.2018.01.002.
Acknowledgments
We wish to thank Dr. Ryan Baker and his Learning Analytics Lab for this collaboration and providing the performance data from his Big Data and Education MOOC from the University of Pennsylvania. The first author also acknowledges the support of the Centre for Medical Education for his postdoctoral fellowship.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Lemay, D.J., Doleck, T. Grade prediction of weekly assignments in MOOCS: mining video-viewing behavior. Educ Inf Technol 25, 1333–1342 (2020). https://doi.org/10.1007/s10639-019-10022-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10639-019-10022-4