Jianing Zhou
ABSTRACT
Consistency of learning behaviors is known to play an important role in learners’ engagement in a course and impact their learning outcomes. Despite significant advances in the area of learning analytics (LA) in measuring various self-regulated learning behaviors, using LA to measure consistency of online course engagement patterns remains largely unexplored. This study focuses on modeling consistency of learners in online courses to address this research gap. Toward this, we propose a novel unsupervised algorithm that combines sequence pattern mining and ideas from information retrieval with a clustering algorithm to first extract engagement patterns of learners, represent learners in a vector space of these patterns and finally group them into groups with similar consistency levels. Using clickstream data recorded in a popular learning management system over two offerings of a STEM course, we validate our proposed approach to detect learners that are inconsistent in their behaviors. We find that our method not only groups learners by consistency levels, but also provides reliable instructor support at an early stage in a course.
- Rakesh Agarwal, Ramakrishnan Srikant, 1994. Fast algorithms for mining association rules. In Proc. of the 20th VLDB Conference. 487–499.Google Scholar
Digital Library
- Rakesh Agrawal and Ramakrishnan Srikant. 1995. Mining sequential patterns. In Proceedings of the eleventh international conference on data engineering. IEEE, 3–14.Google ScholarCross Ref
- Nora’ayu Ahmad Uzir, Dragan Gašević, Wannisa Matcha, Jelena Jovanović, and Abelardo Pardo. 2020. Analytics of time management strategies in a flipped classroom. Journal of Computer Assisted Learning 36, 1 (2020), 70–88.Google ScholarCross Ref
- Abdullah Alsheddy and Mohamed Habib. 2017. On the application of data mining algorithms for predicting student performance: A case study. Int. J. Comput. Sci. Netw. Secur 17, 10 (2017), 189–197.Google Scholar
- Raheela Asif, Saman Hina, and Saba Izhar Haque. 2017. Predicting student academic performance using data mining methods. International Journal of computer science and network security 17, 5(2017), 187–191.Google Scholar
- Azwa Abdul Aziz, Nur Hafieza Ismail, Fadhilah Ahmad, and Hasni Hassan. 2015. A FRAMEWORK FOR STUDENTS’ ACADEMIC PERFORMANCE ANALYSIS USING NAà VE BAYES CLASSIFIER. Jurnal Teknologi 75, 3 (2015).Google Scholar
- Rachel Baker, Di Xu, Jihyun Park, Renzhe Yu, Qiujie Li, Bianca Cung, Christian Fischer, Fernando Rodriguez, Mark Warschauer, and Padhraic Smyth. 2020. The benefits and caveats of using clickstream data to understand student self-regulatory behaviors: opening the black box of learning processes. International Journal of Educational Technology in Higher Education 17(2020), 1–24.Google ScholarCross Ref
- Yoav Bergner, Deirdre Kerr, and David E Pritchard. 2015. Methodological Challenges in the Analysis of MOOC Data for Exploring the Relationship between Discussion Forum Views and Learning Outcomes.International Educational Data Mining Society (2015).Google Scholar
- Fernanda Bonafini, Chungil Chae, Eunsung Park, and Kathryn Jablokow. 2017. How much does student engagement with videos and forums in a MOOC affect their achievement?Online Learning Journal 21, 4 (2017).Google Scholar
- Mina Shirvani Boroujeni and Pierre Dillenbourg. 2018. Discovery and temporal analysis of latent study patterns in MOOC interaction sequences. In Proceedings of the 8th International Conference on Learning Analytics and Knowledge. 206–215.Google ScholarDigital Library
- Mina Shirvani Boroujeni, Kshitij Sharma, Łukasz Kidziński, Lorenzo Lucignano, and Pierre Dillenbourg. 2016. How to quantify student’s regularity?. In European conference on technology enhanced learning. Springer, 277–291.Google ScholarCross Ref
- Nigel Bosch, R Wes Crues, Genevieve M Henricks, Michelle Perry, Lawrence Angrave, Najmuddin Shaik, Suma Bhat, and Carolyn J Anderson. 2018. Modeling key differences in underrepresented students’ interactions with an online STEM course. In Proceedings of the Technology, Mind, and Society. 1–6.Google ScholarDigital Library
- Jim Broadbent and Walter L Poon. 2015. Self-regulated learning strategies & academic achievement in online higher education learning environments: A systematic review. The Internet and Higher Education 27 (2015), 1–13.Google ScholarCross Ref
- Michael Geoffrey Brown, R Matthew DeMonbrun, Steven Lonn, Stephen J Aguilar, and Stephanie D Teasley. 2016. What and when: the role of course type and timing in students’ academic performance. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge. 459–468.Google ScholarDigital Library
- Sahan Bulathwela, María Pérez-Ortiz, Aldo Lipani, Emine Yilmaz, and John Shawe-Taylor. 2020. Predicting Engagement in Video Lectures. arXiv preprint arXiv:2006.00592(2020).Google Scholar
- Yunfan Chen and Ming Zhang. 2017. Mooc student dropout: Pattern and prevention. In Proceedings of the ACM Turing 50th Celebration Conference-China. 1–6.Google ScholarDigital Library
- Evandro B Costa, Baldoino Fonseca, Marcelo Almeida Santana, Fabrísia Ferreira de Araújo, and Joilson Rego. 2017. Evaluating the effectiveness of educational data mining techniques for early prediction of students’ academic failure in introductory programming courses. Computers in Human Behavior 73 (2017), 247–256.Google ScholarDigital Library
- Scott Crossley, Luc Paquette, Mihai Dascalu, Danielle S McNamara, and Ryan S Baker. 2016. Combining click-stream data with NLP tools to better understand MOOC completion. In Proceedings of the sixth international conference on learning analytics & knowledge. 6–14.Google ScholarDigital Library
- R Wes Crues, Nigel Bosch, Michelle Perry, Lawrence Angrave, Najmuddin Shaik, and Suma Bhat. 2018. Refocusing the lens on engagement in MOOCs. In Proceedings of the fifth annual ACM conference on learning at scale. 1–10.Google ScholarDigital Library
- Louis Faucon, Lukasz Kidzinski, and Pierre Dillenbourg. 2016. Semi-Markov Model for Simulating MOOC Students.International Educational Data Mining Society (2016).Google Scholar
- Ed Fincham, Alexander Whitelock-Wainwright, Vitomir Kovanović, Srećko Joksimović, Jan-Paul van Staalduinen, and Dragan Gašević. 2019. Counting clicks is not enough: Validating a theorized model of engagement in learning analytics. In Proceedings of the 9th international conference on learning analytics & knowledge. 501–510.Google ScholarDigital Library
- Chase Geigle and ChengXiang Zhai. 2017. Modeling MOOC student behavior with two-layer hidden Markov models. In Proceedings of the fourth (2017) ACM conference on learning@ scale. 205–208.Google ScholarDigital Library
- Jiawei Han, Jian Pei, Behzad Mortazavi-Asl, Helen Pinto, Qiming Chen, Umeshwar Dayal, and Meichun Hsu. 2001. Prefixspan: Mining sequential patterns efficiently by prefix-projected pattern growth. In proceedings of the 17th international conference on data engineering. IEEE Washington, DC, USA, 215–224.Google Scholar
- Lauren C Hensley, Christopher A Wolters, Sungjun Won, and Anna C Brady. 2018. Academic probation, time management, and time use in a college success course. Journal of College Reading and Learning 48, 2 (2018), 105–123.Google ScholarCross Ref
- Jui-Long Hung, Morgan C Wang, Shuyan Wang, Maha Abdelrasoul, Yaohang Li, and Wu He. 2015. Identifying at-risk students for early interventions—A time-series clustering approach. IEEE Transactions on Emerging Topics in Computing 5, 1 (2015), 45–55.Google ScholarCross Ref
- Ali Shariq Imran, Fisnik Dalipi, and Zenun Kastrati. 2019. Predicting Student Dropout in a MOOC: An Evaluation of a Deep Neural Network Model. In Proceedings of the 2019 5th International Conference on Computing and Artificial Intelligence. 190–195.Google ScholarDigital Library
- Hogyeong Jeong and Gautam Biswas. 2008. Mining student behavior models in learning-by-teaching environments. In Educational data mining 2008.Google Scholar
- John S Kinnebrew, Kirk M Loretz, and Gautam Biswas. 2013. A contextualized, differential sequence mining method to derive students’ learning behavior patterns.Journal of Educational Data Mining 5, 1 (2013), 190–219.Google Scholar
- René F Kizilcec, Chris Piech, and Emily Schneider. 2013. Deconstructing disengagement: analyzing learner subpopulations in massive open online courses. In Proceedings of the third international conference on learning analytics and knowledge. 170–179.Google ScholarDigital Library
- Andrew S Lan, Christopher G Brinton, Tsung-Yen Yang, and Mung Chiang. 2017. Behavior-Based Latent Variable Model for Learner Engagement.International Educational Data Mining Society (2017).Google Scholar
- Xiao Li, Ting Wang, and Huaimin Wang. 2017. Exploring n-gram features in clickstream data for MOOC learning achievement prediction. In International Conference on Database Systems for Advanced Applications. Springer, 328–339.Google ScholarCross Ref
- Stuart Lloyd. 1982. Least squares quantization in PCM. IEEE transactions on information theory 28, 2 (1982), 129–137.Google ScholarDigital Library
- Jorge Maldonado-Mahauad, Mar Pérez-Sanagustín, René F Kizilcec, Nicolás Morales, and Jorge Munoz-Gama. 2018. Mining theory-based patterns from Big data: Identifying self-regulated learning strategies in Massive Open Online Courses. Computers in Human Behavior 80 (2018), 179–196.Google ScholarDigital Library
- Christopher D Manning, Hinrich Schütze, and Prabhakar Raghavan. 2008. Introduction to information retrieval. Cambridge university press.Google Scholar
- R Martinez, Kalina Yacef, Judy Kay, A Al-Qaraghuli, and Ahmed Kharrufa. 2011. Analysing frequent sequential patterns of collaborative learning activity around an interactive tabletop. In 4th International Conference on Educational Data Mining, EDM 2011. CEUR-WS, 111–120.Google Scholar
- Hirokazu Masataki and Yoshinori Sgisaka. 1996. Variable-order N-gram generation by word-class splitting and consecutive word grouping. In 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings, Vol. 1. IEEE, 188–191.Google ScholarDigital Library
- Ewa Młynarska, Derek Greene, and Pádraig Cunningham. 2016. Time series clustering of Moodle activity data. In 24th Irish Conference on Artificial Intelligence and Cognitive Science (AICS’16), University College Dublin, Dublin, Ireland, 20-21 September 2016.Google Scholar
- John C Nesbit, Mingming Zhou, Yabo Xu, and P Winne. [n.d.]. Advancing log analysis of student interactions with cognitive tools.Google Scholar
- Andrew Y Ng, Michael I Jordan, and Yair Weiss. 2002. On spectral clustering: Analysis and an algorithm. In Advances in neural information processing systems. 849–856.Google Scholar
- Jihyun Park, Kameryn Denaro, Fernando Rodriguez, Padhraic Smyth, and Mark Warschauer. 2017. Detecting changes in student behavior from clickstream data. In Proceedings of the Seventh International Learning Analytics & Knowledge Conference. 21–30.Google ScholarDigital Library
- Jihyun Park, Renzhe Yu, Fernando Rodriguez, Rachel Baker, Padhraic Smyth, and Mark Warschauer. 2018. Understanding Student Procrastination via Mixture Models.International Educational Data Mining Society (2018).Google Scholar
- Paul R Pintrich. 2000. The role of goal orientation in self-regulated learning. In Handbook of self-regulation. Elsevier, 451–502.Google ScholarCross Ref
- C Pradana, SS Kusumawardani, and AE Permanasari. 2020. Comparison Clustering Performance Based on Moodle Log Mining. In IOP Conference Series: Materials Science and Engineering, Vol. 722. IOP Publishing, 012012.Google Scholar
- Arti Ramesh, Dan Goldwasser, Bert Huang, Hal Daume, and Lise Getoor. 2018. Interpretable Engagement Models for MOOCs using Hinge-loss Markov Random Fields. IEEE Transactions on Learning Technologies(2018).Google Scholar
- Arti Ramesh, Dan Goldwasser, Bert Huang, Hal Daumé III, and Lise Getoor. 2014. Learning latent engagement patterns of students in online courses. In Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence. 1272–1278.Google Scholar
- Peter J Rousseeuw. 1987. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of computational and applied mathematics 20 (1987), 53–65.Google ScholarDigital Library
- Amjad Abu Saa 2016. Educational data mining & students’ performance prediction. International Journal of Advanced Computer Science and Applications 7, 5(2016), 212–220.Google Scholar
- Varshita Sher, Marek Hatala, and Dragan Gašević. 2020. Analyzing the consistency in within-activity learning patterns in blended learning. In Proceedings of the Tenth International Conference on Learning Analytics & Knowledge. 1–10.Google ScholarDigital Library
- Atsushi Shimada, Yuta Taniguchi, Fumiya Okubo, Shin’ichi Konomi, and Hiroaki Ogata. 2018. Online change detection for monitoring individual student behavior via clickstream data on E-book system. In Proceedings of the 8th International Conference on Learning Analytics and Knowledge. 446–450.Google ScholarDigital Library
- Jordan Thibodeaux, Aaron Deutsch, Anastasia Kitsantas, and Adam Winsler. 2017. First-year college students’ time use: Relations with self-regulation and GPA. Journal of Advanced Academics 28, 1 (2017), 5–27.Google ScholarCross Ref
- Nora’ayu Ahmad Uzir, Dragan Gašević, Jelena Jovanović, Wannisa Matcha, Lisa-Angelique Lim, and Anthea Fudge. 2020. Analytics of time management and learning strategies for effective online learning in blended environments. In Proceedings of the Tenth International Conference on Learning Analytics & Knowledge. 392–401.Google ScholarDigital Library
- Olga Viberg, Mohammad Khalil, and Martine Baars. 2020. Self-regulated learning and learning analytics in online learning environments: a review of empirical research. In Proceedings of the Tenth International Conference on Learning Analytics & Knowledge. 524–533.Google ScholarDigital Library
- Richard Joseph Waddington, SungJin Nam, Steven Lonn, and Stephanie D Teasley. 2016. Improving early warning systems with categorized course resource usage. Journal of Learning Analytics 3, 3 (2016), 263–290.Google ScholarCross Ref
- Joe H Ward Jr. 1963. Hierarchical grouping to optimize an objective function. Journal of the American statistical association 58, 301(1963), 236–244.Google ScholarCross Ref
- Saijing Zheng, Mary Beth Rosson, Patrick C Shih, and John M Carroll. 2015. Understanding student motivation, behaviors and perceptions in MOOCs. In Proceedings of the 18th ACM conference on computer supported cooperative work & social computing. 1882–1895.Google ScholarDigital Library
Recommendations
Engagement and retention in VET MOOCs and online courses: A systematic review of literature from 2013 to 2017
AbstractBuilding stronger structures that encourage deeper levels of learner engagement and retention in Massive Open Online Courses (MOOCs) is of significant interest to teachers of Vocational Education and Training (VET). Previous literature ...
Highlights- The analysis of MOOC and VET online engagement and retention literature classified four key themes.
Student engagement in massive open online courses
Completion rates in massive open online courses MOOCs are disturbingly low. Existing analysis has focused on patterns of resource access and prediction of drop-out using learning analytics. In contrast, the effectiveness of teaching programs in ...
Benefit and Cost Analysis of Massive Open Online Courses: Pedagogical Implications on Higher Education
There has been much research done on online learning including research on online educational activities and methods. The use of technology is gaining rising importance in higher education due to the benefits that it brings. In terms of adopting new ...
Comments