skip to main content
10.1145/3375462.3375510acmotherconferencesArticle/Chapter ViewAbstractPublication PageslakConference Proceedingsconference-collections
research-article
Open access

edX log data analysis made easy: introducing ELAT: An open-source, privacy-aware and browser-based edX log data analysis tool

Published: 23 March 2020 Publication History

Abstract

Massive Open Online Courses (MOOCs), delivered on platforms such as edX and Coursera, have led to a surge in large-scale learning research. MOOC platforms gather a continuous stream of learner traces, which can amount to several Gigabytes per MOOC, that learning analytics researchers use to conduct exploratory analyses as well as to evaluate deployed interventions. edX has proven to be a popular platform for such experiments, as the data each MOOC generates is easily accessible to the institution running the MOOC. One of the issues researchers face is the preprocessing, cleaning and formatting of those large-scale learner traces. It is a tedious process that requires considerable computational skills. To reduce this burden, a number of tools have been proposed and released with the aim of simplifying this process. Those tools though still have a significant setup cost, are already out-of-date or require already preprocessed data as a starting point. In contrast, in this paper we introduce ELAT, the edX Log file Analysis Tool, which is browser-based (i.e., no setup costs), keeps the data local (i.e., no server is necessary and the privacy-sensitive learner data is not send anywhere) and takes edX data dumps as input. ELAT does not only process the raw data, but also generates semantically meaningful units (learner sessions instead of just click events) that are visualized in various ways (learning paths, forum participation, video watching sequences). We report on two evaluations we conducted: (i) a technological evaluation and a (ii) user study with potential end users of ELAT. ELAT is open-source and available at https://mvallet91.github.io/ELAT/.

References

[1]
Y. Chen, Q. Chen, Mingqian Zhao, S. Boyer, K. Veeramachaneni, and H. Qu. 2016. DropoutSeer: Visualizing learning patterns in Massive Open Online Courses for dropout reasoning and prediction. In 2016 IEEE Conference on Visual Analytics Science and Technology (VAST). 111--120.
[2]
Dan Davis, Guanliang Chen, Claudia Hauff, and Geert-Jan Houben. 2016. Gauging MOOC Learners' Adherence to the Designed Learning Path. International Educational Data Mining Society (2016).
[3]
Dan Davis, Guanliang Chen, Claudia Hauff, and Geert-Jan Houben. 2018. Activating learning at scale: A review of innovations in online learning strategies. Computers & Education 125 (2018), 327--344.
[4]
Dan Davis, Guanliang Chen, Ioana Jivet, Claudia Hauff, and Geert-Jan Houben. 2016. Encouraging Metacognition & Self-Regulation in MOOCs through Increased Learner Feedback. In LAL@ LAK. 17--22.
[5]
Dan Davis, Vasileios Triglianos, Claudia Hauff, and Geert-Jan Houben. 2018. Srlx: A personalized learner interface for moocs. In European Conference on Technology Enhanced Learning. Springer, 122--135.
[6]
Sara Isabella De Freitas, John Morgan, and David Gibson. 2015. Will MOOCs transform learning and teaching in higher education? Engagement and course retention in online learning provision. British Journal of Educational Technology 46, 3 (2015), 455--471.
[7]
Oliver Ferschke, Diyi Yang, Gaurav Tomar, and Carolyn Penstein Rosé. 2015. Positive impact of collaborative chat participation in an edX MOOC. In International Conference on Artificial Intelligence in Education. Springer, 115--124.
[8]
S. Fu, J. Zhao, W. Cui, and H. Qu. 2017. Visual Analysis of MOOC Forums with iForum. IEEE Transactions on Visualization and Computer Graphics 23, 1 (Jan 2017), 201--210.
[9]
Jeffrey A Greene, Christopher A Oswald, and Jeffrey Pomerantz. 2015. Predictors of retention and achievement in a massive open online course. American Educational Research Journal 52, 5 (2015), 925--955.
[10]
Philip J Guo, Juho Kim, and Rob Rubin. 2014. How video production affects student engagement: An empirical study of MOOC videos. In Proceedings of the first ACM conference on Learning@ scale conference. ACM, 41--50.
[11]
Kate S Hone and Ghada R El Said. 2016. Exploring the factors affecting MOOC retention: A survey study. Computers & Education 98 (2016), 157--168.
[12]
Juho Kim, Philip J Guo, Daniel T Seaton, Piotr Mitros, Krzysztof Z Gajos, and Robert C Miller. 2014. Understanding in-video dropouts and interaction peaks inonline lecture videos. In Proceedings of the first ACM conference on Learning@ scale conference. ACM, 31--40.
[13]
René F Kizilcec, Mar Pérez-Sanagustín, and Jorge J Maldonado. 2016. Recommending self-regulated learning strategies does not improve performance in a MOOC. In Proceedings of the Third (2016) ACM Conference on Learning@ Scale. ACM, 101--104.
[14]
René F Kizilcec, Chris Piech, and Emily Schneider. 2013. Deconstructing disengagement: analyzing learner subpopulations in massive open online courses. In Proceedings of the third international conference on learning analytics and knowledge. ACM, 170--179.
[15]
Bettina Laugwitz, Theo Held, and Martin Schrepp. 2008. Construction and Evaluation of a User Experience Questionnaire. In HCI and Usability for Education and Work, Andreas Holzinger (Ed.). Berlin, Heidelberg, 63--76.
[16]
Nan Li, Łukasz Kidziński, Patrick Jermann, and Pierre Dillenbourg. 2015. MOOC video interaction patterns: What do they tell us? In Design for teaching and learning in a networked world. 197--210.
[17]
Allison Littlejohn, Nina Hood, Colin Milligan, and Paige Mustain. 2016. Learning in MOOCs: Motivations and self-regulated learning in MOOCs. The Internet and Higher Education 29 (2016), 40--48.
[18]
Glenn Lopez, Daniel T Seaton, Andrew Ang, Dustin Tingley, and Isaac Chuang. 2017. Google BigQuery for education: framework for parsing and analyzing edX MOOC data. In Proceedings of the Fourth (2017) ACM Conference on Learning@Scale. ACM, 181--184.
[19]
Colin Milligan, Allison Littlejohn, and Anoush Margaryan. 2013. Patterns of engagement in connectivist MOOCs. Journal of Online Learning and Teaching 9, 2 (2013), 149--159.
[20]
Zachary A. Pardos and Kevin Kao. 2015. moocRP: An Open-source Analytics Platform. In Proceedings of the Second (2015) ACM Conference on Learning @ Scale (L@S '15). New York, NY, USA, 103--110.
[21]
Zachary A Pardos, Steven Tang, Daniel Davis, and Christopher Vu Le. 2017. Enabling real-time adaptivity in MOOCs with a personalized next-step recommendation framework. In Proceedings of the Fourth (2017) ACM Conference on Learning@ Scale. ACM, 23--32.
[22]
Zachary A Pardos, Anthony Whyte, and Kevin Kao. 2016. moocRP: enabling open learning analytics with an open source platform for data distribution, analysis, and visualization. Technology, Knowledge and Learning 21, 1 (2016), 75--98.
[23]
Trang Phan, Sara G McNeil, and Bernard R Robin. 2016. Students' patterns of engagement and course performance in a Massive Open Online Course. Computers & Education 95 (2016), 36--44.
[24]
Héctor J Pijeira Díaz, Javier Santofimia Ruiz, José A Ruipérez-Valiente, Pedro J Muñoz-Merino, and Carlos Delgado Kloos. 2016. A demonstration of ANALYSE: a learning analytics tool for open edX. In Proceedings of the Third (2016) ACM Conference on Learning@ Scale. ACM, 329--330.
[25]
Oleksandra Poquet, Vitomir Kovanović, Pieter de Vries, Thieme Hennis, Srećko Joksimović, Dragan Gašević, and Shane Dawson. 2018. Social presence in massive open online courses. International Review of Research in Open and Distributed Learning 19, 3 (2018).
[26]
Arti Ramesh, Dan Goldwasser, Bert Huang, Hal Daumé III, and Lise Getoor. 2013. Modeling learner engagement in MOOCs using probabilistic soft logic. In NIPS Workshop on Data Driven Education, Vol. 21. 62.
[27]
Justin Reich and José A. Ruipérez-Valiente. 2019. The MOOC pivot. Science 363, 6423 (2019), 130--131.
[28]
Jan Renz, Daniel Hoffmann, Thomas Staubitz, and Christoph Meinel. 2016. Using A/B testing in MOOC environments. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge. ACM, 304--313.
[29]
Tarmo Robal, Yue Zhao, Christoph Lofi, and Claudia Hauff. 2018. IntelliEye: Enhancing MOOC Learners' Video Watching Experience through Real-Time Attention Tracking. In Proceedings of the 29th on Hypertext and Social Media. ACM, 106--114.
[30]
Javier Santofimia Ruiz, Héctor J Pijeira Díaz, José A Ruipérez-Valiente, Pedro J Muñoz-Merino, and Carlos Delgado Kloos. 2014. Towards the development of a learning analytics extension in open edX. In Proceedings of the Second International Conference on Technological Ecosystems for Enhancing Multiculturality. ACM, 299--306.
[31]
Martin Schrepp, Andreas Hinderks, and Jörg Thomaschewski. 2014. Applying the user experience questionnaire (UEQ) in different evaluation scenarios. In International Conference of Design, User Experience, and Usability. Springer, 383--392.
[32]
Daniel T Seaton, Yoav Bergner, Isaac Chuang, Piotr Mitros, and David E Pritchard. 2014. Who does what in a massive open online course? (2014).
[33]
Conglei Shi, Siwei Fu, Qing Chen, and Huamin Qu. 2015. VisMOOC: Visualizing video clickstream data from massive open online courses. In 2015 IEEE Pacific visualization symposium (PacificVis). IEEE, 159--166.
[34]
Kalyan Veeramachaneni, Franck Dernoncourt, Colin Taylor, Zachary Pardos, and Una-May O'Reilly. 2013. Moocdb: Developing data standards for mooc data science. In AIED 2013 workshops proceedings volume, Vol. 17. Citeseer.
[35]
Kalyan Veeramachaneni, Sherif Halawa, Franck Dernoncourt, Una-May O'Reilly, Colin Taylor, and Chuong Do. 2014. Moocdb: Developing standards and systems to support mooc data science. arXiv preprint arXiv:1406.2015 (2014).
[36]
Sunnie Lee Watson, William R Watson, Ji Hyun Yu, Hamdan Alamri, and Chad Mueller. 2017. Learner profiles of attitudinal learning in a MOOC: An explanatory sequential mixed methods study. Computers & Education 114 (2017), 274--285.
[37]
Miaomiao Wen, Diyi Yang, and Carolyn Rose. 2014. Sentiment Analysis in MOOC Discussion Forums: What does it tell us?. In Educational data mining 2014. Citeseer.
[38]
Julie Wintrup, Kelly Wakefield, and Hugh C Davis. 2015. Engaged learning in MOOCs: a study using the UK Engagement Survey. (2015).

Cited By

View all
  • (2024)Scaling While Privacy Preserving: A Comprehensive Synthetic Tabular Data Generation and Evaluation in Learning AnalyticsProceedings of the 14th Learning Analytics and Knowledge Conference10.1145/3636555.3636921(620-631)Online publication date: 18-Mar-2024
  • (2024)The Sequence Matters in Learning - A Systematic Literature ReviewProceedings of the 14th Learning Analytics and Knowledge Conference10.1145/3636555.3636880(263-272)Online publication date: 18-Mar-2024
  • (2024)Investigating Learning Dashboards AdaptationTechnology Enhanced Learning for Inclusive and Equitable Quality Education10.1007/978-3-031-72315-5_3(34-48)Online publication date: 16-Sep-2024
  • Show More Cited By

Index Terms

  1. edX log data analysis made easy: introducing ELAT: An open-source, privacy-aware and browser-based edX log data analysis tool

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    LAK '20: Proceedings of the Tenth International Conference on Learning Analytics & Knowledge
    March 2020
    679 pages
    ISBN:9781450377126
    DOI:10.1145/3375462
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 23 March 2020

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. edX log
    2. learning analytics
    3. log data analysis
    4. massive open online course

    Qualifiers

    • Research-article

    Funding Sources

    • Delft Data Science, the LDE Centre for Education and Learning

    Conference

    LAK '20

    Acceptance Rates

    LAK '20 Paper Acceptance Rate 80 of 261 submissions, 31%;
    Overall Acceptance Rate 236 of 782 submissions, 30%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)131
    • Downloads (Last 6 weeks)13
    Reflects downloads up to 20 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Scaling While Privacy Preserving: A Comprehensive Synthetic Tabular Data Generation and Evaluation in Learning AnalyticsProceedings of the 14th Learning Analytics and Knowledge Conference10.1145/3636555.3636921(620-631)Online publication date: 18-Mar-2024
    • (2024)The Sequence Matters in Learning - A Systematic Literature ReviewProceedings of the 14th Learning Analytics and Knowledge Conference10.1145/3636555.3636880(263-272)Online publication date: 18-Mar-2024
    • (2024)Investigating Learning Dashboards AdaptationTechnology Enhanced Learning for Inclusive and Equitable Quality Education10.1007/978-3-031-72315-5_3(34-48)Online publication date: 16-Sep-2024
    • (2023)Understanding privacy and data protection issues in learning analytics using a systematic reviewBritish Journal of Educational Technology10.1111/bjet.1338854:6(1715-1747)Online publication date: 15-Sep-2023
    • (2023)Applying Deep Knowledge Tracing Model for University Students’ Programming Learning2023 International Conference on Information Networking (ICOIN)10.1109/ICOIN56518.2023.10048977(574-577)Online publication date: 11-Jan-2023
    • (2022)Analyzing Log Data of Students Who Have Achieved Scores Adjacent to the Minimum Passing Grade for a K-MOOC Completion in the Context of Learning AnalyticsSustainability10.3390/su14181113614:18(11136)Online publication date: 6-Sep-2022
    • (2021)Development of a Learning Analytics extension in Open edX2021 International Conference of Modern Trends in Information and Communication Technology Industry (MTICTI)10.1109/MTICTI53925.2021.9664754(1-6)Online publication date: 4-Dec-2021
    • (2020)OXALIC: an Open edX Advanced Learning Analytics Tool2020 IEEE Learning With MOOCS (LWMOOCS)10.1109/LWMOOCS50143.2020.9234322(185-190)Online publication date: 29-Sep-2020

    View Options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Login options

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media