research-article

Public Access

Experience: Learner Analytics Data Quality for an eTextbook System

Authors:

Mohammed F. Farghally,

Hossameldin Shahin,

Clifford A. ShafferAuthors Info & Claims

Journal of Data and Information Quality (JDIQ), Volume 9, Issue 2

Article No.: 10, Pages 1 - 10

https://doi.org/10.1145/3148240

Published: 15 January 2018 Publication History

Abstract

We present lessons learned related to data collection and analysis from 5 years of experience with the eTextbook system OpenDSA. The use of such cyberlearning systems is expanding rapidly in both formal and informal educational settings. Although the precise issues related to any such project are idiosyncratic based on the data collection technology and goals of the project, certain types of data collection problems will be common. We begin by describing the nature of the data transmitted between the student’s client machine and the database server, and our initial database schema for storing interaction log data. We describe many problems that we encountered, with the nature of the problems categorized as syntactic-level data collection issues, issues with relating events to users, or issues with tracking users over time. Relating events to users and tracking the time spent on tasks are both prerequisites to converting syntactic-level interaction streams to semantic-level behavior needed for higher-order analysis of the data. Finally, we describe changes made to our database schema that helped to resolve many of the issues that we had encountered. These changes help advance our ultimate goal of encouraging a change from ineffective learning behavior by students to more productive behavior.

References

[1]

R. S. Baker, A. T. Corbett, and K. R. Koedinger. 2004a. Detecting student misuse of intelligent tutoring systems. In Proceedings of the 7th International Conference on Intelligent Tutoring Systems. 531--540.

[2]

R. S. Baker, A. T. Corbett, K. R. Koedinger, and A. Z. Wagner. 2004b. Off-task behavior in the cognitive tutor classroom: When students “game the system.” In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’04). 383--390.

Digital Library

[3]

A. R. Basawapatna, A. Repenning, and K. H. Koh. 2015. Closing the cyberlearning loop: Enabling teachers to formatively assess student programming projects. In Proceedings of the 46th ACM Technical Symposium on Computer Science Education (SIGCSE’15). 12--17.

Digital Library

[4]

M. Berland, R. S. Baker, and P. Blikstein. 2014. Educational data mining and learning analytics: Applications to constructionist research. Technology, Knowledge and Learning 19, 1--2, 205--220.

[5]

S. Boyer, B. U. Gelman, B. Schreck, and K. Veeramachaneni. 2015. Data science foundry for MOOCs. In Proceedings of the IEEE International Conference on Data Science and Advanced Analytics. 1--10.

[6]

Stephen H. Edwards and Krishnan Panamalai Murali. 2016. Codeworkout. Retrieved December 1, 2017, from https://github.com/web-cat/code-workout.

[7]

M. F. Farghally, K. H. Koh, H. Shahin, and C. A. Shaffer. 2017. Evaluating the effectiveness of algorithm analysis visualizations. In Proceedings of the 48th ACM Technical Symposium on Computer Science Education (SIGCSE’17).

Digital Library

[8]

E. Fouh, D. A. Breakiron, S. Hamouda, M. F. Farghally, and C. A. Shaffer. 2014a. Exploring students learning behavior with an interactive eTextbook in computer science courses. Computers in Human Behavior 41, C, 478--485.

Digital Library

[9]

E. Fouh, S. Hamouda, M. F. Farghally, and C. A. Shaffer. 2016. Automating learner feedback in an eTextbook for data structures and algorithms courses. In Formative Assessment, Learning Data Analytics and Gamification in ICT Education, S. Caballo and R. Clariso (Eds.). Elsevier, Amsterdam, Netherlands, 135--166.

[10]

E. Fouh, V. Karavirta, D. A. Breakiron, S. Hamouda, S. Hall, T. L. Naps, and C. A. Shaffer. 2014b. Design and architecture of an interactive eTextbook—the OpenDSA system. Science of Computer Programming 88, 1, 22--40.

[11]

P. J. Guo, J. Kim, and R. Rubin. 2014. How video production affects student engagement: An empirical study of MOOC videos. In Proceedings of the 1st ACM Conference on Learning @ Scale Conference (L@S’14). 41--50.

Digital Library

[12]

J. Kim, P. J. Guo, C. J. Cai, S. W. Li, K. Z. Gajos, and R. C. Miller. 2014. Data-driven interaction techniques for improving navigation of educational videos. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST’14). 563--572.

Digital Library

[13]

K. H. Koh, A. Basawapatna, H. Nickerson, and A. Repenning. 2014. Real time assessment of computational thinking. In Proceedings of the Conference on Visual Languages and Human-Centric Computing (VL/HCC’14). 49--52.

[14]

D. Roberge, A. Rojas, and R. S. Baker. 2012. Does the length of time off-task matter? In Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (LAK’12). 234--237.

Digital Library

[15]

J. Sheard. 2010. Basics of statistical analysis of interactions data from Web-based learning environments, 27--42. In Handbook of Educational Data Mining, C. Romero, S. Ventura, M. Pechenizkiy, and R. S. J. D. Baker (Eds.). CRC Press, Boca Raton, FL, 27--42.

Cited By

Domino MShaffer C(2025)Using a wider digital ecosystem to improve self-regulated learningFrontiers in Education10.3389/feduc.2025.148734410Online publication date: 27-Feb-2025
https://doi.org/10.3389/feduc.2025.1487344
Joy JRaj NV. G. R(2021)Ontology-based E-learning Content Recommender System for Addressing the Pure Cold-start ProblemJournal of Data and Information Quality10.1145/342925113:3(1-27)Online publication date: 27-Apr-2021
https://dl.acm.org/doi/10.1145/3429251
Purnama FUsagawa T(2020)Using real-time online preprocessed mouse tracking for lower storage and transmission costsJournal of Big Data10.1186/s40537-020-00304-x7:1Online publication date: 10-Apr-2020
https://doi.org/10.1186/s40537-020-00304-x
Show More Cited By

Index Terms

Experience: Learner Analytics Data Quality for an eTextbook System
1. Social and professional topics
  1. Professional topics
    1. Computing education
      1. Computing education programs
        Computer science education
        Information science education

Recommendations

OpenDSA: An Interactive eTextbook for Computer Science Courses
SIGCSE '16: Proceedings of the 47th ACM Technical Symposium on Computing Science Education

The open-source OpenDSA eTextbook system is seeing rapidly growing use around the world. It combines textbook-quality tutorial material with many algorithm visualizations (AVs), simulations, and a large collection of interactive exercises, including ...
OpenDSA: using an active eTextbook to teach data structures and algorithms (abstract only)
SIGCSE '13: Proceeding of the 44th ACM technical symposium on Computer science education

We present a study to evaluate OpenDSA, an open source, online system combining textbook-quality content with algorithm visualizations and interactive exercises for data structures and algorithms courses. We hypothesize that answering many questions and ...
Potential Issues on Initiatively Utilizing E-Textbooks in K-12 Classrooms
ICALT '13: Proceedings of the 2013 IEEE 13th International Conference on Advanced Learning Technologies

In this study, an experiment has been conducted in two elementary schools with 9 teachers and 203 students involved to identify the potential issues when initiatively utilizing eTextbooks in 14 classes by using a mix-method design of lecture video coding,...

Comments

Information & Contributors

Information

Published In

cover image Journal of Data and Information Quality

Journal of Data and Information Quality Volume 9, Issue 2

Challenge Paper, Experience Paper and Research Paper

June 2017

77 pages

ISSN:1936-1955

EISSN:1936-1963

DOI:10.1145/3155015

Editor:
Tiziana Catarci
Sapienza University of Rome, Rome, Italy

Issue’s Table of Contents

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 January 2018

Accepted: 01 September 2017

Revised: 01 July 2017

Received: 01 April 2016

Published in JDIQ Volume 9, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

National Science Foundation

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
516
Total Downloads

Downloads (Last 12 months)78
Downloads (Last 6 weeks)7

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Domino MShaffer C(2025)Using a wider digital ecosystem to improve self-regulated learningFrontiers in Education10.3389/feduc.2025.148734410Online publication date: 27-Feb-2025
https://doi.org/10.3389/feduc.2025.1487344
Joy JRaj NV. G. R(2021)Ontology-based E-learning Content Recommender System for Addressing the Pure Cold-start ProblemJournal of Data and Information Quality10.1145/342925113:3(1-27)Online publication date: 27-Apr-2021
https://dl.acm.org/doi/10.1145/3429251
Purnama FUsagawa T(2020)Using real-time online preprocessed mouse tracking for lower storage and transmission costsJournal of Big Data10.1186/s40537-020-00304-x7:1Online publication date: 10-Apr-2020
https://doi.org/10.1186/s40537-020-00304-x
Purnama FSukhbaatar OChoimaa LUsagawa T(2020)Implementation of real-time online mouse tracking on overseas quiz sessionEducation and Information Technologies10.1007/s10639-020-10141-325:5(3845-3880)Online publication date: 1-Sep-2020
https://dl.acm.org/doi/10.1007/s10639-020-10141-3
Gong CXie T(2018)Exploring Desirable Features of e-Textbooks for K-12 Classes: A Case Study2018 Seventh International Conference of Educational Innovation through Technology (EITT)10.1109/EITT.2018.00032(123-127)Online publication date: Dec-2018
https://doi.org/10.1109/EITT.2018.00032

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Figures

Tables

Media

View Issue’s Table of Contents