short-paper

Incorporating Explainable Learning Analytics to Assist Educators with Identifying Students in Need of Attention

Authors:

Shiva Shabaninejad,

Hassan Khosravi,

Marta Indulska,

Shazia SadiqAuthors Info & Claims

L@S '22: Proceedings of the Ninth ACM Conference on Learning @ Scale

Pages 384 - 388

https://doi.org/10.1145/3491140.3528292

Published: 01 June 2022 Publication History

Abstract

Increased enrolments in higher education, and the shift to online learning that has been escalated by the recent COVID pandemic, have made it challenging for instructors to assist their students with their learning needs. Contributing to the growing literature on instructor-facing systems, this paper reports on the development of a learning analytics (LA) technique called Student Inspection Facilitator (SIF) that provides an explainable interpretation of students learning behaviour to support instructors with the identification of students in need of attention. Unlike many previous predictive systems that automatically label students, our approach provides explainable recommendations to guide data exploration while still reserving judgement about interpreting student learning to instructors. The insights derived from applying SIF in an introductory Information Systems course with 407 enrolled students suggest that SIF can be utilised independent of the context and can provide a meaningful interpretation of students' learning behaviour towards facilitating proactive support of students.

References

[1]

Solmaz Abdi, Hassan Khosravi, Shazia Sadiq, and Dragan Gasevic. 2020. Complementing educational recommender systems with open learner models. In Proceedings of the Tenth International Conference on Learning Analytics & Knowledge. 360--365.

Digital Library

[2]

Nora'ayu Ahmad Uzir, Dragan Gasevic, Wannisa Matcha, Jelena Jovanovic, and Abelardo Pardo. 2020. Analytics of time management strategies in a flipped classroom. Journal of Computer Assisted Learning 36, 1 (2020), 70--88.

[3]

Gökhan Akçapinar, Arif Altun, and Petek Askar. 2019. Using learning analytics to develop early-warning system for at-risk students. IntInternational Journal Educ. Tech. Higher Educ. 16, 1 (2019), 1--20.

[4]

Mennatallah Amer and Markus Goldstein. 2012. Nearest-neighbor and clustering based anomaly detection algorithms for rapidminer. In Proc. of the 3rd RapidMiner Community Meeting and Conference (RCOMM 2012). 1--12.

[5]

Natasha Arthars, Mollie Dollinger, Lorenzo Vigentini, Danny Y-T Liu, Elsuida Kondo, and Deborah M King. 2019. Empowering teachers to personalize learning support. In Utilizing learning analytics to support study success. 223--248.

[6]

et al Atherton. 2017. Using learning analytics to assess student engagement and academic outcomes in open access enabling programmes. Open Learning: The Journal of Open, Distance and e-Learning (2017), 119--136.

[7]

David Baneres, M Elena Rodríguez-Gonzalez, and Montse Serra. 2019. An early feedback prediction system for learners at-risk within a first-year higher education course. IEEE Transactions on Learning Technologies 12, 2 (2019), 249--263.

[8]

Hamsa Bastani, Osbert Bastani, and Carolyn Kim. 2018. Interpreting predictive models for human-in-the-loop analytics. arXiv preprint arXiv:1705.08504 (2018), 1--45.

[9]

Yizhou Fan, Wannisa Matcha, Qiong Wang, Dragan Gasevic, et al . 2021. Learning Analytics to Reveal Links Between Learning Design and Self-Regulated Learning. International Journal of Artificial Intelligence in Education (2021), 1--42.

[10]

Ed Foster and Rebecca Siddle. 2020. The effectiveness of learning analytics for identifying at-risk students in higher education. Assessment & Evaluation in Higher Education 45, 6 (2020), 842--854.

[11]

D Gasevic, S Dawson, and A Pardo. 2016. How do we start? State and directions of learning analytics adoption. International Council for Open and Distance Education.

[12]

Mary Sarah-Jane Gregory and Jason Michael Lodge. 2015. Academic workload: the silent barrier to the implementation of technology-enhanced learning strategies in higher education. Distance education 36, 2 (2015), 210--230.

[13]

David Gunning and David Aha. 2019. DARPA's explainable artificial intelligence (XAI) program. AI Magazine 40, 2 (2019), 44--58.

Digital Library

[14]

Lauren C Hensley, Christopher A Wolters, Sungjun Won, and Anna C Brady. 2018. Academic probation, time management, and time use in a college success course. Journal of College Reading and Learning 48, 2 (2018), 105--123.

[15]

et al Holmes. 2021. Ethics of AI in education: towards a community-wide framework. Int.International Journal of Artificial Intelligence in Education (2021), 1--23.

[16]

Jelena Jovanovic, Dragan Gasevic, Shane Dawson, Abelardo Pardo, Negin Mirriahi, et al. 2017. Learning analytics to unveil learning strategies in a flipped classroom. The Internet and Higher Education 33, 4 (2017), 74--85.

[17]

Jelena Jovanovic, Dragan Gasevic, Abelardo Pardo, Shane Dawson, and Alexander Whitelock-Wainwright. 2019. Introducing Meaning to Clicks: Towards Traced-Measures of Self-Efficacy and Cognitive Load. In Proceedings of the 9th International Conference on Learning Analytics & Knowledge (Tempe, AZ, USA) (LAK19). Association for Computing Machinery, New York, NY, USA, 511--520. https://doi.org/10.1145/3303772.3303782

Digital Library

[18]

Hassan Khosravi, Gianluca Demartini, Shazia Sadiq, and Dragan Gasevic. 2021. Charting the design and analytics agenda of learnersourcing systems. In LAK21: 11th International Learning Analytics and Knowledge Conference. 32--42.

Digital Library

[19]

Hassan Khosravi, Shiva Shabaninejad, Aneesha Bakharia, Shazia Sadiq, Marta Indulska, and Dragan Gasevic. 2021. Intelligent Learning Analytics Dashboards: Automated Drill-Down Recommendations to Support Teacher Data Exploration. Journal of Learning Analytics 8, 3 (2021), 133--154.

[20]

Haim Levy and Myles Robinson. 2006. Stochastic dominance: Investment decision making under uncertainty. Vol. 34.

[21]

Wannisa Matcha, Dragan Gasevic, Abelardo Pardo, et al. 2019. A systematic review of empirical studies on learning analytics dashboards: A self-regulated learning perspective. IEEE Transactions on Learning Technologies 13, 2 (2019), 226--245.

Digital Library

[22]

Christoph Molnar. 2020. Interpretable machine learning. Lulu. com.

[23]

Abelardo Pardo et al . 2018. OnTask: Delivering data-informed, personalized learning support actions. Journal of Learning Analytics 5, 3 (2018), 235--249.

[24]

Dan Pelleg, Andrew W Moore, et al . 2000. X-means: Extending k-means with efficient estimation of the number of clusters. In Icml. 727--734.

[25]

Luis P Prieto, María Jesús Rodríguez-Triana, Roberto Martínez-Maldonado, Yannis Dimitriadis, and Dragan Gasevic. 2019. Orchestrating learning analytics (OrLA): Supporting inter-stakeholder communication about adoption of learning analytics at the classroom level. Australasian Journal of Educational Technology 35, 4 (2019).

[26]

Irina Rets, Christothea Herodotou, Vaclav Bayer, Martin Hlosta, and Bart Rienties. 2021. Exploring critical factors of the perceived usefulness of a learning analytics dashboard for distance university students. Int J. Educ. Tech. Higher Educ. (2021), In--Press.

[27]

Mohammed Saqr, Uno Fors, and Matti Tedre. 2017. How learning analytics can early predict under-achieving students in a blended medical education course. Medical teacher (2017), 757--767.

[28]

Beat A Schwendimann, Maria Jesus Rodriguez-Triana, Andrii Vozniuk, Luis P Prieto, Mina Shirvani Boroujeni, Adrian Holzer, Denis Gillet, and Pierre Dillenbourg. 2016. Perceiving learning at a glance: A systematic literature review of learning dashboard research. IEEE Transactions on Learning Technologies 10, 1 (2016), 30--41.

Digital Library

[29]

Shiva Shabaninejad, Hassan Khosravi, Marta Indulska, Aneesha Bakharia, and Pedro Isaias. 2020. Automated insightful drill-down recommendations for learning analytics dashboards. In Int.Proceedings of the Tenth International Conference on Learning Analytics & Knowledge. 41--46.

Digital Library

[30]

Shiva Shabaninejad, Hassan Khosravi, Sander JJ Leemans, Shazia Sadiq, and Marta Indulska. 2020. Recommending insightful drill-downs based on learning processes for learning analytics dashboards. In Int.International Conference on Artificial Intelligence in Education. 486--499.

Digital Library

[31]

Varshita Sher, Marek Hatala, and Dragan Gasevic. 2020. Analyzing the consistency in within-activity learning patterns in blended learning. In Proceedings of the Tenth Int. International Conference on Learning Analytics & Knowledge. 1--10.

Digital Library

[32]

Billy Tak-ming Wong and Kam Cheong Li. 2020. A review of learning analytics intervention in higher education (2011--2018). Journal of Computers in Education 7, 1 (2020), 7--28.

[33]

Jianing Zhou and Suma Bhat. 2021. Modeling Consistency Using Engagement Patterns in Online Courses. In LAK21: 11th International Learning Analytics and Knowledge Conference. 226--236.

Cited By

Wang YCarpenter ZSwiecki ZShaffer D(2025)Qualitative Parameter Triangulation: A Conceptual and Methodological Framework for Event-Based Temporal ModelsProceedings of the 15th International Learning Analytics and Knowledge Conference10.1145/3706468.3706538(537-546)Online publication date: 3-Mar-2025
https://dl.acm.org/doi/10.1145/3706468.3706538
Lim TGottipati SCheong M(2024)Educational Technologies and Assessment PracticesReshaping Learning with Next Generation Educational Technologies10.4018/979-8-3693-1310-7.ch009(136-172)Online publication date: 26-Apr-2024
https://doi.org/10.4018/979-8-3693-1310-7.ch009
Ujkani BMinkovska DHinov N(2024)Course Success Prediction and Early Identification of At-Risk Students Using Explainable Artificial IntelligenceElectronics10.3390/electronics1321415713:21(4157)Online publication date: 23-Oct-2024
https://doi.org/10.3390/electronics13214157
Show More Cited By

Index Terms

Incorporating Explainable Learning Analytics to Assist Educators with Identifying Students in Need of Attention
1. Applied computing
  1. Education
    1. Computer-assisted instruction
    2. E-learning
2. Information systems
  1. Information systems applications
    1. Decision support systems
      1. Data analytics

Recommendations

Learning analytics dashboards: the past, the present and the future
LAK '20: Proceedings of the Tenth International Conference on Learning Analytics & Knowledge

Learning analytics dashboards are at the core of the LAK vision to involve the human into the decision-making process. The key focus of these dashboards is to support better human sense-making and decision-making by visualising data about learners to a ...
Developing a Learning Analytics Intervention in E-learning to Enhance Students’ Learning Performance: A Case Study
Abstract
The emergence of Learning Analytics has brought benefits to the educational field, as it can be used to analyse authentic data from students to identify the problems encountered in e-learning and to provide intervention to assist students. However,...
Linking learning behavior analytics and learning science concepts: Designing a learning analytics dashboard for feedback to support learning regulation
Abstract
Technological advancements have generated a strong interest in exploring learner behavior data through learning analytics to provide both learner and instructor with process-oriented feedback in the form of dashboards. However, little ...
Highlights
- Little is known about the typology of dashboard feedback.
- Most dashboards ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

L@S '22: Proceedings of the Ninth ACM Conference on Learning @ Scale

June 2022

491 pages

ISBN:9781450391580

DOI:10.1145/3491140

General Chair:
René F. Kizilcec
Cornell University, USA
,
Program Chairs:
Katie Davis
University of Washington, USA
,
Xavier Ochoa
New York University, USA

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 June 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

Australian Research Council's Industrial Transformation Training Centre for Information Resilience (CIRES)

Conference

L@S '22

L@S '22: Ninth (2022) ACM Conference on Learning @ Scale

June 1 - 3, 2022

NY, New York City, USA

Acceptance Rates

Overall Acceptance Rate 117 of 440 submissions, 27%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
191
Total Downloads

Downloads (Last 12 months)46
Downloads (Last 6 weeks)7

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang YCarpenter ZSwiecki ZShaffer D(2025)Qualitative Parameter Triangulation: A Conceptual and Methodological Framework for Event-Based Temporal ModelsProceedings of the 15th International Learning Analytics and Knowledge Conference10.1145/3706468.3706538(537-546)Online publication date: 3-Mar-2025
https://dl.acm.org/doi/10.1145/3706468.3706538
Lim TGottipati SCheong M(2024)Educational Technologies and Assessment PracticesReshaping Learning with Next Generation Educational Technologies10.4018/979-8-3693-1310-7.ch009(136-172)Online publication date: 26-Apr-2024
https://doi.org/10.4018/979-8-3693-1310-7.ch009
Ujkani BMinkovska DHinov N(2024)Course Success Prediction and Early Identification of At-Risk Students Using Explainable Artificial IntelligenceElectronics10.3390/electronics1321415713:21(4157)Online publication date: 23-Oct-2024
https://doi.org/10.3390/electronics13214157
Zhong HNiu JLi J(2024)InsProg: Supporting Teaching Through Visual Analysis of Students’ Programming ProcessesProceedings of the 2024 International Conference on Advanced Visual Interfaces10.1145/3656650.3656668(1-5)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3656650.3656668
Liu QPinto JPaquette L(2024)Applications of Explainable AI (XAI) in EducationTrust and Inclusion in AI-Mediated Education10.1007/978-3-031-64487-0_5(93-109)Online publication date: 28-Sep-2024
https://doi.org/10.1007/978-3-031-64487-0_5
Lim TGottipati SCheong M(2023)Ethical Considerations for Artificial Intelligence in Educational AssessmentsCreative AI Tools and Ethical Implications in Teaching and Learning10.4018/979-8-3693-0205-7.ch003(32-79)Online publication date: 29-Dec-2023
https://doi.org/10.4018/979-8-3693-0205-7.ch003
Sabuncuoglu ASezgin T(2023)Developing a Multimodal Classroom Engagement Analysis Dashboard for Higher-EducationProceedings of the ACM on Human-Computer Interaction10.1145/35932407:EICS(1-23)Online publication date: 19-Jun-2023
https://dl.acm.org/doi/10.1145/3593240
Ujkani BMinkovska DNakov O(2023)Understanding Student Success Prediction Using SHapley Additive exPlanations2023 International Scientific Conference on Computer Science (COMSCI)10.1109/COMSCI59259.2023.10315938(1-4)Online publication date: 18-Sep-2023
https://doi.org/10.1109/COMSCI59259.2023.10315938

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten