Skip to main content
SpringerLink
Log in

Visual analytics of potential dropout behavior patterns in online learning based on counterfactual explanation

  • Regular Paper
  • Published:
Journal of Visualization Aims and scope Submit manuscript

Abstract

Online learning is gradually becoming a popular way of learning due to the high flexibility in time and space. Reducing the high dropout rate is important to promote the further development of smart education. However, learners’ learning is a dynamic temporal process, which is influenced by multiple factors synergistically. How to identify the key influencing factors of dropout in an interpretable way is still a challenging problem. In this paper, we propose a pattern identification method of dropout behavior, including the prediction of the dropout probability and the mining of potential impact factors, to gain a comprehensive insight into the dropout behavior hidden in the data. A CNN-LSTM model for dropout prediction is constructed, which can automatically extract features and learn the temporal dependence of dropout behavior. By introducing the counterfactual explanation, the dropout impacts of different learning behavior can be revealed quantitatively. Moreover, we design and develop an interactive visual analytics system, DropoutVis, for exploring learning behavior, extracting the various dropout patterns and providing a basis for formulating strategies. The effectiveness and usefulness of DropoutVis have been demonstrated through case studies with a real dataset.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  • Amnueypornsakul B, Bhat S, Chinprutthiwong P (2014) Predicting attrition along the way: the UIUC model. In: Proceedings of the EMNLP 2014 workshop on analysis of large scale social interaction in MOOCs, pp 55–59

  • Chen Q, Chen Y, Liu D, Shi C, Wu Y, Qu H (2016) Peakvizor: visual analytics of peaks in video clickstreams from massive open online courses. IEEE Trans Visual Comput Graph 22(10):2315–2330

    Article  Google Scholar 

  • Chen Q, Yue X, Plantaz X, Chen Y, Shi C, Pong T, Qu H (2020) Viseq: visual analytics of learning sequence in massive open online courses. IEEE Trans Visual Comput Graph 26(3):1622–1636

    Article  Google Scholar 

  • Chen Y,Chen Q, Zhao M, Boyer S, Veeramachaneni K, Qu H (2016) Dropoutseer: visualizing learning patterns in massive open online courses for dropout reasoning and prediction. In: 11th IEEE conference on visual analytics science and technology, IEEE VAST 2016, Baltimore, MD, USA, October 23–28, 2016. IEEE Computer Society, pp 111–120

  • Cheng F, Ming Y, Qu H (2021) DECE: decision explorer with counterfactual explanations for machine learning models. IEEE Trans Visual Comput Graph 27(2):1438–1447

    Article  Google Scholar 

  • Fisnik D, Shariq IA, Zenun K (2018) Mooc dropout prediction using machine learning techniques: review and research challenges. In 2018 IEEE global engineering education conference (EDUCON), pp 1007–1014

  • Fu S, Wang Y, Yang Y, Bi Q, Guo F, Qu H (2018) Visforum: A visual analysis system for exploring user groups in online forums. ACM Trans Interact Intell Syst 8(1):3:1-3:21

    Article  Google Scholar 

  • Fu S, Zhao J, Cui W, Qu H (2017) Visual analysis of MOOC forums with iForum. IEEE Trans Visual Comput Graph 23(1):201–210

    Article  Google Scholar 

  • Guidotti R, Monreale A, Turini F, Pedreschi D, Giannotti F (2018) A survey of methods for explaining black box models. ACM Comput Surv 51(5):1–42

    Article  Google Scholar 

  • Hohman F, Park H, Robinson C, Chau DHP (2020) Summit: Scaling deep learning interpretability by visualizing activation and attribution summarizations. IEEE Trans Visual Comput Graph 26(1):1096–1106

    Article  Google Scholar 

  • Huijie Z, Ren K, Yiming L, Dezhan Q, Li Z (2019) Airinsight: visual exploration and interpretation of latent patterns and anomalies in air quality data. Sustainability 11(10):2944

    Article  Google Scholar 

  • Jiazhi X, Jie L, Siming C, Hongxing Q, Shixia L (2021) A survey on interdisciplinary research of visualization and artificial intelligence. Sci Sin Inf 51:1777–1801

    Article  Google Scholar 

  • Jin C (2020) Mooc student dropout prediction model based on learning behavior features and parameter optimization. Interact Learn Environ 1–19. https://doi.org/10.1080/10494820.2020.1802300

  • Jinghan B, Huijie Z, Dezhan Q, Cheng L, Weizhang S (2021) Fgvis: visual analytics of human mobility patterns and urban areas based on f-glove. J Visual 24:1319–1335

    Article  Google Scholar 

  • Kahng M, Andrews PY, Kalro A, Chau DHP (2018) Activis: Visual exploration of industry-scale deep neural network models. IEEE Trans Visual Comput Graph 24(1):88–97

    Article  Google Scholar 

  • Chiu TKF, Hew TKF (2018) Factors influencing peer learning and performance in mooc asynchronous online discussion forum. Austral J Educ Technol 34(4):16–28

    Google Scholar 

  • Kőrösi G, Farkas R (2020) Mooc performance prediction by deep learning from raw clickstream data. In: International conference on advances in computing and data sciences. Springer, Berlin, pp 474–485

  • Liu M, Shi J, Cao K, Zhu J, Liu S (2018) Analyzing the training processes of deep generative models. IEEE Trans Visual Comput Graph 24(1):77–87

    Article  Google Scholar 

  • Mehmet S (2021) A comparative analysis of dropout prediction in massive open online courses. Arab J Sci Eng 46:1845–1861

    Article  Google Scholar 

  • Ming Y, Cao S, Zhang R, Li Z, Chen Y, Song Y, Qu H (2017) Understanding hidden memories of recurrent neural networks. In: 12th IEEE conference on visual analytics science and technology, IEEE VAST 2017, Phoenix, AZ, USA, Oct 3–6, 2017. IEEE Computer Society, pp 13–24

  • Mothilal RK, Sharma A, Tan C (2020) Explaining machine learning classifiers through diverse counterfactual explanations. In: Proceedings of the 2020 conference on fairness, accountability, and transparency, pp 607–617

  • Mu X, Xu K, Chen Q, Du F, Wang Y, Qu H (2019) Moocad: visual analysis of anomalous learning activities in massive open online courses. In: 21st eurographics conference on visualization, EuroVis 2019—short papers, Porto, Portugal, June 3–7, 2019. Eurographics Association, pp 91–95

  • Pawelczyk M, Broelemann K, Kasneci G (2020) Learning model-agnostic counterfactual explanations for tabular data. In: Proceedings of the web conference 2020, pp 3126–3132

  • Prashan M, Tim M, Liz S, Frank V (2020) Explainable reinforcement learning through a causal lens. In: Proceedings of the AAAI conference on artificial intelligence, vol 34, pp 2493–2500, 04

  • Qian F, Zhanghao G, Junyi Z, Yafeng Z (2021) CLSA: a novel deep learning model for MOOC dropout prediction. Comput Electr Eng 94:107315

    Article  Google Scholar 

  • Reuben B, Max VK, Michael V, Ulrik L, Jun Z, Nigel S (2018) ’It’s reducing a human being to a percentage’: perceptions of justice in algorithmic decisions. In: Proceedings of the 2018 CHI conference on human factors in computing systems, pp 1–14

  • Siby P, Manish G (2020) Onet c a temporal meta embedding network for MOOC dropout prediction. In: 2018 IEEE global engineering education conference (EDUCON), pp 5209–5217, 12

  • Strobelt H, Gehrmann S, Pfister H, Rush AM (2018) LSTMVIS: a tool for visual analysis of hidden state dynamics in recurrent neural networks. IEEE Trans Visual Comput Graph 24(1):667–676

    Article  Google Scholar 

  • Sun Z, Harit A, Yu J, Cristea AI, Shi L (2021) A brief survey of deep learning approaches for learning analytics on MOOCs. In: International conference on intelligent tutoring systems. Springer, Berlin, pp 28–37

  • Tharindu PP, Liyanagunawardena R, Williams S (2014) Dropout: Mooc participants perspective. pp 95–100, 02

  • Tulio RM, Sameer S, Carlos G (2016) “Why should I trust you?” Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp 1135–1144, 08

  • Verma S, Dickerson J, Hines K (2020) Counterfactual explanations for machine learning: a review. arXiv preprint arXiv:2010.10596

  • Wachter S, Mittelstadt B, Russell C (2017) Counterfactual explanations without opening the black box: automated decisions and the GDPR. JOLT 31:841

    Google Scholar 

  • Wang L, Wang H(2019) Learning behavior analysis and dropout rate prediction based on MOOCs data. In: 2019 10th international conference on information technology in medicine and education (ITME). IEEE, pp 419–423

  • Wang W, Yu H, Miao C (2017) Deep model for dropout prediction in MOOCs. In: Proceedings of the 2nd international conference on crowd science and engineering, pp 26–32

  • Wangli X, Dongping D (2018) Dropout prediction in MOOCs: using deep learning for personalized intervention. J Educ Comput Res 57:073563311875701

    Google Scholar 

  • Wen Y, Tian Y, Wen B, Zhou Q, Cai G, Liu S (2019) Consideration of the local correlation of learning behaviors to predict dropouts from MOOCs. Tsinghua Sci Technol 25(3):336–347

    Article  Google Scholar 

  • Wenzheng F, Jie T, Tracy L (2019) Understanding dropouts in MOOCs. In: Proceedings of the AAAI conference on artificial intelligence, vol 33, pp 517–524, 07

  • Wong J-S et al (2018) Messagelens: a visual analytics system to support multifaceted exploration of MOOC forum discussions. Visual Inform 2(1):37–49

    Article  Google Scholar 

  • Wu N, Zhang L, Gao Y, Zhang M, Sun X, Feng J (2019) CLMS-Net: dropout prediction in MOOCs with deep learning. In: Proceedings of the ACM turing celebration conference, China, pp 1–6

  • Xia M, Sun M, Wei H, Chen Q, Wang Y, Shi L, Qu H, Ma X (2019) Peerlens: peer-inspired interactive learning path planning in online question pool. In: Proceedings of the 2019 CHI conference on human factors in computing systems, CHI 2019, Glasgow, Scotland, UK, May 04-09. ACM, p 634

  • Xia M, Velumani RP, Wang Y, Qu H, Ma X (2021) Qlens: visual analytics of multi-step problem-solving behaviors for improving question design. IEEE Trans Visual Comput Graph 27(2):870–880

    Article  Google Scholar 

  • Xia M, Xu M, Lin C, Cheng TY, Qu H, Ma X (2020) Seqdynamics: visual analytics for evaluating online problem–solving dynamics. Comput Graph Forum 39(3):511–522

    Article  Google Scholar 

  • Xiong F, Zou K, Liu Z, Wang H (2019) Predicting learning status in MOOCs using LSTM. In: Proceedings of the ACM turing celebration conference, China, pp 1–5

  • Xueyu W, Gang Z, Xiao L (2017) Research on the learners’ dropout prediction based on the MOOC data. Mod Educ Technol 27(06):94–100

    Google Scholar 

  • Ye C , Biswas G (2014) Early prediction of student dropout and performance in MOOCs using higher granularity temporal information. J Learn Anal 1(3):169–172

    Article  Google Scholar 

  • Yonggu W, Qing Z (2014) MOOC: characteristics and learning mechanism. Educ Res 35(09):112–120133

    Google Scholar 

  • Yu C-H, Wu J, Liu A-C (2019) Predicting learning outcomes with MOOC clickstreams. Educ Sci 9(2):104

    Article  Google Scholar 

  • Zaporozhko VV, Parfenov DI Shardakov VM (2019) Development approach of formation of individual educational trajectories based on neural network prediction of student learning outcomes. In: International conference of artificial intelligence, medical engineering, education. Springer, Berlin, pp 305–314

  • Zhao Y, Shi J, Liu J, Zhao J, Zhou F, Zhang W, Chen K, Zhao X, Zhu C, Chen W (2021) Evaluating effects of background stories on graph perception. IEEE Trans Visual Comput Graph 28:4839–4854

    Article  Google Scholar 

  • Zhenguo X, Guanwen Z, Lin S, Jing A (2017) Research on the factors affecting the MOOC learner’ dropout behavior. Mod Educ Technol 27(09):100–106

    Google Scholar 

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China under Grant 42171450, the Key Research and Development Project of Science and Technology Development Plan of Science and Technology Department of Jilin Province No. 20210201074GX, National Natural Science Foundation of China under Grant 41671379 and National Key R&D Program of China No. 2020YFA0714102.

Author information

Authors and Affiliations

  1. School of Information Science and Technology, Northeast Normal University, Changchun, China

    Huijie Zhang, Jialu Dong, Cheng Lv, Yiming Lin & Jinghan Bai

  2. Key Laboratory of Intelligent Information Processing of Jilin Universities, Changchun, 130024, China

    Huijie Zhang, Jialu Dong, Cheng Lv, Yiming Lin & Jinghan Bai

Authors
  1. Huijie Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jialu Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Cheng Lv
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yiming Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jinghan Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huijie Zhang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, H., Dong, J., Lv, C. et al. Visual analytics of potential dropout behavior patterns in online learning based on counterfactual explanation. J Vis 26, 723–741 (2023). https://doi.org/10.1007/s12650-022-00899-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12650-022-00899-8

Keywords

Search

Navigation