Skip to main content
SpringerLink
Log in

Modeling learner’s dynamic knowledge construction procedure and cognitive item difficulty for knowledge tracing

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Knowledge tracing (KT) is essential for adaptive learning to obtain learners’ current states of knowledge for the purpose of providing adaptive service. Generally, the knowledge construction procedure is constantly evolving because students dynamically learn and forget over time. Unfortunately, to the best of our knowledge most existing approaches consider only a fragment of the information that relates to learning or forgetting, and the problem of making use of rich information during learners’ learning interactions to achieve more precise prediction of learner performance in KT remains under-explored. Moreover, existing work either neglects the problem difficulty or assumes that it is constant, and this is unrealistic in the actual learning process as problem difficulty affects performance undoubtedly and also varies overtime in terms of the cognitive challenge it presents to individual learners. To this end, we herein propose a novel model, KTM-DLF (Knowledge Tracing Machine by modeling cognitive item Difficulty and Learning and Forgetting), to trace the evolution of each learner’s knowledge acquisition during exercise activities by modeling his or her dynamic knowledge construction procedure and cognitive item difficulty. Specifically, we first specify the concept of cognitive item difficulty and propose a method to model the cognitive item difficulty adaptively based on learners’ learning histories. Then, based on two classical theories (the learning curve theory and the Ebbinghaus forgetting curve theory), we propose a method for modeling learners’ learning and forgetting over time. Finally, the KTM-DLF model is proposed to incorporate learners’ abilities, the cognitive item difficulty, and the two dynamic procedures (learning and forgetting) together. We then use the factorization machine framework to embed features in high dimensions and model pairwise interactions to increase the model’s accuracy. Extensive experiments have been conducted on three public real-world datasets, and the results confirm that our proposed model outperforms the other state-of-the-art educational data mining models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. KCs are atomistic components of knowledge in a domain; in KT, an item is usually tagged by the involved KCs to solve the item.

  2. This paper will interchangeably use “student”, “user” and “learner”.

  3. We will interchangeably refer to exercises as questions, items or problems.

  4. The skills involved in an observation of a student over a question are those regarding KCs, in this paper skill and KC will be used interchangeably.

  5. https://github.com/jfloff/pywFM

  6. https://github.com/jilljenn/ktm

  7. https://github.com/BenoitChoffin/das3h

References

  1. Abdelrahman G, Wang Q (2019) Knowledge tracing with sequential key-value memory networks. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, New York, NY, USA, pp 175–184

  2. Adamopoulos P (2013) What makes a great mooc? an interdisciplinary analysis of student retention in online courses. In: Proceedings of the 34th International Conference on Information Systems, pp. 1–21. Association for Information Systems

  3. Adler PS, Clark KB (1991) Behind the learning curve: a sketch of the learning process. Manag Sci 37(3):267–281

    Google Scholar 

  4. Ai F, Chen Y, Guo Y, Zhao Y, Wang Z, Fu G, Wang G (2019) Concept-aware deep knowledge tracing and exercise recommendation in an online learning system. In: Proceedings of The 12th International Conference on Educational Data Mining, pp. 240–245

  5. Anderson JR, Corbett AT, Koedinger KR, Pelletier R (1995) Cognitive tutors: Lessons learned. The journal of the learning sciences 4(2):167–207

    Google Scholar 

  6. d Baker RS, Corbett AT, Aleven V (2008) More accurate student modeling through contextual estimation of slip and guess probabilities in bayesian knowledge tracing. In: International conference on intelligent tutoring systems, pp 406-415, Springer, Berlin, Heidelberg

  7. Cen H, Koedinger K, Junker B (2006) Learning factors analysis–a general method for cognitive model evaluation and improvement. In: Ikeda M, Ashley KD, Chan TW (eds) Intelligent tutoring systems, ITS 2006, pp 164-175, Springer, Berlin, Heidelberg

  8. Chen Y, Liu Q, Huang Z, Wu L, Chen E, Wu R, Su Y, Hu G (2017) Tracking knowledge proficiency of students with educational priors. In: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, pp. 989–998. ACM

  9. Choffin B, Popineau F, Bourda Y, Vie JJ (2019) DAS3h: Modeling student learning and forgetting for optimally scheduling distributed practice of skills. In: Proceedings of the 12th International Conference on Educational Data Mining (EDM), pp. 29–38

  10. Corbett AT, Anderson JR (1994) Knowledge tracing: Modeling the acquisition of procedural knowledge. User modeling and user-adapted interaction 4(4):253–278

    Google Scholar 

  11. Daroczy G, Wolska M, Meurers WD, Nuerk HC (2015) Word problems: a review of linguistic and numerical factors contributing to their difficulty. Frontiers in psychology 6:348

    Google Scholar 

  12. De La Torre J (2009) Dina model and parameter estimation: a didactic. Journal of educational and behavioral statistics 34(1):115–130

    MathSciNet  Google Scholar 

  13. Delacre M, Lakens D, Leys C (2017) Why psychologists should by default use welch’s t-test instead of student’s t-test. International Review of Social Psychology 30(1):92–101

    Google Scholar 

  14. Desmarais MC, Baker RS (2012) A review of recent advances in learner and skill modeling in intelligent learning environments. User Model User-Adap Inter 22:9–38

    Google Scholar 

  15. Ebbinghaus H (2013) Memory: a contribution to experimental psychology. Annals of neurosciences 20(4):155–156

    Google Scholar 

  16. Feng M, Heffernan N, Koedinger K (2009) Addressing the assessment challenge with an online system that tutors as it assesses. User Model User-Adap Inter 19(3):243–266

    Google Scholar 

  17. Gan W, Sun Y, Ye S, Fan Y, Sun Y (2019) Field-aware knowledge tracing machine by modelling students’ dynamic learning procedure and item difficulty. In: 2019 International conference on data mining workshops (ICDMW), pp. 1045–1046. IEEE

  18. Gan W, Sun Y, Ye S, Fan Y, Sun Y (2019) AI-tutor: Generating tailored remedial questions and answers based on cognitive diagnostic assessment. In: 2019 6Th international conference on behavioral, economic and socio-cultural computing (BESC), pp. 1–6. IEEE

  19. Ha H, Hwang U, Hong Y, Yoon S (2018) Memory-augmented neural networks for knowledge tracing from the perspective of learning and forgetting. arXiv:1805.10768

  20. Huang Z, Yin Y, Chen E, Xiong H, Su Y, Hu G, et al. (2019) EKT: Exercise-aware knowledge tracing for student performance prediction. IEEE Trans Knowl Data Eng, pp 1

  21. Khajah M, Lindsey RV, Mozer MC (2016) How deep is knowledge tracing?. In: Proceedings of the 9th International Conference on Educational Data Mining, pp. 94–101

  22. Koedinger KR, Brunskill E, Baker RS, McLaughlin EA, Stamper J (2013) New potentials for data-driven intelligent tutoring system development and optimization. AI Magazine 34(3):27–41

    Google Scholar 

  23. Kotovsky K, Hayes JR, Simon HA (1985) Why are some problems hard? evidence from tower of hanoi. Cognitive psychology 17(2):248–294

    Google Scholar 

  24. Kotovsky K, Simon HA (1990) What makes some problems really hard: Explorations in the problem space of difficulty. Cognitive psychology 22(2):143–183

    Google Scholar 

  25. Kubinger KD, Gottschall CH (2007) Item difficulty of multiple choice tests dependant on different item response formats–an experiment in fundamental research on psychological assessment. Psychology science 49 (4):361–374

    Google Scholar 

  26. Lindsey RV, Shroyer JD, Pashler H, Mozer MC (2014) Improving students’ long-term knowledge retention through personalized review. Psychological science 25(3):639–647

    Google Scholar 

  27. Liu Q, Wu R, Chen E, Xu G, Su Y, Chen Z, Hu G (2018) Fuzzy cognitive diagnosis for modelling examinee performance. ACM Transactions on Intelligent Systems and Technology 9(4):1–26

    Google Scholar 

  28. Minn S, Yu Y, Desmarais MC, Zhu F, Vie JJ (2018) Deep knowledge tracing and dynamic student classification for knowledge tracing. In: 2018 IEEE International conference on data mining (ICDM), pp. 1182–1187. IEEE

  29. Minn S, Zhu F, Desmarais MC (2018) Improving knowledge tracing model by integrating problem difficulty. In: 2018 IEEE International conference on data mining workshops (ICDMW), pp. 1505–1506. IEEE

  30. Nagatani K, Zhang Q, Sato M, Chen YY, Chen F, Ohkuma T (2019) Augmenting knowledge tracing by considering forgetting behavior. In: The world wide web conference, pp. 3101–3107. ACM

  31. Pardos ZA, Bergner Y, Seaton DT, Pritchard DE (2013) Adapting bayesian knowledge tracing to a massive open online course in edx. In: Proceedings of the 6th Annual International Conference on Educational Data Mining, pp. 137–144

  32. Pardos ZA, Heffernan NT (2011) KT-IDEM: introducing item difficulty to the knowledge tracing model. In: International conference on user modeling, adaptation, and personalization, pp. 243–254. Springer

  33. Pavlik PI, Cen H, Koedinger KR (2009) Performance factors analysis–a new alternative to knowledge tracing, pp 531–538

  34. Piech C, Bassen J, Huang J, Ganguli S, Sahami M, Guibas LJ, Sohl-Dickstein J (2015) Deep knowledge tracing. In: Advances in neural information processing systems, pp. 505–513

  35. Rasch G (1960) Studies in mathematical psychology: I. Probabilistic models for some intelligence and attainment tests Nielsen & Lydiche

  36. Rathore AS, Arjaria SK (2020) Intelligent tutoring system. In: Utilizing educational data mining techniques for improved learning: emerging research and opportunities, pp. 121–144. IGI global

  37. Reckase MD, McKinley RL (1991) The discriminating power of items that measure more than one dimension. Applied psychological measurement 15(4):361–373

    Google Scholar 

  38. Rendle S (2010) Factorization machines. In: 2010 IEEE International conference on data mining, pp. 995–1000. IEEE

  39. Rendle S (2012) Factorization machines with libfm. ACM Transactions on Intelligent Systems and Technology 3(3):57

    Google Scholar 

  40. Saranti A, Taraghi B, Ebner M, Holzinger A (2019) Insights into learning competence through probabilistic graphical models. In: International cross-domain conference for machine learning and knowledge extraction, pp. 250–271. Springer

  41. Settles B, Meeder B (2016) A trainable spaced repetition model for language learning. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1848–1858

  42. Stamper J, Niculescu-Mizil A, Ritter S, Gordon G, Koedinger K (2010) Algebra i 2005-2006 and bridge to algebra 2006-2007. challenge data set from kdd cup 2010 educational data mining challenge. Find it at, http://pslcdatashop.web.cmu.edu/KDDCup/downloads.jsp

  43. Sun Y, Ye S, Inoue S, Sun Y (2014) Alternating recursive method for q-matrix learning. In: Proceedings of the 7th International Conference on Educational Data Mining, pp. 14–20

  44. Thai-Nghe N, Schmidt-Thieme L (2015) Multi-relational factorization models for student modeling in intelligent tutoring systems. In: 2015 Seventh international conference on knowledge and systems engineering (KSE), pp. 61–66. IEEE

  45. Vie JJ, Kashima H (2019) Knowledge tracing machines: Factorization machines for knowledge tracing. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 750–757

  46. Wang X, Berger JO, Burdick DS, et al. (2013) Bayesian analysis of dynamic item response models in educational testing. The Annals of Applied Statistics 7(1):126–153

    MathSciNet  MATH  Google Scholar 

  47. Wang Z, Feng X, Tang J, Huang GY, Liu Z (2019) Deep knowledge tracing with side information. In: International conference on artificial intelligence in education, pp. 303–308. Springer

  48. Wang Z, Zhu J, Li X, Hu Z, Zhang M (2016) Structured knowledge tracing models for student assessment on coursera. In: Proceedings of the Third ACM Conference on Learning at Scale, pp. 209–212. ACM

  49. Wilson KH, Karklin Y, Han B, Ekanadham C (2016) Back to the basics: Bayesian extensions of irt outperform neural networks for proficiency estimation. In: Proceedings of the 9th International Conference on Educational Data Mining, pp. 539–544

  50. Wilson KH, Xiong X, Khajah M, Lindsey RV, Zhao S, Karklin Y, Van Inwegen EG, Han B, Ekanadham C, Beck JE, et al. (2016) Estimating student proficiency: Deep learning is not the panacea. In: In 30th conference on neural information processing systems, workshop on machine learning for education, pp. 1–8

  51. Yeung CK (2019) Deep-IRT,: Make deep learning based knowledge tracing explainable using item response theory. arXiv:1904.11738

  52. Zhang J, Huang T, Zhang Z (2019) FAT-DeepFFM,: Field attentive deep field-aware factorization machine. arXiv:1905.06336

  53. Zhang J, Shi X, King I, Yeung DY (2017) Dynamic key-value memory networks for knowledge tracing. In: Proceedings of the 26th international conference on World Wide Web, pp. 765–774. International World Wide Web Conferences Steering Committee

Download references

Acknowledgements

This work is supported by JSPS KAKENHI Grant Number 18K11597. The authors also gratefully acknowledge the helpful comments and suggestions of the anonymous reviewers.

Author information

Authors and Affiliations

  1. National Institute of Informatics, SOKENDAI, Tokyo, Japan

    Wenbin Gan & Yuan Sun

  2. Zhejiang University, Hangzhou, China

    Xian Peng

  3. University of Chinese Academy of Sciences, Beijing, China

    Yi Sun

Authors
  1. Wenbin Gan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuan Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xian Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yi Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wenbin Gan.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gan, W., Sun, Y., Peng, X. et al. Modeling learner’s dynamic knowledge construction procedure and cognitive item difficulty for knowledge tracing. Appl Intell 50, 3894–3912 (2020). https://doi.org/10.1007/s10489-020-01756-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-01756-7

Keywords

Search

Navigation