skip to main content
10.1145/3404835.3462827acmconferencesArticle/Chapter ViewAbstractPublication PagesirConference Proceedingsconference-collections
research-article

Tracing Knowledge State with Individual Cognition and Acquisition Estimation

Published: 11 July 2021 Publication History

Abstract

Knowledge tracing, which dynamically estimates students' learning states by predicting their performance on answering questions, is an essential task in online education. One typical solution for knowledge tracing is based on Recurrent Neural Networks (RNNs), which represent students' knowledge states with the hidden states of RNNs. Such type of methods normally assumes that students have the same cognition level and knowledge acquisition sensitivity on the same question. Thus, they (i) predict students' responses by referring to their knowledge states and question representations, and (ii) update the knowledge states according to the question representations and students' responses. No explicit cognition level or knowledge acquisition sensitivity is considered in the above two processes. However, in real-world scenarios, students have different understandings on a question and have various knowledge acquisition after they finish the same question. In this paper, we propose a novel model called Individual Estimation Knowledge Tracing (IEKT), which estimates the students' cognition on the question before response prediction and assesses their knowledge acquisition sensitivity on the questions before updating the knowledge state. In the experiments, we compare IEKT with 11 knowledge tracing baselines on four benchmark datasets, and the results show IEKT achieves the state-of-the-art performance.

Supplementary Material

MP4 File (iekt-video.mp4)
Presentation video.

References

[1]
Ghodai Abdelrahman and Qing Wang. 2019. Knowledge Tracing with Sequential Key-Value Memory Networks. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 175--184.
[2]
Herman Aguinis and Steven A Branstetter. 2007. Teaching the concept of the sampling distribution of the mean. Journal of Management Education, Vol. 31, 4 (2007), 467--483.
[3]
Bhaskar Bhattacharya and Desale Habtzghi. 2002. Median of the p value under the alternative hypothesis. The American Statistician, Vol. 56, 3 (2002), 202--206.
[4]
Haw-Shiuan Chang, Hwai-Jung Hsu, and Kuan-Ta Chen. 2015. Modeling Exercise Relationships in E-Learning: A Unified Approach. In EDM. 532--535.
[5]
Penghe Chen, Yu Lu, Vincent W Zheng, and Yang Pian. 2018. Prerequisite-driven deep knowledge tracing. In 2018 IEEE International Conference on Data Mining (ICDM). IEEE, 39--48.
[6]
Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014).
[7]
Youngduck Choi, Youngnam Lee, Dongmin Shin, Junghyun Cho, Seoyon Park, Seewoo Lee, Jineon Baek, Chan Bae, Byungsoo Kim, and Jaewe Heo. 2020. Ednet: A large-scale hierarchical dataset in education. In International Conference on Artificial Intelligence in Education. Springer, 69--73.
[8]
Albert T Corbett and John R Anderson. 1994. Knowledge tracing: Modeling the acquisition of procedural knowledge. User modeling and user-adapted interaction, Vol. 4, 4 (1994), 253--278.
[9]
Paul De Boeck and Mark Wilson. 2004. Explanatory item response models: A generalized linear and nonlinear approach. Springer Science & Business Media.
[10]
Phung Do, Kha Nguyen, Thanh Nguyen Vu, Tran Nam Dung, and Tuan Dinh Le. 2017. Integrating knowledge-based reasoning algorithms and collaborative filtering into e-learning material recommendation system. In International Conference on Future Data and Security Engineering. Springer, 419--432.
[11]
Neil J Dorans. 2002. The recentering of SAT® scales and its effects on score distributions and score interpretations. ETS Research Report Series, Vol. 2002, 1 (2002), i--21.
[12]
Pragya Dwivedi, Vibhor Kant, and Kamal K Bharadwaj. 2018. Learning path recommendation based on modified variable length genetic algorithm. Education and Information Technologies, Vol. 23, 2 (2018), 819--836.
[13]
Mingyu Feng, Neil Heffernan, and Kenneth Koedinger. 2009. Addressing the assessment challenge with an online system that tutors as it assesses. User Modeling and User-Adapted Interaction, Vol. 19, 3 (2009), 243--266.
[14]
Howard Gardner et al. 1992. Multiple intelligences. Vol. 5. Minnesota Center for Arts Education.
[15]
Aritra Ghosh, Neil Heffernan, and Andrew S Lan. 2020. Context-aware attentive knowledge tracing. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2330--2339.
[16]
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, Vol. 9, 8 (1997), 1735--1780.
[17]
Zhenya Huang, Yu Yin, Enhong Chen, Hui Xiong, Yu Su, Guoping Hu, et al. 2019. Ekt: Exercise-aware knowledge tracing for student performance prediction. IEEE Transactions on Knowledge and Data Engineering (2019).
[18]
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
[19]
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems. 1097--1105.
[20]
Qi Liu, Shiwei Tong, Chuanren Liu, Hongke Zhao, Enhong Chen, Haiping Ma, and Shijin Wang. 2019. Exploiting Cognitive Structure for Adaptive Learning. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 627--635.
[21]
Yunfei Liu, Yang Yang, Chen Xianyu, Shen Jian, Zhang Haifeng, and Yu Yong. 2020. Improving Knowledge Tracing via Pre-training Question Embeddings. In Twenty-Ninth International Joint Conference on Artificial Intelligence.
[22]
Yu Lu, Deliang Wang, Qinggang Meng, and Penghe Chen. 2020. Towards Interpretable Deep Learning Models for Knowledge Tracing. arXiv preprint arXiv:2005.06139 (2020).
[23]
Simon J Mason and Nicholas E Graham. 2002. Areas beneath the relative operating characteristics (ROC) and relative operating levels (ROL) curves: Statistical significance and interpretation. Quarterly Journal of the Royal Meteorological Society: A journal of the atmospheric sciences, applied meteorology and physical oceanography, Vol. 128, 584 (2002), 2145--2166.
[24]
Koki Nagatani, Qian Zhang, Masahiro Sato, Yan-Ying Chen, Francine Chen, and Tomoko Ohkuma. 2019. Augmenting knowledge tracing by considering forgetting behavior. In The World Wide Web Conference. 3101--3107.
[25]
Hiromi Nakagawa, Yusuke Iwasawa, and Yutaka Matsuo. 2019. Graph-based Knowledge Tracing: Modeling Student Proficiency Using Graph Neural Network. In 2019 IEEE/WIC/ACM International Conference on Web Intelligence (WI). IEEE, 156--163.
[26]
Shalini Pandey and George Karypis. 2019. A Self-Attentive model for Knowledge Tracing. arXiv preprint arXiv:1907.06837 (2019).
[27]
Chris Piech, Jonathan Bassen, Jonathan Huang, Surya Ganguli, Mehran Sahami, Leonidas J Guibas, and Jascha Sohl-Dickstein. 2015. Deep knowledge tracing. In Advances in neural information processing systems. 505--513.
[28]
LJ Salomon, JP Bernard, and Y Ville. 2005. Analysis of Z-score distribution for the quality control of fetal ultrasound measurements at 20--24 weeks. Ultrasound in Obstetrics and Gynecology: The Official Journal of the International Society of Ultrasound in Obstetrics and Gynecology, Vol. 26, 7 (2005), 750--754.
[29]
Shuanghong Shen, Qi Liu, Enhong Chen, Han Wu, Zhenya Huang, Weihao Zhao, Yu Su, Haiping Ma, and Shijin Wang. 2020. Convolutional Knowledge Tracing: Modeling Individualization in Student Learning Process. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 1857--1860.
[30]
Yu Su, Qingwen Liu, Qi Liu, Zhenya Huang, Yu Yin, Enhong Chen, Chris Ding, Si Wei, and Guoping Hu. 2018. Exercise-enhanced sequential modeling for student performance prediction. In Thirty-Second AAAI Conference on Artificial Intelligence.
[31]
Richard S Sutton, David A McAllester, Satinder P Singh, Yishay Mansour, et al. 1999. Policy gradient methods for reinforcement learning with function approximation. In NIPs, Vol. 99. Citeseer, 1057--1063.
[32]
Hanshuang Tong, Yun Zhou, and Zhen Wang. 2020. HGKT: Introducing Problem Schema with Hierarchical Exercise Graph for Knowledge Tracing. arXiv preprint arXiv:2006.16915 (2020).
[33]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998--6008.
[34]
Jill-Jênn Vie and Hisashi Kashima. 2019. Knowledge tracing machines: Factorization machines for knowledge tracing. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 750--757.
[35]
Shuhan Wang, Hao Wu, Ji Hun Kim, and Erik Andersen. 2019 b. Adaptive learning material recommendation in online language education. In International Conference on Artificial Intelligence in Education. Springer, 298--302.
[36]
Tianqi Wang, Fenglong Ma, and Jing Gao. 2019 a. Deep hierarchical knowledge tracing. In Proceedings of the 12th International Conference on Educational Data Mining.
[37]
Kevin H Wilson, Yan Karklin, Bojian Han, and Chaitanya Ekanadham. 2016a. Back to the basics: Bayesian extensions of IRT outperform neural networks for proficiency estimation. arXiv preprint arXiv:1604.02336 (2016).
[38]
Kevin H Wilson, Xiaolu Xiong, Mohammad Khajah, Robert V Lindsey, Siyuan Zhao, Yan Karklin, Eric G Van Inwegen, Bojian Han, Chaitanya Ekanadham, Joseph E Beck, et al. 2016b. Estimating student proficiency: Deep learning is not the panacea. In In Neural Information Processing Systems, Workshop on Machine Learning for Education. 3.
[39]
Shanghui Yang, Mengxia Zhu, Jingyang Hou, and Xuesong Lu. 2020. Deep Knowledge Tracing with Convolutions. arXiv preprint arXiv:2008.01169 (2020).
[40]
Michael V Yudelson, Kenneth R Koedinger, and Geoffrey J Gordon. 2013. Individualized bayesian knowledge tracing models. In International conference on artificial intelligence in education. Springer, 171--180.
[41]
Jiani Zhang, Xingjian Shi, Irwin King, and Dit-Yan Yeung. 2017. Dynamic key-value memory networks for knowledge tracing. In Proceedings of the 26th international conference on World Wide Web. 765--774.

Cited By

View all
  • (2025)MAHKT: Knowledge tracing with multi-association heterogeneous graph embedding based on knowledge transferKnowledge-Based Systems10.1016/j.knosys.2025.112958310(112958)Online publication date: Feb-2025
  • (2025)EduStudio: towards a unified library for student cognitive modelingFrontiers of Computer Science: Selected Publications from Chinese Universities10.1007/s11704-024-40372-319:8Online publication date: 1-Aug-2025
  • (2025)Modeling Learning Transfer Effects in Knowledge Tracing: A Dynamic and Bidirectional PerspectiveDatabase Systems for Advanced Applications10.1007/978-981-97-5779-4_7(99-114)Online publication date: 11-Jan-2025
  • Show More Cited By

Index Terms

  1. Tracing Knowledge State with Individual Cognition and Acquisition Estimation

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval
      July 2021
      2998 pages
      ISBN:9781450380379
      DOI:10.1145/3404835
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 11 July 2021

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. cognition
      2. knowledge acquisition sensitivity
      3. knowledge tracing
      4. reinforcement learning

      Qualifiers

      • Research-article

      Funding Sources

      Conference

      SIGIR '21
      Sponsor:

      Acceptance Rates

      Overall Acceptance Rate 792 of 3,983 submissions, 20%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)141
      • Downloads (Last 6 weeks)26
      Reflects downloads up to 14 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2025)MAHKT: Knowledge tracing with multi-association heterogeneous graph embedding based on knowledge transferKnowledge-Based Systems10.1016/j.knosys.2025.112958310(112958)Online publication date: Feb-2025
      • (2025)EduStudio: towards a unified library for student cognitive modelingFrontiers of Computer Science: Selected Publications from Chinese Universities10.1007/s11704-024-40372-319:8Online publication date: 1-Aug-2025
      • (2025)Modeling Learning Transfer Effects in Knowledge Tracing: A Dynamic and Bidirectional PerspectiveDatabase Systems for Advanced Applications10.1007/978-981-97-5779-4_7(99-114)Online publication date: 11-Jan-2025
      • (2024)A Question-Centric Multi-Experts Contrastive Learning Framework for Improving the Accuracy and Interpretability of Deep Sequential Knowledge Tracing ModelsACM Transactions on Knowledge Discovery from Data10.1145/367484019:2(1-25)Online publication date: 28-Jun-2024
      • (2024)Revisiting Knowledge Tracing: A Simple and Powerful ModelProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681205(263-272)Online publication date: 28-Oct-2024
      • (2024)DGEKT: A Dual Graph Ensemble Learning Method for Knowledge TracingACM Transactions on Information Systems10.1145/363835042:3(1-24)Online publication date: 22-Jan-2024
      • (2024)DyGKT: Dynamic Graph Learning for Knowledge TracingProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671773(409-420)Online publication date: 25-Aug-2024
      • (2024)Towards Multi-Objective Behavior and Knowledge Modeling in StudentsAdjunct Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization10.1145/3631700.3664880(183-188)Online publication date: 27-Jun-2024
      • (2024)SINKT: A Structure-Aware Inductive Knowledge Tracing Model with Large Language ModelProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679760(632-642)Online publication date: 21-Oct-2024
      • (2024)Hybrid Models for Knowledge Tracing: A Systematic Literature ReviewIEEE Transactions on Learning Technologies10.1109/TLT.2023.334869017(1021-1036)Online publication date: 1-Jan-2024
      • Show More Cited By

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media