skip to main content
10.1145/3589334.3645718acmconferencesArticle/Chapter ViewAbstractPublication PagesthewebconfConference Proceedingsconference-collections
research-article

HD-KT: Advancing Robust Knowledge Tracing via Anomalous Learning Interaction Detection

Published: 13 May 2024 Publication History

Abstract

Knowledge tracing (KT) is a crucial task in online learning, aimed at tracing and predicting each student's knowledge states throughout their learning process. Over the past decade, it has garnered widespread attention due to it provides the potential for more tailored and adaptive online learning experiences. Although most current KT methodologies emphasize optimizing network structures to enhance predictive accuracy for future student performance, they often neglect anomalous interactions in students' learning processes, which may arise from low data quality (i.e., inferior question quality) and abnormal student behaviors (i.e., guessing and mistakes). To this end, in this paper, we propose a novel framework, termed HD-KT, designed to enhance the robustness of existing KT methodologies with Hybrid learning interactions Denoising approach. Specifically, we introduce two detectors for anomalous learning interactions, namely knowledge state-guided anomaly detector and student profile-guided anomaly detector. In the first detection module, we design a sequential autoencoder to identify anomalous learning interactions by detecting atypical student knowledge states. In the second module, we incorporate an attention mechanism by modeling a student's long-term profile to capture irregular interactions. Extensive experiments on four real-world benchmark datasets have decisively shown our HD-KT markedly boosts the robustness of numerous prevailing KT models, consequently increasing the accuracy of future student performance predictions. Additionally, our case studies highlight the versatility of HD-KT in addressing diverse downstream tasks, such as exercise quality analysis and learning behavior-based student clustering.

Supplemental Material

MP4 File
Video presentation
MP4 File
Supplemental video

References

[1]
Ahmed Abdulaal, Zhuanghua Liu, and Tomer Lancewicki. 2021. Practical approach to asynchronous multivariate time series anomaly detection and localization. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, 2485--2494.
[2]
Andrea Borghesi, Andrea Bartolini, Michele Lombardi, Michela Milano, and Luca Benini. 2019. Anomaly detection using autoencoders in high performance computing systems. In Proceedings of the AAAI Conference on artificial intelligence number 01. Vol. 33, 9428--9433.
[3]
Haw-Shiuan Chang, Hwai-Jung Hsu, and Kuan-Ta Chen. 2015. Modeling exercise relationships in e-learning: a unified approach. In EDM, 532--535.
[4]
Albert T Corbett and John R Anderson. 1994. Knowledge tracing: modeling the acquisition of procedural knowledge. User modeling and user-adapted interaction, 4, 253--278.
[5]
Le Dai, Yu Yin, Chuan Qin, Tong Xu, Xiangnan He, Enhong Chen, and Hui Xiong. 2020. Enterprise cooperation and competition analysis with a signoriented preference network. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 774--782.
[6]
Ailin Deng and Bryan Hooi. 2021. Graph neural network-based anomaly detection in multivariate time series. In Proceedings of the AAAI conference on artificial intelligence number 5. Vol. 35, 4027--4035.
[7]
Leyan Deng, Defu Lian, Chenwang Wu, and Enhong Chen. 2022. Graph convolution network based recommender systems: learning guarantee and item mixture powered strategy. Advances in Neural Information Processing Systems, 35, 3900--3912.
[8]
Mingyu Feng, Neil Heffernan, and Kenneth Koedinger. 2009. Addressing the assessment challenge with an online system that tutors as it assesses. User modeling and user-adapted interaction, 19, 243--266.
[9]
Xiaopeng Guo, Zhijie Huang, Jie Gao, Mingyu Shang, Maojing Shu, and Jun Sun. 2021. Enhancing knowledge tracing via adversarial training. In Proceedings of the 29th ACM International Conference on Multimedia, 367--375.
[10]
Diederik P Kingma and Jimmy Ba. 2014. Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980.
[11]
Wonsung Lee, Jaeyoon Chun, Youngmin Lee, Kyoungsoo Park, and Sungrae Park. 2022. Contrastive learning for knowledge tracing. In Proceedings of the ACM Web Conference 2022, 2330--2338.
[12]
Dan Li, Dacheng Chen, Baihong Jin, Lei Shi, Jonathan Goh, and See-Kiong Ng. 2019. Mad-gan: multivariate anomaly detection for time series data with generative adversarial networks. In International conference on artificial neural networks. Springer, 703--716.
[13]
Shuhuan Liu, Xiaoshan Yu, Haiping Ma, Ziwen Wang, Chuan Qin, and Xingyi Zhang. 2023. Homogeneous cohort-aware group cognitive diagnosis: a multigrained modeling perspective. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, 4094--4098.
[14]
Haiping Ma, Changqian Wang, Hengshu Zhu, Shangshang Yang, Xiaoming Zhang, and Xingyi Zhang. 2023. Enhancing cognitive diagnosis using uninteracted exercises: a collaboration-aware mixed sampling approach. arXiv preprint arXiv:2312.10110.
[15]
Haiping Ma, Jingyuan Wang, Hengshu Zhu, Xin Xia, Haifeng Zhang, Xingyi Zhang, and Lei Zhang. 2022. Reconciling cognitive modeling with knowledge forgetting: a continuous time-aware neural network approach. In Proceedings of the 31st International Joint Conference on Artificial Intelligence, 2174--2181.
[16]
Haiping Ma, Jinwei Zhu, Shangshang Yang, Qi Liu, Haifeng Zhang, Xingyi Zhang, Yunbo Cao, and Xuemin Zhao. 2022. A prerequisite attention model for knowledge proficiency diagnosis of students. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, 4304--4308.
[17]
Radek Pelánek, and Vít Stanislav. 2016. Adaptive geography practice data set. Journal of Learning Analytics, 3, 2, 317--321.
[18]
Thanaporn Patikorn, Neil T Heffernan, and Ryan S Baker. 2018. Assistments longitudinal data mining competition 2017: a preface. In Proceedings of the Workshop on Scientific Findings from the ASSISTments Longitudinal Data Competition, International Conference on Educational Data Mining.
[19]
Philip I Pavlik Jr, Hao Cen, and Kenneth R Koedinger. 2009. Performance factors analysis--a new alternative to knowledge tracing. Online Submission.
[20]
Chris Piech, Jonathan Bassen, Jonathan Huang, Surya Ganguli, Mehran Sahami, Leonidas J Guibas, and Jascha Sohl-Dickstein. 2015. Deep knowledge tracing. Advances in neural information processing systems, 28.
[21]
Chuan Qin, Kaichun Yao, Hengshu Zhu, Tong Xu, Dazhong Shen, Enhong Chen, and Hui Xiong. 2022. Towards automatic job description generation with capability-aware neural networks. IEEE Transactions on Knowledge and Data Engineering, 35, 5, 5341--5355.
[22]
Chuan Qin, Le Zhang, Rui Zha, Dazhong Shen, Qi Zhang, Ying Sun, Chen Zhu, Hengshu Zhu, and Hui Xiong. 2023. A comprehensive survey of artificial intelligence techniques for talent analytics. arXiv preprint arXiv:2307.03195.
[23]
Dazhong Shen, Chuan Qin, Chao Wang, Zheng Dong, Hengshu Zhu, and Hui Xiong. 2021. Topic modeling revisited: a document graph-based neural network perspective. Advances in neural information processing systems, 34, 14681--14693.
[24]
Dazhong Shen, Chuan Qin, Chao Wang, Hengshu Zhu, Enhong Chen, and Hui Xiong. 2021. Regularizing variational autoencoder with diversity and uncertainty awareness. arXiv preprint arXiv:2110.12381.
[25]
Shuanghong Shen, Qi Liu, Enhong Chen, Zhenya Huang, Wei Huang, Yu Yin, Yu Su, and Shijin Wang. 2021. Learning process-consistent knowledge tracing. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, 1452--1460.
[26]
Ya Su, Youjian Zhao, Chenhao Niu, Rong Liu, Wei Sun, and Dan Pei. 2019. Robust anomaly detection for multivariate time series through stochastic recurrent neural network. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, 2828--2837.
[27]
Ying Sun, Fuzhen Zhuang, Hengshu Zhu, Qi Zhang, Qing He, and Hui Xiong. 2021. Market-oriented job skill valuation with cooperative composition neural network. Nature communications, 12, 1, 1992.
[28]
Chenyang Wang, Weizhi Ma, Min Zhang, Chuancheng Lv, Fengyuan Wan, Huijie Lin, Taoran Tang, Yiqun Liu, and Shaoping Ma. 2021. Temporal crosseffects in knowledge tracing. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining, 517--525.
[29]
FeiWang, Qi Liu, Enhong Chen, Zhenya Huang, Yu Yin, ShijinWang, and Yu Su. 2022. Neuralcd: a general framework for cognitive diagnosis. IEEE Transactions on Knowledge and Data Engineering.
[30]
Qinyong Wang, Hongzhi Yin, Hao Wang, Quoc Viet Hung Nguyen, Zi Huang, and Lizhen Cui. 2019. Enhancing collaborative filtering with generative augmentation. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 548--556.
[31]
Le Wu, Yong Ge, Qi Liu, Enhong Chen, Richang Hong, Junping Du, and Meng Wang. 2017. Modeling the evolution of users' preferences and social links in social networking services. IEEE Transactions on Knowledge and Data Engineering, 29, 6, 1240--1253.
[32]
Shangshang Yang, Haiping Ma, Cheng Zhen, Ye Tian, Limiao Zhang, Yaochu Jin, and Xingyi Zhang. 2023. Designing novel cognitive diagnosis models via evolutionary multi-objective neural architecture search. arXiv preprint arXiv:2307.04429.
[33]
Shangshang Yang, HaoyuWei, Haiping Ma, Ye Tian, Xingyi Zhang, Yunbo Cao, and Yaochu Jin. 2023. Cognitive diagnosis-based personalized exercise group assembly via a multi-objective evolutionary algorithm. IEEE Transactions on Emerging Topics in Computational Intelligence.
[34]
Shangshang Yang, Xiaoshan Yu, Ye Tian, Xueming Yan, Haiping Ma, and Xingyi Zhang. 2023. Evolutionary neural architecture search for transformer in knowledge tracing. In Thirty-seventh Conference on Neural Information Processing Systems.
[35]
Shangshang Yang, Cheng Zhen, Ye Tian, Haiping Ma, Yuanchao Liu, Panpan Zhang, and Xingyi Zhang. 2023. Evolutionary multi-objective neural architecture search for generalized cognitive diagnosis models. In 2023 5th International Conference on Data-driven Optimization of Complex Systems (DOCS). IEEE, 1--10.
[36]
Yang Yang, Jian Shen, Yanru Qu, Yunfei Liu, Kerong Wang, Yaoming Zhu, Weinan Zhang, and Yong Yu. 2021. Gikt: a graph-based interaction model for knowledge tracing. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2020, Ghent, Belgium, September 14--18, 2020, Proceedings, Part I. Springer, 299--315.
[37]
Yu Yin, Le Dai, Zhenya Huang, Shuanghong Shen, Fei Wang, Qi Liu, Enhong Chen, and Xin Li. 2023. Tracing knowledge instead of patterns: stable knowledge tracing with diagnostic transformer. In Proceedings of the ACM Web Conference 2023, 855--864.
[38]
Xiaoshan Yu, Chuan Qin, Dazhong Shen, Haiping Ma, Le Zhang, Xingyi Zhang, Hengshu Zhu, and Hui Xiong. 2024. Rdgt: enhancing group cognitive diagnosis with relation-guided dual-side graph transformer. IEEE Transactions on Knowledge and Data Engineering.
[39]
Jiahao Yuan, Zihan Song, Mingyou Sun, Xiaoling Wang, and Wayne Xin Zhao. 2021. Dual sparse attention network for session-based recommendation. In Proceedings of the AAAI conference on artificial intelligence number 5. Vol. 35, 4635--4643.
[40]
Michael V Yudelson, Kenneth R Koedinger, and Geoffrey J Gordon. 2013. Individualized bayesian knowledge tracing models. In Artificial Intelligence in Education: 16th International Conference, AIED 2013, Memphis, TN, USA, July 9--13, 2013. Proceedings 16. Springer, 171--180.
[41]
Yunfei Zhang, Chuan Qin, Dazhong Shen, Haiping Ma, Le Zhang, Xingyi Zhang, and Hengshu Zhu. 2023. Relicd: a reliable cognitive diagnosis framework with confidence awareness. arXiv preprint arXiv:2401.10749.
[42]
Hang Zhao et al. 2020. Multivariate time-series anomaly detection via graph attention network. In 2020 IEEE International Conference on Data Mining (ICDM). IEEE, 841--850.
[43]
HZhu, Y Zuo, C Zhu, J Wu, and H Xiong. 2017. Collaborative company profiling: insights from an employee's perspective. arXiv preprint arXiv:1712.02987.

Cited By

View all
  • (2025)Examination Process Modeling for Intelligent Patent Management: A Multi-aspect Neural Sequential ApproachACM Transactions on Management Information Systems10.1145/3712309Online publication date: 14-Jan-2025
  • (2025)csKT: Addressing cold-start problem in knowledge tracing via kernel bias and cone attentionExpert Systems with Applications10.1016/j.eswa.2024.125988266(125988)Online publication date: Mar-2025
  • (2024)RIGL: A Unified Reciprocal Approach for Tracing the Independent and Group Learning ProcessesProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671711(4047-4058)Online publication date: 25-Aug-2024
  • Show More Cited By

Index Terms

  1. HD-KT: Advancing Robust Knowledge Tracing via Anomalous Learning Interaction Detection

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    WWW '24: Proceedings of the ACM Web Conference 2024
    May 2024
    4826 pages
    ISBN:9798400701719
    DOI:10.1145/3589334
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 13 May 2024

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. anomaly detection
    2. intelligent education
    3. knowledge tracing

    Qualifiers

    • Research-article

    Conference

    WWW '24
    Sponsor:
    WWW '24: The ACM Web Conference 2024
    May 13 - 17, 2024
    Singapore, Singapore

    Acceptance Rates

    Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)413
    • Downloads (Last 6 weeks)45
    Reflects downloads up to 14 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)Examination Process Modeling for Intelligent Patent Management: A Multi-aspect Neural Sequential ApproachACM Transactions on Management Information Systems10.1145/3712309Online publication date: 14-Jan-2025
    • (2025)csKT: Addressing cold-start problem in knowledge tracing via kernel bias and cone attentionExpert Systems with Applications10.1016/j.eswa.2024.125988266(125988)Online publication date: Mar-2025
    • (2024)RIGL: A Unified Reciprocal Approach for Tracing the Independent and Group Learning ProcessesProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671711(4047-4058)Online publication date: 25-Aug-2024
    • (2024)Knowledge tracing via multiple-state diffusion representationExpert Systems with Applications10.1016/j.eswa.2024.124797255(124797)Online publication date: Dec-2024
    • (2024)Enhanced Knowledge Tracing via Frequency Integration and Order SensitivityPRICAI 2024: Trends in Artificial Intelligence10.1007/978-981-96-0116-5_34(403-415)Online publication date: 12-Nov-2024

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media