Wei Zhang
ABSTRACT
Online education is playing a more and more important role in today's education. The key link of online education is to model students' knowledge mastery according to their historical behaviors, so as to obtain the knowledge tracing represented by students' current knowledge state. Previous Transformer-based knowledge tracing models have disadvantages such as inefficient model computation and redundant information on the one hand. On the other hand, the traditional knowledge tracing model cannot solve the problem of imbalanced positive and negative samples in the data well. In order to better model the current knowledge state of students, this paper proposes a knowledge tracing model based on the collaborative multi-head attention mechanism. The model uses a collaborative multi-head attention mechanism to solve the information redundancy problem in the previous Transformer-based knowledge tracing model, and improves the computational efficiency and performance of the model. The model also introduces a focal loss function, which not only solves the problem of imbalanced question labeling divisions in knowledge tracing but also improves the differentiation of difficulty level among the questions and enhances the accuracy of model prediction. The experimental results on three public experimental datasets show that the knowledge tracing model based on the collaborative multi-head attention mechanism proposed in this paper outperforms other recent knowledge tracing models in terms of evaluation metric AUC and also has better performance in predicting students' responses.
- Vaswani A, Shazeer N, Parmar N, Attention Is All You Need[J]. arXiv, 2017.Google Scholar
- Pandey S, Karypis G . A Self-Attentive model for Knowledge Tracing[J]. 2019.Google Scholar
- Towards an Appropriate Query, Key, and Value Computation for Knowledge Tracing[J]. 2020.Google Scholar
- D Shin, Shim Y, Yu H, SAINT+: Integrating Temporal Features for EdNet Correctness Prediction[J]. 2020.Google Scholar
- Voita E, Talbot D, Moiseev F, Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned[J]. 2019.Google Scholar
- Michel P, Levy O, Neubig G. Are sixteen heads really better than one?[J]. arXiv preprint arXiv:1905.10650, 2019.Google Scholar
- Cordonnier J B, Loukas A, Jaggi M . Multi-Head Attention: Collaborate Instead of Concatenate[J]. 2020.Google Scholar
- Lin T Y, Goyal P, Girshick R, Focal Loss for Dense Object Detection[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, PP(99):2999-3007.Google ScholarCross Ref
- Pardos Z A, Heffernan N T. T.: Modeling Individualization in a Bayesian Networks Implementation of Knowledge Tracing[C]// User Modeling, Adaptation, & Personalization, International Conference, Umap, Big Island, Hi, Usa, June. Springer Berlin Heidelberg, 2010.Google Scholar
- C. Piech, J. Spencer, J. Huang, S. Ganguli,M. Sahami, L. Guibas, and J. Sohl-Dickstein. Deep knowledge tracing. In Advances in Neural Information Processing Systems, 2015.Google Scholar
- hang J, Shi X, King I, Dynamic Key-Value Memory Networks for Knowledge Tracing[J]. 2016.Google Scholar
- Girshick R. Fast R-CNN[J]. arXiv e-prints, 2015.Google Scholar
- He K, Zhang X, Ren S, Deep Residual Learning for Image Recognition[J]. IEEE, 2016.Google ScholarCross Ref
- Ba J L, Kiros J R, Hinton G E. Layer Normalization[J]. 2016.Google Scholar
- Kingma D, Ba J. Adam: A Method for Stochastic Optimization[J]. Computer Science, 2014.Google Scholar
- Ji S, Pan S, Cambria E, A Survey on Knowledge Graphs: Representation, Acquisition and Applications[J]. 2020.Google Scholar
Recommendations
Knowledge tracing based on multi-feature fusion
AbstractKnowledge tracing involves modeling student knowledge states over time so that we can accurately predict student performance in future interactions and recommend personalized student learning paths. However, existing methods, such as deep ...
SQKT: A Student Attention-Based and Question-Aware Model for Knowledge Tracing
Web and Big DataAbstractThe goal of Knowledge Tracing (KT) is to trace student’s knowledge states in relation to different knowledge concepts and make prediction of student’s performance on new exercises. With the growing number of online learning platforms, personalized ...
MPSKT: Multi-head ProbSparse Self-Attention for Knowledge Tracing
CSAE '22: Proceedings of the 6th International Conference on Computer Science and Application EngineeringOver the past two years, COVID-19 has led to a widespread rise in online education, and knowledge tracing has been used on various educational platforms. However, most existing knowledge tracing models still suffer from long-term dependence. To address ...
Comments