skip to main content
10.1145/3529466.3529477acmotherconferencesArticle/Chapter ViewAbstractPublication PagesiciaiConference Proceedingsconference-collections
research-article

A Novel Knowledge Tracing Model Based on Collaborative Multi-Head Attention

Published: 04 June 2022 Publication History

Abstract

Online education is playing a more and more important role in today's education. The key link of online education is to model students' knowledge mastery according to their historical behaviors, so as to obtain the knowledge tracing represented by students' current knowledge state. Previous Transformer-based knowledge tracing models have disadvantages such as inefficient model computation and redundant information on the one hand. On the other hand, the traditional knowledge tracing model cannot solve the problem of imbalanced positive and negative samples in the data well. In order to better model the current knowledge state of students, this paper proposes a knowledge tracing model based on the collaborative multi-head attention mechanism. The model uses a collaborative multi-head attention mechanism to solve the information redundancy problem in the previous Transformer-based knowledge tracing model, and improves the computational efficiency and performance of the model. The model also introduces a focal loss function, which not only solves the problem of imbalanced question labeling divisions in knowledge tracing but also improves the differentiation of difficulty level among the questions and enhances the accuracy of model prediction. The experimental results on three public experimental datasets show that the knowledge tracing model based on the collaborative multi-head attention mechanism proposed in this paper outperforms other recent knowledge tracing models in terms of evaluation metric AUC and also has better performance in predicting students' responses.

References

[1]
Vaswani A, Shazeer N, Parmar N, Attention Is All You Need[J]. arXiv, 2017.
[2]
Pandey S, Karypis G . A Self-Attentive model for Knowledge Tracing[J]. 2019.
[3]
Towards an Appropriate Query, Key, and Value Computation for Knowledge Tracing[J]. 2020.
[4]
D Shin, Shim Y, Yu H, SAINT+: Integrating Temporal Features for EdNet Correctness Prediction[J]. 2020.
[5]
Voita E, Talbot D, Moiseev F, Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned[J]. 2019.
[6]
Michel P, Levy O, Neubig G. Are sixteen heads really better than one?[J]. arXiv preprint arXiv:1905.10650, 2019.
[7]
Cordonnier J B, Loukas A, Jaggi M . Multi-Head Attention: Collaborate Instead of Concatenate[J]. 2020.
[8]
Lin T Y, Goyal P, Girshick R, Focal Loss for Dense Object Detection[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, PP(99):2999-3007.
[9]
Pardos Z A, Heffernan N T. T.: Modeling Individualization in a Bayesian Networks Implementation of Knowledge Tracing[C]// User Modeling, Adaptation, & Personalization, International Conference, Umap, Big Island, Hi, Usa, June. Springer Berlin Heidelberg, 2010.
[10]
C. Piech, J. Spencer, J. Huang, S. Ganguli,M. Sahami, L. Guibas, and J. Sohl-Dickstein. Deep knowledge tracing. In Advances in Neural Information Processing Systems, 2015.
[11]
hang J, Shi X, King I, Dynamic Key-Value Memory Networks for Knowledge Tracing[J]. 2016.
[12]
Girshick R. Fast R-CNN[J]. arXiv e-prints, 2015.
[13]
He K, Zhang X, Ren S, Deep Residual Learning for Image Recognition[J]. IEEE, 2016.
[14]
Ba J L, Kiros J R, Hinton G E. Layer Normalization[J]. 2016.
[15]
Kingma D, Ba J. Adam: A Method for Stochastic Optimization[J]. Computer Science, 2014.
[16]
Ji S, Pan S, Cambria E, A Survey on Knowledge Graphs: Representation, Acquisition and Applications[J]. 2020.

Cited By

View all
  • (2024)Hybrid Models for Knowledge Tracing: A Systematic Literature ReviewIEEE Transactions on Learning Technologies10.1109/TLT.2023.334869017(1021-1036)Online publication date: 1-Jan-2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
ICIAI '22: Proceedings of the 2022 6th International Conference on Innovation in Artificial Intelligence
March 2022
240 pages
ISBN:9781450395502
DOI:10.1145/3529466
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 June 2022

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Collaborative multi-head attention
  2. Deep learning
  3. Key words: Knowledge Tracing
  4. Loss function

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

  • Bing-tuan Science and Technology Public Relations Project

Conference

ICIAI 2022

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)30
  • Downloads (Last 6 weeks)1
Reflects downloads up to 30 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Hybrid Models for Knowledge Tracing: A Systematic Literature ReviewIEEE Transactions on Learning Technologies10.1109/TLT.2023.334869017(1021-1036)Online publication date: 1-Jan-2024

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media