Abstract
Knowledge tracing is the task of dynamically tracking a student’s mastery of skills based on their assessment performances and learning-related information (e.g., time spent answering a particular question). Traditional approaches (e.g., Bayesian knowledge tracing [BKT] and performance factors analysis [PFA]), are easy to interpret. Modern approaches (e.g., deep knowledge tracing [DKT] and dynamic key-value memory networks [DKVMN]) usually produce superior performance on certain datasets, but their model complexity causes difficulty in scaling and linking them to existing educational measurement studies. In this paper, we present a simple but effective model, deep performance factors analysis (DPFA) (Source code is available at https://github.com/scott-pu-pennstate/dpfa), to resolve this problem. DPFA consistently outperforms PFA and DKT and has results comparable to those of DKVMN when tested on widely used public datasets. In addition, DPFA’s light weight in parameters makes it easy to scale. Finally, we demonstrate a straightforward approach to enhance the base DPFA by incorporating features from the educational measurement literature. The enhanced DPFA showed superior performance than DKVMN.
S. Pu—The work is done before this author joined ETS Canada, Inc.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Abadi, M., et al.: TensorFlow: large-scale machine learning on heterogeneous systems (2015). https://www.tensorflow.org/, software available from tensorflow.org
Choi, Y., et al.: Towards an appropriate query, key, and value computation for knowledge tracing. arXiv preprint arXiv:2002.07033 (2020)
Corbett, A.T., Anderson, J.R.: Knowledge tracing: modeling the acquisition of procedural knowledge. In: User Model User-Adapted Interaction, vol. 4 (1995)
Ghosh, A., Heffernan, N., Lan, A.S.: Context-aware attentive knowledge tracing. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2330–2339 (2020)
Graves, A., et al.: Hybrid computing using a neural network with dynamic external memory. Nature 538(7626), 471–476 (2016)
Huang, Z., Yin, Y., Chen, E., Xiong, H., Su, Y., Hu, G., et al.: Ekt: exercise-aware knowledge tracing for student performance prediction. IEEE Trans. Knowl. Data Eng. 33(1), 100–115 (2019)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Liu, Y., Yang, Y., Chen, X., Shen, J., Zhang, H., Yu, Y.: Improving knowledge tracing via pre-training question embeddings. arXiv preprint arXiv:2012.05031 (2020)
Maaten, L.V.d., Hinton, G.J.: Visualizing data using t-SNE. Mach. Learn. Res. 9(Nov), 2579–2605 (2008)
Pandey, S., Karypis, G.: A self-attentive model for knowledge tracing. arXiv preprint arXiv:1907.06837 (2019)
Pavlik Jr., P.I., Cen, H., Koedinger, K.R.: Performance factors analysis-a new alternative to knowledge tracing. Online Submission (2009)
Piech, C., et al.: Deep knowledge tracing. In: Advances in Neural Information Processing Systems, pp. 505–513 (2015)
Pu, S., Yudelson, M., Ou, L., Huang, Y.: Deep knowledge tracing with transformers. In: Bittencourt, I.I., Cukurova, M., Muldner, K., Luckin, R., Millán, E. (eds.) AIED 2020. LNCS (LNAI), vol. 12164, pp. 252–256. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-52240-7_46
Su, Y., et al.: Exercise-enhanced sequential modeling for student performance prediction. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)
Tong, H., Zhou, Y., Wang, Z.: HGKT: introducing problem schema with hierarchical exercise graph for knowledge tracing. arXiv preprint arXiv:2006.16915 (2020)
Wang, T., Ma, F., Gao, J.: Deep hierarchical knowledge tracing. In: Proceedings of the 12th International Conference on Educational Data Mining (2019)
Wang, T., Hanson, B.A.: Development and calibration of an item response model that incorporates response time. Appl. Psychol. Meas. 29(5), 323–339 (2005)
Wise, S.L., Kong, X.: Response time effort: a new measure of examinee motivation in computer-based tests. Appl. Meas. Educ. 18(2), 163–183 (2005)
Xiong, X., Zhao, S., Van Inwegen, E.G., Beck, J.E.: Going deeper with deep knowledge tracing. International Educational Data Mining Society (2016)
Zhang, J., Shi, X., King, I., Yeung, D.Y.: Dynamic key-value memory networks for knowledge tracing. In: Proceedings of the 26th International Conference on World Wide Web, pp. 765–774 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Pu, S., Converse, G., Huang, Y. (2021). Deep Performance Factors Analysis for Knowledge Tracing. In: Roll, I., McNamara, D., Sosnovsky, S., Luckin, R., Dimitrova, V. (eds) Artificial Intelligence in Education. AIED 2021. Lecture Notes in Computer Science(), vol 12748. Springer, Cham. https://doi.org/10.1007/978-3-030-78292-4_27
Download citation
DOI: https://doi.org/10.1007/978-3-030-78292-4_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-78291-7
Online ISBN: 978-3-030-78292-4
eBook Packages: Computer ScienceComputer Science (R0)