Abstract
Performance prediction is an important research facet of educational data mining. Most models extract student behavior features from campus card data for prediction. However, most of these methods have coarse time granularity, difficulty in extracting useful high-order behavior combination features, dependence on 6 historical achievements, etc. To solve these problems, this paper utilizes prediction of grade point average (GPA prediction) and whether a specific student has failing subjects (failing prediction) in a term as the goal of performance prediction and proposes a comprehensive performance prediction model of college students based on behavior features. First, a method for representing campus card data based on behavior flow is introduced to retain higher time accuracy. Second, a method for extracting student behavior features based on multi-head self-attention mechanism is proposed to automatically select more important high-order behavior combination features. Finally, a performance prediction model based on student behavior feature mode difference is proposed to improve the model’s prediction accuracy and increases the model’s robustness for students with significant changes in performance. The performance of the model is verified on actual data collected by the teaching monitoring big data platform of Xi’an Jiaotong University. The results show that the model’s prediction performance is better than the comparison algorithms on both the failing prediction and GPA prediction.














Similar content being viewed by others
References
Butcher B, Smith BJ (2020) Feature engineering and selection: a practical approach for predictive models. CRC Press
Ca OY, Gao J, Lian D, Rong Z, Shi J, Wang Q et al (2017) Orderness predicts academic performance: behavioral analysis on campus lifestyle. J R Soc Interface 15(146):20180210
Chaturvedi R, and Ezeife CI (2017) Predicting student performance in an ITS using task-driven features. In: 2017 IEEE international conference on computer and information technology (CIT). IEEE, pp 168–175
Chen Y, Zheng Q, Ji S, Tian F, and Liu M (2020) Identifying at-risk students based on the phased prediction model. Knowl Inf Syst. https://doi.org/10.1007/s10115-019-01374-x
Conijn R, Van den Beemt A, Cuijpers P (2017) Predicting student performance in a blended MOOC. J Comput Ass Learn
Donahue J, Hendricks LA, Rohrbach M, Venugopalan S, Guadarrama S, and Saenko K, et al. (2017) Long-term recurrent convolutional networks for visual recognition and description. In: 2015 IEEE conference on computer vision and mode recognition (CVPR), vol 39, pp 677–691. IEEE, pp 2625–2634
Fu X, Ch’Ng E, Aickelin U, and See S (2017) CRNN: a joint neural network for redundancy detection. SSRN Electron J 1–8
Hu YH, Lo CL, Shih SP (2014) Developing early warning systems to predict students’ online learning performance. Comput Hum Behav 36:469–478
Huang S, Fang N (2013) Predicting student academic performance in an engineering dynamics course: a comparison of four types of predictive mathematical models. Comput Educ 61:133–145
Ji SG (2019) Research on the predictive model of undergraduates. Dissertation, Xi’an Jiaotong University
Kim BH, Vizitei E, Ganapathi V (2018) Gritnet: student performance prediction with deep learning. arXiv preprint arXiv:1804.07405
Kim J, El-Khamy M, Lee J (2020) T-GSA: transformer with gaussian-weighted self-attention for speech enhancement. In: ICASSP 2020–2020 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, pp 6649–6653
Liao J, Tang J, Zhao X (2019) Course drop-out prediction on MOOC platform via clustering and tensor completion. Tsinghua Sci Technol 24(4):412–422
Liu M(2016) Research on characteristic analysis and identification of learners with poor achievements in online education. Dissertation, Xi’an Jiaotong University
Li S, Zhou Y, Wu X, Li A, and Zhou B (2017) A method of emotional analysis of movie based on convolution neural network and bi-directional LSTM RNN. In: 2017 IEEE 2nd international conference on data science in cyberspace (DSC). IEEE, pp 156–161
Lu OHT, Huang AYQ, Huang JCH, Lin AJQ, Ogata H, Yang SJH (2018) Applying learning analytics for the early prediction of students’ academic performance in blended learning. Educ Technol Soc 21(2):220–232
Ma, Y, Zong J, Cui C, Zhang C, and Yin Y (2019) Dual path convolutional neural network for student performance prediction. In: International conference on web information systems engineering. Springer, Cham, pp 133–146
Mayilvaganan M, Kalpanadevi D (2014) Comparison of classification techniques for predicting the performance of students academic environment. In: International conference on communication & network technologies. IEEE, pp 113–118
Mccredie MN, Kurtz JE (2019) Prospective prediction of academic performance in college using self- and informant-rated personality traits. J Res Pers 85:103911
Mitchell TM (1999) Machine learning and data mining. Commun ACM 42(11):30–36
Moreno-Marcos PM, Pong TC, Munoz-Merino PJ, and Kloos CD (2020) Analysis of the factors influencing learners’ performance prediction with learning analytics. IEEE Access PP(99):1
Osmanbegović E, Suljić M, Agić H (2014) Determining dominant factor for students performance prediction by using data mining classification algorithms. Trans: J Econ Polit Trans/Tranz: Cas Econ Pol Tranz 16(34):147–158
Romero Cristóbal, López Manuel-Ignacio, Luna Jose-María, Ventura Sebastián (2013) Predicting students’ final performance from participation in on-line discussion forums. Comput Edu 68:458–472
Song J (2020) Analysis of learning behavior and prediction of learning achievement based on campus big data. Dissertation, Central China Normal University. https://doi.org/10.27159/d.cnki.ghzsu.2020.001990
Satyanarayana A, Ravichandran G (2016) Mining student data by ensemble classification and clustering for profiling and prediction of student academic performance. American Society for Engineering Education
Shah S (2009) Impact of teacher’s behaviour on the academic achievement of university students. J Coll Teach Learn 6(1):69–74
Song X, Li J (2021) Sequential engagement-based online learning analytics and prediction. IEEE Intell Syst 36(1):46–53
Song-Jiang LI, Yu SU, Huang C Y, Wang P, Ren T (2019) A combination of performance change trends and student behavior prediction models. Journal of Changchun University of Science and Technology (Natural Science Edition)
Song W, Shi C, Xiao Z, Duan Z, and Tang J (2019) AutoInt: automatic feature interaction learning via self-attentive neural networks. In: The 28th ACM international conference. ACM, pp 1161–1170
Su Y, Liu Q, Liu Q, Huang Z, Yin Y, Chen E, Ding C, Wei S and Hu G (2018) Exercise-enhanced sequential modeling for student performance prediction. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 2435–2443
Tsironi E, Barros P, Weber C, Wermter S (2017) An analysis of convolutional long short-term memory recurrent neural networks for gesture recognition. Neurocomputing 268:76–86
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, and Gomez AN, et al. (2017) Attention is all you need. arXiv preprint arXiv:1706.03762
Veeramanickam M, Mohanapriya M, Pandey BK, Akhade S, Kale SA, Patil R et al (2018) Map-reduce framework based cluster architecture for academic student’s performance prediction using cumulative dragonfly based neural network. Clust Comput 22(1):1259–1275
Wakelam E, Jefferies A, Davey N, Sun Y (2020) The potential for student performance prediction in small cohorts with minimal available attributes. Br J Educ Technol 51:347–370. https://doi.org/10.1111/bjet.12836
Wang X, Yu X, Liu F, Xu L, and Guo L (2020) Student performance prediction with short-term sequential campus behaviors. Information (Switzerland). https://doi.org/10.3390/INFO11040201
Wei W, Han Y, Miao C (2017) Deep model for dropout prediction in MOOCs. In: The 2nd international conference, pp 26–32
Zhao H, Jia J, Koltun V (2020) Exploring self-attention for image recognition. IEEE/CVF Confer Comput Vis Mode Recogn (CVPR) 2020:10076–10085. https://doi.org/10.1109/CVPR42600.2020.01009
Zhou M, Ma M, Zhang Y, Suia K, Pei D, Moscibroda T (2016) EDUM: classroom education measurements via large-scale WiFi networks. In: The 2016 ACM international joint conference on pervasive and ubiquitous computing. ACM, pp 316–327
Zhou Q, Quan W, Zhong Y, Xiao W, Mou C, Wang Y (2018) Predicting high-risk students using internet access logs. Knowl Inf Syst 55(2):393–413. https://doi.org/10.1007/s10115-017-1086-5
Zong J, Cui C, Ma Y, Yao L, and Yin Y (2020) Behavior-driven student performance prediction with tri-branch convolutional neural network. In: CIKM ’20: the 29th ACM international conference on information and knowledge management. ACM, pp 2353–2356
Acknowledgements
This work was supported by the National Key Research and Development Program of China (2020AAA0108800), National Natural Science Foundation of China (62137002, 61721002, 61937001, 61877048, 62177038), Innovation Research Team of Ministry of Education (IRT_17R86), Project of China Knowledge Centre for Engineering Science and Technology, The Natural Science Basic Research Plan in Shaanxi Province of China (2020JM-070), MoE-CMCC “Artificial Intelligence” Project (MCM20190701), Project of Chinese Academy of Engineering “The Online and Offline Mixed Educational Service System for ‘The Belt and Road’ Training in MOOC China,” “LENOVO-XJTU” Intelligent Industry Joint Laboratory Project.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Chen, Y., Wei, G., Liu, J. et al. A prediction model of student performance based on self-attention mechanism. Knowl Inf Syst 65, 733–758 (2023). https://doi.org/10.1007/s10115-022-01774-6
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10115-022-01774-6