Skip to main content

Advertisement

SpringerLink
Log in

A prediction model of student performance based on self-attention mechanism

  • Regular Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

Performance prediction is an important research facet of educational data mining. Most models extract student behavior features from campus card data for prediction. However, most of these methods have coarse time granularity, difficulty in extracting useful high-order behavior combination features, dependence on 6 historical achievements, etc. To solve these problems, this paper utilizes prediction of grade point average (GPA prediction) and whether a specific student has failing subjects (failing prediction) in a term as the goal of performance prediction and proposes a comprehensive performance prediction model of college students based on behavior features. First, a method for representing campus card data based on behavior flow is introduced to retain higher time accuracy. Second, a method for extracting student behavior features based on multi-head self-attention mechanism is proposed to automatically select more important high-order behavior combination features. Finally, a performance prediction model based on student behavior feature mode difference is proposed to improve the model’s prediction accuracy and increases the model’s robustness for students with significant changes in performance. The performance of the model is verified on actual data collected by the teaching monitoring big data platform of Xi’an Jiaotong University. The results show that the model’s prediction performance is better than the comparison algorithms on both the failing prediction and GPA prediction.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Butcher B, Smith BJ (2020) Feature engineering and selection: a practical approach for predictive models. CRC Press

  2. Ca OY, Gao J, Lian D, Rong Z, Shi J, Wang Q et al (2017) Orderness predicts academic performance: behavioral analysis on campus lifestyle. J R Soc Interface 15(146):20180210

    Article  Google Scholar 

  3. Chaturvedi R, and Ezeife CI (2017) Predicting student performance in an ITS using task-driven features. In: 2017 IEEE international conference on computer and information technology (CIT). IEEE, pp 168–175

  4. Chen Y, Zheng Q, Ji S, Tian F, and Liu M (2020) Identifying at-risk students based on the phased prediction model. Knowl Inf Syst. https://doi.org/10.1007/s10115-019-01374-x

  5. Conijn R, Van den Beemt A, Cuijpers P (2017) Predicting student performance in a blended MOOC. J Comput Ass Learn

  6. Donahue J, Hendricks LA, Rohrbach M, Venugopalan S, Guadarrama S, and Saenko K, et al. (2017) Long-term recurrent convolutional networks for visual recognition and description. In: 2015 IEEE conference on computer vision and mode recognition (CVPR), vol 39, pp 677–691. IEEE, pp 2625–2634

  7. Fu X, Ch’Ng E, Aickelin U, and See S (2017) CRNN: a joint neural network for redundancy detection. SSRN Electron J 1–8

  8. Hu YH, Lo CL, Shih SP (2014) Developing early warning systems to predict students’ online learning performance. Comput Hum Behav 36:469–478

  9. Huang S, Fang N (2013) Predicting student academic performance in an engineering dynamics course: a comparison of four types of predictive mathematical models. Comput Educ 61:133–145

    Article  Google Scholar 

  10. Ji SG (2019) Research on the predictive model of undergraduates. Dissertation, Xi’an Jiaotong University

  11. Kim BH, Vizitei E, Ganapathi V (2018) Gritnet: student performance prediction with deep learning. arXiv preprint arXiv:1804.07405

  12. Kim J, El-Khamy M, Lee J (2020) T-GSA: transformer with gaussian-weighted self-attention for speech enhancement. In: ICASSP 2020–2020 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, pp 6649–6653

  13. Liao J, Tang J, Zhao X (2019) Course drop-out prediction on MOOC platform via clustering and tensor completion. Tsinghua Sci Technol 24(4):412–422

    Article  Google Scholar 

  14. Liu M(2016) Research on characteristic analysis and identification of learners with poor achievements in online education. Dissertation, Xi’an Jiaotong University

  15. Li S, Zhou Y, Wu X, Li A, and Zhou B (2017) A method of emotional analysis of movie based on convolution neural network and bi-directional LSTM RNN. In: 2017 IEEE 2nd international conference on data science in cyberspace (DSC). IEEE, pp 156–161

  16. Lu OHT, Huang AYQ, Huang JCH, Lin AJQ, Ogata H, Yang SJH (2018) Applying learning analytics for the early prediction of students’ academic performance in blended learning. Educ Technol Soc 21(2):220–232

  17. Ma, Y, Zong J, Cui C, Zhang C, and Yin Y (2019) Dual path convolutional neural network for student performance prediction. In: International conference on web information systems engineering. Springer, Cham, pp 133–146

  18. Mayilvaganan M, Kalpanadevi D (2014) Comparison of classification techniques for predicting the performance of students academic environment. In: International conference on communication & network technologies. IEEE, pp 113–118

  19. Mccredie MN, Kurtz JE (2019) Prospective prediction of academic performance in college using self- and informant-rated personality traits. J Res Pers 85:103911

    Article  Google Scholar 

  20. Mitchell TM (1999) Machine learning and data mining. Commun ACM 42(11):30–36

    Article  Google Scholar 

  21. Moreno-Marcos PM, Pong TC, Munoz-Merino PJ, and Kloos CD (2020) Analysis of the factors influencing learners’ performance prediction with learning analytics. IEEE Access PP(99):1

  22. Osmanbegović E, Suljić M, Agić H (2014) Determining dominant factor for students performance prediction by using data mining classification algorithms. Trans: J Econ Polit Trans/Tranz: Cas Econ Pol Tranz 16(34):147–158

    Google Scholar 

  23. Romero Cristóbal, López Manuel-Ignacio, Luna Jose-María, Ventura Sebastián (2013) Predicting students’ final performance from participation in on-line discussion forums. Comput Edu 68:458–472

  24. Song J (2020) Analysis of learning behavior and prediction of learning achievement based on campus big data. Dissertation, Central China Normal University. https://doi.org/10.27159/d.cnki.ghzsu.2020.001990

  25. Satyanarayana A, Ravichandran G (2016) Mining student data by ensemble classification and clustering for profiling and prediction of student academic performance. American Society for Engineering Education

  26. Shah S (2009) Impact of teacher’s behaviour on the academic achievement of university students. J Coll Teach Learn 6(1):69–74

  27. Song X, Li J (2021) Sequential engagement-based online learning analytics and prediction. IEEE Intell Syst 36(1):46–53

    Article  Google Scholar 

  28. Song-Jiang LI, Yu SU, Huang C Y, Wang P, Ren T (2019) A combination of performance change trends and student behavior prediction models. Journal of Changchun University of Science and Technology (Natural Science Edition)

  29. Song W, Shi C, Xiao Z, Duan Z, and Tang J (2019) AutoInt: automatic feature interaction learning via self-attentive neural networks. In: The 28th ACM international conference. ACM, pp 1161–1170

  30. Su Y, Liu Q, Liu Q, Huang Z, Yin Y, Chen E, Ding C, Wei S and Hu G (2018) Exercise-enhanced sequential modeling for student performance prediction. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 2435–2443

  31. Tsironi E, Barros P, Weber C, Wermter S (2017) An analysis of convolutional long short-term memory recurrent neural networks for gesture recognition. Neurocomputing 268:76–86

    Article  Google Scholar 

  32. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, and Gomez AN, et al. (2017) Attention is all you need. arXiv preprint arXiv:1706.03762

  33. Veeramanickam M, Mohanapriya M, Pandey BK, Akhade S, Kale SA, Patil R et al (2018) Map-reduce framework based cluster architecture for academic student’s performance prediction using cumulative dragonfly based neural network. Clust Comput 22(1):1259–1275

  34. Wakelam E, Jefferies A, Davey N, Sun Y (2020) The potential for student performance prediction in small cohorts with minimal available attributes. Br J Educ Technol 51:347–370. https://doi.org/10.1111/bjet.12836

    Article  Google Scholar 

  35. Wang X, Yu X, Liu F, Xu L, and Guo L (2020) Student performance prediction with short-term sequential campus behaviors. Information (Switzerland). https://doi.org/10.3390/INFO11040201

  36. Wei W, Han Y, Miao C (2017) Deep model for dropout prediction in MOOCs. In: The 2nd international conference, pp 26–32

  37. Zhao H, Jia J, Koltun V (2020) Exploring self-attention for image recognition. IEEE/CVF Confer Comput Vis Mode Recogn (CVPR) 2020:10076–10085. https://doi.org/10.1109/CVPR42600.2020.01009

    Article  Google Scholar 

  38. Zhou M, Ma M, Zhang Y, Suia K, Pei D, Moscibroda T (2016) EDUM: classroom education measurements via large-scale WiFi networks. In: The 2016 ACM international joint conference on pervasive and ubiquitous computing. ACM, pp 316–327

  39. Zhou Q, Quan W, Zhong Y, Xiao W, Mou C, Wang Y (2018) Predicting high-risk students using internet access logs. Knowl Inf Syst 55(2):393–413. https://doi.org/10.1007/s10115-017-1086-5

  40. Zong J, Cui C, Ma Y, Yao L, and Yin Y (2020) Behavior-driven student performance prediction with tri-branch convolutional neural network. In: CIKM ’20: the 29th ACM international conference on information and knowledge management. ACM, pp 2353–2356

Download references

Acknowledgements

This work was supported by the National Key Research and Development Program of China (2020AAA0108800), National Natural Science Foundation of China (62137002, 61721002, 61937001, 61877048, 62177038), Innovation Research Team of Ministry of Education (IRT_17R86), Project of China Knowledge Centre for Engineering Science and Technology, The Natural Science Basic Research Plan in Shaanxi Province of China (2020JM-070), MoE-CMCC “Artificial Intelligence” Project (MCM20190701), Project of Chinese Academy of Engineering “The Online and Offline Mixed Educational Service System for ‘The Belt and Road’ Training in MOOC China,” “LENOVO-XJTU” Intelligent Industry Joint Laboratory Project.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Xi’an Jiaotong University, Xi’an, 710049, Shaanxi, China

    Yan Chen, Ganglin Wei, Jiaxin Liu, Yunwei Chen, Qinghua Zheng, Feng Tian & Haiping Zhu

  2. Lenovo (Beijing) Co., Ltd., Beijing, 100089, Beijing, China

    Qianying Wang & Yaqiang Wu

Authors
  1. Yan Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ganglin Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiaxin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yunwei Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Qinghua Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Feng Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Haiping Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Qianying Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Yaqiang Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ganglin Wei.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, Y., Wei, G., Liu, J. et al. A prediction model of student performance based on self-attention mechanism. Knowl Inf Syst 65, 733–758 (2023). https://doi.org/10.1007/s10115-022-01774-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-022-01774-6

Keywords

Search

Navigation