Skip to main content

Advertisement

Predictive analytics in education: a comparison of deep learning frameworks

  • Published:
Education and Information Technologies Aims and scope Submit manuscript

Abstract

Large swaths of data are readily available in various fields, and education is no exception. In tandem, the impetus to derive meaningful insights from data gains urgency. Recent advances in deep learning, particularly in the area of voice and image recognition and so-called complete knowledge games like chess, go, and StarCraft, have resulted in a flurry of research. Using two educational datasets, we explore the utility and applicability of deep learning for educational data mining and learning analytics. We compare the predictive accuracy of popular deep learning frameworks/libraries, including, Keras, Theano, Tensorflow, fast.ai, and Pytorch. Experimental results reveal that performance, as assessed by predictive accuracy, varies depending on the optimizer used. Further, findings from additional experiments by tuning network parameters yield similar results. Moreover, we find that deep learning displays comparable performance to other machine learning algorithms such as support vector machines, k-nearest neighbors, naive Bayes classifier, and logistic regression. We argue that statistical learning techniques should be selected to maximize interpretability and should contribute to our understanding of educational and learning phenomena; hence, in most cases, educational data mining and learning analytics researchers should aim for explanation over prediction.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Aldowah, H., Al-Samarraie, H., & Fauzy, W. (2019). Educational data mining and learning analytics for 21st century higher education: A review and synthesis. Telematics and Informatics, 37, 13–49. https://doi.org/10.1016/j.tele.2019.01.007.

    Article  Google Scholar 

  • Avella, J., Kebritchi, M., Nunn, S., & Kanai, T. (2016). Learning analytics methods, benefits, and challenges in higher education: A systematic literature review. Online Learning, 20(2), 13–29.

    Google Scholar 

  • Baker, R. S., & Inventado, P. S. (2014). Educational data mining and learning analytics. In Learning analytics (pp. 61–75). New York, NY: Springer.

    Chapter  Google Scholar 

  • Batmaz, Z., Yurekli, A., Bilge, A., & Kaleli, C. (2018). A review on deep learning for recommender systems: Challenges and remedies. Artificial Intelligence Review, 52(1), 1–37. https://doi.org/10.1007/s10462-018-9654-y.

    Article  Google Scholar 

  • Bazelais, P., Lemay, D. J., Doleck, T., Hu, X. S., Vu, A., & Yao, J. (2018). Grit, mindset, and academic performance: A study of pre-University science students. Eurasia Journal of Mathematics, Science and Technology Education, 14(12), 1–10. https://doi.org/10.29333/ejmste/94570.

  • Botelho, A. F., Baker, R. S., & Heffernan, N. T. (2017). Improving sensor-free affect detection using deep learning. In E. André, R. Baker, X. Hu, M. M. T. Rodrigo, & B. Boulay (Eds.), Proceedings of the 18th international conference on artificial intelligence in education (pp. 40–51). London, UK: Springer International Publishing.

    Chapter  Google Scholar 

  • Brinton, C. G., & Chiang, M. (2015). MOOC performance prediction via clickstream data and social learning networks. IEEE Conference on Computer Communications (INFOCOM), 2299–2307.

  • Costa, E., Fonseca, B., Santana, M., de Araújo, F., & Rego, J. (2017). Evaluating the effectiveness of educational data mining techniques for early prediction of students’ academic failure in introductory programming courses. Computers in Human Behavior, 73, 247–256. https://doi.org/10.1016/j.chb.2017.01.047.

    Article  Google Scholar 

  • Deng, L., & Yu, D. (2014). Deep learning: Methods and applications. Foundations And Trends In Signal Processing, 7(3–4), 197–387. https://doi.org/10.1561/2000000039.

    Article  MathSciNet  MATH  Google Scholar 

  • Doleck, T., Basnet, R. B., Poitras, E. G., & Lajoie, S. P. (2015). Mining learner-system interaction data: Implications for modeling learner behaviors and improving overlay models. Journal of Computers in Education, 2(4), 421–447. https://doi.org/10.1007/s40692-015-0040-3.

    Article  Google Scholar 

  • Doleck, T., Jarrell, A., Poitras, E. G., Chaouachi, M., & Lajoie, S. P. (2016). A tale of three cases: Examining accuracy, efficiency, and process differences in diagnosing virtual patient cases. Australasian Journal of Educational Technology, 36(5), 61–76. https://doi.org/10.14742/ajet.2759.

  • Doleck, T., Poitras, E., & Lajoie, S. (2019). Assessing the utility of deep learning: Using learner-system interaction data from BioWorld. In J. Theo Bastiaens (Ed.), Proceedings of EdMedia + innovate learning (pp. 734–738). Amsterdam, Netherlands: AACE.

    Google Scholar 

  • Fast.ai. (2019). Retrieved from https://www.fast.ai/

  • Home-Keras Documentation. (2019). Retrieved from https://keras.io/

  • Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., & Muller, P. (2019). Deep learning for time series classification: A review. Data Mining and Knowledge Discovery, 33(4), 917–963. https://doi.org/10.1007/s10618-019-00619-1.

    Article  MathSciNet  Google Scholar 

  • Jiang, Y., Bosch, N., Baker, R., Paquette, L., Ocumpaugh, J., Andres, J. M. A. L., Moore, A. L., & Biswas, G. (2018). Expert feature-engineering vs. deep neural networks: Which is better for sensor-free affect detection? In Proceedings of the 19th international conference on artificial intelligence in education (pp. 198–211). London, UK: Springer.

    Chapter  Google Scholar 

  • Jordan, M., & Mitchell, T. (2015). Machine learning: Trends, perspectives, and prospects. Science, 349(6245), 255–260. https://doi.org/10.1126/science.aaa8415.

    Article  MathSciNet  MATH  Google Scholar 

  • Kotsiantis, S. B. (2007). Supervised machine learning: A review of classification techniques. In I. Maglogiannis et al. (Eds.), Emerging artificial intelligence applications in computer engineering (pp. 3–24). Amsterdam, Netherlands: IOS Press.

    Google Scholar 

  • LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444. https://doi.org/10.1038/nature14539.

    Article  Google Scholar 

  • Lemay, D. J., & Doleck, T. (2019). Grade prediction of weekly assignments in MOOCs: Mining Video-Viewing Behavior. Education and Information Technologies. https://doi.org/10.1007/s10639-019-10022-4.

  • Lykourentzou, I., Giannoukos, I., Nikolopoulos, V., Mpardis, G., & Loumos, V. (2009). Dropout prediction in e-learning courses through the combination of machine learning techniques. Computers & Education, 53(3), 950–965. https://doi.org/10.1016/j.compedu.2009.05.010.

    Article  Google Scholar 

  • Mao, Y., Lin, C., & Chi, M. (2018). Deep learning vs. Bayesian knowledge tracing: Student models for interventions. JEDM | Journal of Educational Data Mining, 10(2), 28–54.

    Google Scholar 

  • Marcus, G. (2018). Deep learning: A critical appraisal. arXiv preprint arXiv:1801.00631.

  • Nguyen, G., Dlugolinsky, S., Bobák, M., Tran, V., López García, Á., Heredia, I., et al. (2019). Machine learning and deep learning frameworks and libraries for large-scale data mining: A survey. Artificial Intelligence Review, 52(1), 77–124. https://doi.org/10.1007/s10462-018-09679-z.

    Article  Google Scholar 

  • Pang, B., Nijkamp, E., & Wu, Y. (2019). Deep learning with TensorFlow: A Review. Journal of Educational and Behavioral Statistics. https://doi.org/10.3102/1076998619872761.

  • Papamitsiou, Z., & Economides, A. (2014). Learning analytics and educational data mining in practice: A systematic literature review of empirical evidence. Educational Technology & Society, 17(4), 49–64.

    Google Scholar 

  • Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., et al. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830.

    MathSciNet  MATH  Google Scholar 

  • Peña-Ayala, A. (2014). Educational data mining: A survey and a data mining-based analysis of recent works. Expert Systems with Applications, 41(4), 1432–1462. https://doi.org/10.1016/j.eswa.2013.08.042.

    Article  Google Scholar 

  • Piech, C., Bassen, J., Huang, J., Ganguli, S., Sahami, M., Guibas, L. J., & Sohl-Dickstein, J. (2015). Deep knowledge tracing. In Advances in Neural Information Processing Systems (pp. 505–513).

  • Poitras, E. G., Lajoie, S. P., Doleck, T., & Jarrell, A. (2016). Subgroup discovery with user interaction data: An empirically guided approach to improving intelligent tutoring systems. Educational Technology & Society, 19(2), 204–214.

    Google Scholar 

  • PyTorch. (2019). Retrieved from https://pytorch.org/

  • Rajni, J., & Malaya, D. (2015). Predictive analytics in a higher education context. IT Professional, 17(4), 24–33. https://doi.org/10.1109/mitp.2015.68.

    Article  Google Scholar 

  • Romero, C., & Ventura, S. (2016). Educational data science in massive open online courses. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 7(1), e1187. https://doi.org/10.1002/widm.1187.

  • Siemens, G., & Baker, R. S. (2012). Learning analytics and educational data mining: Towards communication and collaboration. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 252–254). ACM.

  • Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1), 1929–1958.

    MathSciNet  MATH  Google Scholar 

  • Sünderhauf, N., Brock, O., Scheirer, W., Hadsell, R., Fox, D., Leitner, J., et al. (2018). The limits and potentials of deep learning for robotics. The International Journal of Robotics Research, 37(4–5), 405–420. https://doi.org/10.1177/0278364918770733.

    Article  Google Scholar 

  • TensorFlow. (2019). Retrieved from https://www.tensorflow.org/

  • Theano 1.0.0 documentation. (2019). Retrieved from http://deeplearning.net/software/theano/

  • Wilson, K. H., Karklin, Y., Han, B., Ekanadham, C. (2016). Back to the basics: Bayesian extensions of IRT outperform neural networks for proficiency estimation. In Proceedings of Educational Data Mining (pp. 539–544).

  • Xiao, C., Choi, E., & Sun, J. (2018). Opportunities and challenges in developing deep learning models using electronic health records data: A systematic review. Journal of the American Medical Informatics Association, 25(10), 1419–1428. https://doi.org/10.1093/jamia/ocy068.

    Article  Google Scholar 

  • Xiong, X., Zhao, S., Van Inwegen, E. G., & Beck, J. E. (2016). Going deeper with deep knowledge tracing. In Proceedings of 9th International Conference on Educational Data Mining (pp. 545–550).

  • Zhang, Q., Yang, L., Chen, Z., & Li, P. (2018). A survey on deep learning for big data. Information Fusion, 42, 146–157. https://doi.org/10.1016/j.inffus.2017.10.006.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Tenzin Doleck.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Doleck, T., Lemay, D.J., Basnet, R.B. et al. Predictive analytics in education: a comparison of deep learning frameworks. Educ Inf Technol 25, 1951–1963 (2020). https://doi.org/10.1007/s10639-019-10068-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10639-019-10068-4

Keywords