Abstract
Gradient Descent is an algorithm very used by Machine Learning methods, as Recommender Systems in Collaborative Filtering. It tries to find the optimal values of some parameters in order to minimize a particular cost function. In our research case, we consider Matrix Factorization as application of Gradient Descent, where the optimal values of two matrices must be calculated for minimizing the Root Mean Squared Error criterion, given a particular training dataset. However, there are two important parameters in Gradient Descent, both constant real numbers, whose values are set without any strict rule and have a certain influence on the algorithm accuracy: the learning rate and regularization factor. In this work we apply a evolutionary metaheuristic for finding the optimal values of these two parameters. To that end, we consider as experimental framework the Prediction Student Performance problem, a problem tackled as Recommender System with training and test datasets extracted for real cases. After performing a direct search of the optimal values, we apply a Genetic Algorithm obtaining best results of the Gradient Descent accuracy with less computational effort.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Murphy, K.P.: Machine Learning. A Probabilistic Perspective. The Massachusetts Institute of Technology Press, Cambridge (2012)
Alpaydin, E.: Introduction to Machine Learning. The Massachusetts Institute of Technology Press, Cambridge (2010)
Jannach, D., Zanker, M., Felfernig, A., Friedrich, G.: Recommender Systems. An Introduction. Cambridge University Press, New York (2011)
Thai-Nghe, N., Drumond, L., Horvath, T., Krohn-Grimberghe, A., Nanopoulos, A., Schmidt-Thieme, L.: Factorization techniques for predicting student performance. In: Educational Recommender Systems and Technologies: Practices and Challenges, pp. 129–153. IGI-Global (2012)
Koren, Y., Bell, R., Volinsky, C.: Matrix factorization techniques for recommender systems. Computer 42, 30–37 (2009)
Koren, Y.: Factor in the neighbors: scalable and accurate collaborative filtering. ACM Trans. Knowl. Discov. from Data 4, 1–24 (2010)
Rendle, S., Schmidt-Thieme, L.: Online-updating regularized kernel matrix factorization models for large-scale recommender systems. In: Proceedings of the 2008 ACM Conference on Recommender Systems, pp. 251–258 (2008)
Bottou, L.: Large-scale machine learning with stochastic gradient descent. In: Lechevallier, Y., Saporta, G. (eds.) Proceedings of COMPSTAT 2010, pp. 177–186. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-7908-2604-3_16
Tempelaar, D.T., et al.: Stability and sensitivity of Learning Analytics based prediction models. In: Helfert, M., Restivo, M.T., Zvacek, S., Uhomoibhi, J. (eds.) 7th International Conference on Computer Supported Education Lisbon, Portugal, pp. 156–166 (2015)
Baker, R.S.J., Corbett, A.T., Aleven, V.: More accurate student modeling through contextual estimation of slip and guess probabilities in bayesian knowledge tracing. In: Woolf, B.P., Aïmeur, E., Nkambou, R., Lajoie, S. (eds.) ITS 2008. LNCS, vol. 5091, pp. 406–415. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-69132-7_44
Zhang, R., Gong, W., Grzeda, V., Yaworski, A., Greenspan, M.: An adaptive learning rate method for improving adaptability of background models. IEEE Signal Proc. Lett. 20, 1266–1269 (2013)
Ranganath, R., Wang, C., Blei, D.M., Xing, E.P.: An adaptive learning rate for stochastic variational inference. In: Proceedings of 30th International Conference on Machine Learning, pp. 1–9 (2013)
Gendreau, M., Potvin, J. (eds.): Handbook of Metaheuristics. Springer, New York (2010). https://doi.org/10.1007/978-1-4419-1665-5
Fogel, G., Corne, D.: Evolutionary Computation in Bioinformatics. Morgan Kaufmann Publishers, San Francisco (2003)
Reeves, C., Rowe, J.: Genetic Algorithms. Principles and perspectives. A guide to GA Theory. Kluwer Academic Publisher, Boston (2003)
Wild, I.: Moodle 3.x Developer’s Guide. Packt Publishing, Birmingham (2017)
Acknowledgments
This work was partially funded by the Government of Extremadura under the project IB16002, and by the AEI (State Research Agency, Spain) and the ERDF (European Regional Development Fund, EU) under the contract TIN2016-76259-P.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Duran-Dominguez, A., Gomez-Pulido, J.A., Rodriguez-Lozano, D. (2018). Improving the Accuracy of Prediction Applications by Efficient Tuning of Gradient Descent Using Genetic Algorithms. In: de Cos Juez, F., et al. Hybrid Artificial Intelligent Systems. HAIS 2018. Lecture Notes in Computer Science(), vol 10870. Springer, Cham. https://doi.org/10.1007/978-3-319-92639-1_18
Download citation
DOI: https://doi.org/10.1007/978-3-319-92639-1_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-92638-4
Online ISBN: 978-3-319-92639-1
eBook Packages: Computer ScienceComputer Science (R0)