Abstract
In this short paper, a theoretical analysis of Occam’s razor formulation through statistical learning theory is presented, showing that pathological situations exist for which regularization may slow down supervised learning instead of making it faster.
G. Cevolani acknowledges support from MIUR PRIN grant no. 201743F9YE.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Without such noise, one would have always a 0 minimum empirical risk in the correct family of models, which would make its detection easier.
References
Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer, New York (2000). https://doi.org/10.1007/978-1-4757-3264-1
Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification. Wiley, Hoboken (2000)
Corfield, D., Schölkopf, B., Vapnik, V.N.: Falsificationism and statistical learning theory: comparing the Popper and Vapnik-Chervonenkis dimensions. J. Gen. Philos. Sci. 4(1), 51–58 (2009)
Wolpert, D.H.: The lack of a priori distinctions between learning algorithms. Neural Comput. 8, 1341–1390 (1996)
Lauc, D.: Machine learning and the philosophical problems of induction. In: Skansi, S. (ed.) Guide to Deep Learning Basics, pp. 93–106. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-37591-1_9
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Bargagli-Stoffi, F., Cevolani, G., Gnecco, G. (2020). Should Simplicity Be Always Preferred to Complexity in Supervised Machine Learning?. In: Nicosia, G., et al. Machine Learning, Optimization, and Data Science. LOD 2020. Lecture Notes in Computer Science(), vol 12565. Springer, Cham. https://doi.org/10.1007/978-3-030-64583-0_6
Download citation
DOI: https://doi.org/10.1007/978-3-030-64583-0_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-64582-3
Online ISBN: 978-3-030-64583-0
eBook Packages: Computer ScienceComputer Science (R0)