Abstract
A high-level relationPopper dimension—( Exclusion dimension—( VC dimension—( between Karl Popper’s ideas on “falsifiability of scientific theories” and the notion of “overfitting”Overfitting in statistical learning theory can be easily traced. However, it was pointed out that at the level of technical details the two concepts are significantly different. One possible explanation that we suggest is that the process of falsification is an active process, whereas statistical learning theory is mainly concerned with supervised learningSupervised learning, which is a passive process of learning from examples arriving from a stationary distribution. We show that concepts that are closer (although still distant) to Karl Popper’s definitions of falsifiability can be found in the domain of learning using membership queries, and derive relations between Popper’s dimension, exclusion dimension, and the VC-dimensionVC dimension.
This work was primarily done when YS was with the Max Planck Institute for Intelligent Systems.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Angluin, D.: Queries revisited. Theor. Comput. Sci. 313(2), 175–194 (2004)
Corfield, D., Schölkopf, B., Vapnik, V.N.: Falsification and statistical learning theory: comparing the Popper and Vapnik-Chervonenkis dimensions. J. Gen. Philos. Sci. 40, 51–58 (2009)
Goldman, S.A., Kearns, M.J.: On the complexity of teaching. J. Comput. Syst. Sci. 50(1), 20–31 (1995)
Popper, K.: Logik der Forschung. Mohr Siebeck, Vienna (1934). English translation: The Logic of Scientific Discovery, 1959
Schölkopf, B., Smola, A.: Learning with Kernels. Support Vector Machines, Regularization, Optimization and Beyond. MIT Press, Cambridge (2002)
Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer, New York (1995)
Vapnik, V.N.: Statistical Learning Theory. Wiley, New York (1998)
Vapnik, V.N., Chervonenkis, A.Y.: On the uniform convergence of relative frequencies of events to their probabilities. Proc. USSR Acad. Sci. 181(4), 781–783 (1968). English translation: Soviet Math. Dokl. 9, 915–918, 1968
Vapnik, V.N., Chervonenkis, A.Y.: On the uniform convergence of relative frequencies of events to their probabilities. Theory Prob. Appl. 16(2), 264–281 (1971)
Vapnik, V.N., Chervonenkis, A.Y.: Theory of pattern recognition. Nauka, Moscow (1974) (in Russian). German translation: W.N. Wapnik, A.Ya. Tschervonenkis (1979), Theorie der Zeichenerkennug, Akademia, Berlin
Acknowledgements
We would like to thank Vladimir Vovk for his careful reading of and comments onPopper dimension—) Exclusion dimension—) VC dimension—) this manuscript.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Seldin, Y., Schölkopf, B. (2013). On the Relations and Differences Between Popper Dimension, Exclusion Dimension and VC-Dimension. In: Schölkopf, B., Luo, Z., Vovk, V. (eds) Empirical Inference. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-41136-6_6
Download citation
DOI: https://doi.org/10.1007/978-3-642-41136-6_6
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-41135-9
Online ISBN: 978-3-642-41136-6
eBook Packages: Computer ScienceComputer Science (R0)