Abstract
In this article we introduce a new combinatorial parameter which generalizes the VC dimension and the fat-shattering dimension, and extends beyond the function-class setup. Using this parameter we establish entropy bounds for subsets of the n-dimensional unit cube, and in particular, we present new bounds on the empirical covering numbers and gaussian averages associated with classes of functions in terms of the fat-shattering dimension.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
N. Alon, S. Ben-David, N. Cesa-Bianchi, D. Hausser, Scale sensitive dimensions, uniform convergence and learnability, Journal of the ACM 44 (1997), 615–631.
M. Anthony, P. L. Bartlett, Neural Network Learning, Theoretical Foundations, Cambridge University Press, 1999.
P. Bartlett, P. Long, Prediction, learning, uniform convergence, and scale-sensitive dimensions, J. Comput. System Sci. 56 (1998), 174–190.
R. M. Dudley, Uniform central limit theorems, Cambridge University Press, 1999.
E. Giné, J. Zinn, Gaussian charachterization of uniform Donsker classes of functions, Annals of Probability, 19 (1991), 758–782.
D. Haussler, Sphere packing numbers for subsets of Boolean n-cube with bounded Vapnik-Chervonenkis dimension, Journal of Combinatorial Theory A 69 (1995), 217–232.
W. Hoeffding, Probability inequalities for sums of bounded random variables, J. Amer. Statist. Assoc. 58 (1963), 13–30.
M. Ledoux and M. Talagrand, Probability in Banach spaces, Springer, 1991.
S. Mendelson, Rademacher averages and phase transitions in Glivenko-Cantelli classes, IEEE transactions on Information Thery, Jan 2002.
S. Mendelson, Improving the sample complexity using global data To appear, IEEE transactions on Information Theory.
S. Mendelson, Geometric parameters of Kernel Machines, These proceedings.
V. Milman, G. Schechtman, Asymptotic theory of finite dimensional normed spaces, Lecture Notes in Math., vol. 1200, Springer Verlag, 1986.
A. Pajor, Sous espaces l 1 n des espaces de Banach, Hermann, Paris, 1985.
G. Pisier, The volume of convex bodies and Banach space geometry, Cambridge University Press, 1989.
M. Talagrand, Type, infratype, and Elton-Pajor Theorem, Inventiones Math. 107 (1992), 41–59.
M. Talagrand, Sharper bounds for Gaussian and empirical processes, Annals of Probability, 22(1), 28–76, 1994.
V. Vapnik, A. Chervonenkis, Necessary and sufficient conditions for uniform convergence of means to mathematical expectations, Theory Prob. Applic. 26(3), 532–553, 1971.
A. W. Van-der-Vaart, J. A. Wellner, Weak convergence and Empirical Processes, Springer-Verlag, 1996.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Mendelson, S., Vershynin, R. (2002). Entropy, Combinatorial Dimensions and Random Averages. In: Kivinen, J., Sloan, R.H. (eds) Computational Learning Theory. COLT 2002. Lecture Notes in Computer Science(), vol 2375. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45435-7_2
Download citation
DOI: https://doi.org/10.1007/3-540-45435-7_2
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-43836-6
Online ISBN: 978-3-540-45435-9
eBook Packages: Springer Book Archive