Skip to main content
SpringerLink
Log in
Book cover

International Conference on Computational Learning Theory

COLT 2002: Computational Learning Theory pp 14–28Cite as

Entropy, Combinatorial Dimensions and Random Averages

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 2375))

Abstract

In this article we introduce a new combinatorial parameter which generalizes the VC dimension and the fat-shattering dimension, and extends beyond the function-class setup. Using this parameter we establish entropy bounds for subsets of the n-dimensional unit cube, and in particular, we present new bounds on the empirical covering numbers and gaussian averages associated with classes of functions in terms of the fat-shattering dimension.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. N. Alon, S. Ben-David, N. Cesa-Bianchi, D. Hausser, Scale sensitive dimensions, uniform convergence and learnability, Journal of the ACM 44 (1997), 615–631.

    Article  MATH  MathSciNet  Google Scholar 

  2. M. Anthony, P. L. Bartlett, Neural Network Learning, Theoretical Foundations, Cambridge University Press, 1999.

    Google Scholar 

  3. P. Bartlett, P. Long, Prediction, learning, uniform convergence, and scale-sensitive dimensions, J. Comput. System Sci. 56 (1998), 174–190.

    Article  MATH  MathSciNet  Google Scholar 

  4. R. M. Dudley, Uniform central limit theorems, Cambridge University Press, 1999.

    Google Scholar 

  5. E. Giné, J. Zinn, Gaussian charachterization of uniform Donsker classes of functions, Annals of Probability, 19 (1991), 758–782.

    Article  MATH  MathSciNet  Google Scholar 

  6. D. Haussler, Sphere packing numbers for subsets of Boolean n-cube with bounded Vapnik-Chervonenkis dimension, Journal of Combinatorial Theory A 69 (1995), 217–232.

    MathSciNet  Google Scholar 

  7. W. Hoeffding, Probability inequalities for sums of bounded random variables, J. Amer. Statist. Assoc. 58 (1963), 13–30.

    Article  MATH  MathSciNet  Google Scholar 

  8. M. Ledoux and M. Talagrand, Probability in Banach spaces, Springer, 1991.

    Google Scholar 

  9. S. Mendelson, Rademacher averages and phase transitions in Glivenko-Cantelli classes, IEEE transactions on Information Thery, Jan 2002.

    Google Scholar 

  10. S. Mendelson, Improving the sample complexity using global data To appear, IEEE transactions on Information Theory.

    Google Scholar 

  11. S. Mendelson, Geometric parameters of Kernel Machines, These proceedings.

    Google Scholar 

  12. V. Milman, G. Schechtman, Asymptotic theory of finite dimensional normed spaces, Lecture Notes in Math., vol. 1200, Springer Verlag, 1986.

    Google Scholar 

  13. A. Pajor, Sous espaces l 1 n des espaces de Banach, Hermann, Paris, 1985.

    MATH  Google Scholar 

  14. G. Pisier, The volume of convex bodies and Banach space geometry, Cambridge University Press, 1989.

    Google Scholar 

  15. M. Talagrand, Type, infratype, and Elton-Pajor Theorem, Inventiones Math. 107 (1992), 41–59.

    Article  MATH  MathSciNet  Google Scholar 

  16. M. Talagrand, Sharper bounds for Gaussian and empirical processes, Annals of Probability, 22(1), 28–76, 1994.

    Article  MATH  MathSciNet  Google Scholar 

  17. V. Vapnik, A. Chervonenkis, Necessary and sufficient conditions for uniform convergence of means to mathematical expectations, Theory Prob. Applic. 26(3), 532–553, 1971.

    Article  MathSciNet  Google Scholar 

  18. A. W. Van-der-Vaart, J. A. Wellner, Weak convergence and Empirical Processes, Springer-Verlag, 1996.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. RSISE, The Australian National University, Canberra, ACT, 0200, Australia

    Shahar Mendelson

  2. Department of Mathematical Sciences, University of Alberta, Edmonton, Alberta, T6G 2G1, Canada

    Roman Vershynin

Authors
  1. Shahar Mendelson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Roman Vershynin
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Research School of Information Sciences and Engineering, Australian National University, Canberra, ACT, 0200, Australia

    Jyrki Kivinen

  2. Computer Science Department, University of Illinois at Chicago, 851 S. Morgan St., Chicago, IL, 60607, USA

    Robert H. Sloan

Rights and permissions

Reprints and permissions

Copyright information

© 2002 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Mendelson, S., Vershynin, R. (2002). Entropy, Combinatorial Dimensions and Random Averages. In: Kivinen, J., Sloan, R.H. (eds) Computational Learning Theory. COLT 2002. Lecture Notes in Computer Science(), vol 2375. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45435-7_2

Download citation

  • DOI: https://doi.org/10.1007/3-540-45435-7_2

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-43836-6

  • Online ISBN: 978-3-540-45435-9

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation