Skip to main content
SpringerLink
Log in

Pseudo-dimension and entropy of manifolds formed by affine-invariant dictionary

  • Published:
Advances in Computational Mathematics Aims and scope Submit manuscript

Abstract

We consider the manifolds H n(φ) formed by all possible linear combinations of n functions from the set {φ(A⋅+b)}, where x→Ax+b is arbitrary affine mapping in the space ℝd. For example, neural networks and radial basis functions are the manifolds of type H n(φ). We obtain estimates for pseudo-dimension of the manifold H n(φ) for wide collection of the generator function φ. The estimates have the order O(d 2 n) in degree scale, that is the order is proportional to number of parameters of the manifold H n(φ). Moreover the estimates for ɛ-entropy of the manifold H n(φ) are obtained.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. M. Anthony and P.L. Bartlett, Neural Networks Learning: Theoretical Foundation (Cambridge Univ. Press, Cambridge, 1999).

    Google Scholar 

  2. P.L. Bartlett, V. Maiorov and R. Meir, Almost linear VC dimension bounds for piecewise polynomial networks, Neural Networks 10 (1998) 2159–2173.

    Google Scholar 

  3. M.D. Buhmann, Multivariate Approximation and Application, eds. N. Dyn et al. (Cambridge Univ. Press, Cambridge, 2001) pp. 25–43.

  4. F. Cucker and S. Smale, On the mathematical foundations on learning, Bull. Amer. Math. Soc. 39 (2001) 1–49.

    Article  MathSciNet  Google Scholar 

  5. R. DeVore and G. Lorentz, Constructive Approximation (Springer, Berlin, 1991).

    MATH  Google Scholar 

  6. D. Haussler, Sphere packing numbers for subset of the Boolean n-cube with bounded Vapnik–Chervonenkis dimension, J. Combinatorical Theory 69 (1995) 217–232.

    Article  MATH  MathSciNet  Google Scholar 

  7. M. Karpinski and A.J. Macintyre, Polynomial bounds for VC dimension of sigmoidal neural networks, in: Proc. of the 27th ACM Symp. Thy. of Computing (1995) pp. 200–208.

  8. V. Maiorov, Approximation by neural networks and learning theory, submitted.

  9. V. Maiorov and R. Meir, Lower bounds for multivariate approximation by affine-invariant dictionaries, IEEE Trans. Inform. Theory 47 (2001) 1569–1575.

    Article  MATH  MathSciNet  Google Scholar 

  10. V. Maiorov and A. Pinkus, Lower bounds for approximation by MLP neural networks, Neurocomputing 25 (1999) 81–91.

    Article  MATH  Google Scholar 

  11. S. Mendelson and R. Vershinin, Enropy and the combinatorial dimension, Inventiones Mathematicae 152 (2003) 37–55.

    Article  MATH  MathSciNet  Google Scholar 

  12. J. Milnor, On the Betti numbers of real varieties, Proc. Amer. Math. Soc. 15 (1964) 275–280.

    Article  MATH  MathSciNet  Google Scholar 

  13. I.G. Petrovskii and O.A. Oleinik, On the topology of real algebraic surfaces, Izv. Akad. Nauk SSSR Ser. Mat. 13 (1949) 389–402.

    MATH  MathSciNet  Google Scholar 

  14. A. Pinkus, Approximation theory of the MLP model in neural networks, Acta Numerica (Cambridge Univ. Press, Cambridge, 1999) 1–52.

  15. M.J.D. Powell, The of radial basis approximation in 1990, in: Advances in Numerical Analysis, Vol. 2, ed. W.A. Light (Oxford Univ. Press, Oxford) pp. 105–210.

  16. S. Smale, Mathematical problems for the next century, in: Mathematics: Frontiers and Perspectives, eds. V. Arnold, M. Atiyah, P. Lax and B. Vfzur (AMS, Providence, RI, 2000) CMP 13 (2000) 271–294.

  17. M. Talagrand, Vapnik–Chervonenkis type conditions and uniform Donsker classes of functions, Ann. Probab., to appear.

  18. V. Vapnik and A. Chervonenkis, Necessary and sufficient conditions for the uniform convergence of empirical means to their expectations, Theory Probab. Appl. 3 (1981) 532–553.

    Google Scholar 

  19. M. Vidyasagar, A Theory of Learning and Generalization (Springer, London, 1997).

    MATH  Google Scholar 

  20. H.E. Warren, Lower bounds for approximation by nonlinear manifold, Trans. Amer. Math. Soc. 133 (1968) 167–178.

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Technion I.I.T., 32000, Haifa, Israel

    V. Maiorov

Authors
  1. V. Maiorov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to V. Maiorov.

Additional information

Communicated by C.A. Micchelli

Mathematics subject classifications (2000)

41A46, 41A50, 42A61, 42C10

V. Maiorov: Supported by the Center for Absorption in Science, Ministry of Immigrant Absorption, State of Israel.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Maiorov, V. Pseudo-dimension and entropy of manifolds formed by affine-invariant dictionary. Adv Comput Math 25, 435–450 (2006). https://doi.org/10.1007/s10444-004-7645-9

Download citation

  • Received:

  • Accepted:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10444-004-7645-9

Keywords

Search

Navigation