Skip to main content
SpringerLink
Log in

Generalization in Learning from Examples

  • Chapter
Challenges for Computational Intelligence

Part of the book series: Studies in Computational Intelligence ((SCI,volume 63))

Summary

Capability of generalization in learning from examples can be modeled using regularization, which has been developed as a tool for improving stability of solutions of inverse problems. Theory of inverse problems has been developed to solve various tasks in applied science such as acoustics, geophysics and computerized tomography. Such problems are typically described by integral operators. It is shown that learning from examples can be reformulated as an inverse problem defined by an evaluation operator. This reformulation allows one to characterize optimal solutions of learning tasks and design learning algorithms based on numerical solutions of systems of linear equations.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Aizerman M A, Braverman E M, Rozonoer L I (1964) Theoretical foundations of potential function method in pattern recognition learning. Automation and Remote Control 28:821-837

    Google Scholar 

  2. Aronszajn N (1950) Theory of reproducing kernels. Transactions of AMS 68:33-404

    MathSciNet  Google Scholar 

  3. Berg C, Christensen J P R, Ressel P (1984) Harmonic Analysis on Semigroups. New York, Springer-Verlag

    MATH  Google Scholar 

  4. Bertero M (1989) Linear inverse and ill-posed problems. Advances in Electronics and Electron Physics 75:1-120

    Google Scholar 

  5. Bjorck A (1996) Numerical methods for least squares problem. SIAM

    Google Scholar 

  6. Boser B, Guyon I, Vapnik V N (1992) A training algorithm for optimal margin clasifiers. In Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory (Ed. Haussler D), pp. 144-152. ACM Press

    Google Scholar 

  7. Cortes C, Vapnik V N (1995) Support-vector networks. Machine Learning 20:273-297

    MATH  Google Scholar 

  8. Courant R, Hilbert D (1962) Methods of Mathematical Physics, vol. 2. New York, Wiley

    MATH  Google Scholar 

  9. Cristianini N, Shawe-Taylor J (2000) An Introduction to Support Vector Machines. Cambridge, Cambridge University Press

    Google Scholar 

  10. Cucker F, Smale S (2002) On the mathematical foundations of learning. Bulletin of the AMS 39:1-49

    Article  MATH  MathSciNet  Google Scholar 

  11. De Mol C (1992) A critical survey of regularized inversion method. In Inverse Problems in Scattering and Imaging (Eds. Bertero M, Pike E R), pp. 346-370. Bristol, Adam Hilger

    Google Scholar 

  12. De Vito E, Rosasco L, Caponnetto A, De Giovannini U, Odone F (2005) Learning from examples as an inverse problem. Journal of Machine Learning Research 6:883-904

    Google Scholar 

  13. Di Chiro G, Brooks R A (1979) The 1979 Nobel prize in physiology or medicine. Science 206:1060-1062

    Article  Google Scholar 

  14. Dunford N, Schwartz J T (1963) Linear Operators. Part II: Spectral Theory. New York, Interscience Publishers

    MATH  Google Scholar 

  15. Engl H W, Hanke M, Neubauer A (2000) Regularization of Inverse Problems. Dordrecht, Kluwer

    Google Scholar 

  16. Friedman A (1982) Modern Analysis. New York, Dover

    MATH  Google Scholar 

  17. Girosi F (1998) An equivalence between sparse approximation and support vector machines. Neural Computation 10:1455-1480 (AI Memo No 1606, MIT)

    Article  Google Scholar 

  18. Girosi F, Jones M, Poggio T (1995) Regularization theory and neural network architectures. Neural Computation 7:219-269

    Article  Google Scholar 

  19. Groetch C W (1977) Generalized Inverses of Linear Operators. New York, Dekker

    Google Scholar 

  20. Hadamard J (1902) Sur les problèmes aux dérivées partielles et leur signification physique. Bulletin of University of Princeton 13:49

    MathSciNet  Google Scholar 

  21. Hansen P C (1998) Rank-Deficient and Discrete Ill-Posed Problems. Philadelphia, SIAM

    Google Scholar 

  22. Kecman V (2001) Learning and Soft Computing. Cambridge, MIT Press.

    MATH  Google Scholar 

  23. Kůrková V (2003) High-dimensional approximation by neural networks. Chapter 4 in Advances in Learning Theory: Methods, Models and Applications (Eds. Suykens J et al.), pp. 69-88. Amsterdam, IOS Press

    Google Scholar 

  24. Kůrková V (2004) Learning from data as an inverse problem. In: COMP-STAT 2004 - Proceedings on Computational Statistics (Ed. Antoch J), pp. 1377-1384. Heidelberg, Physica-Verlag/Springer

    Google Scholar 

  25. Kůrková V (2005) Neural network learning as an inverse problem. Logic Journal of IGPL 13:551-559

    Article  MATH  Google Scholar 

  26. Kůrková V, Sanguineti M (2005) Error estimates for approximate optimization by the extended Ritz method. SIAM Journal on Optimization 15:461-487

    Article  MATH  Google Scholar 

  27. Kůrková V, Sanguineti M (2005) Learning with generalization capability by kernel methods with bounded complexity. Journal of Complexity 21:350-367

    Article  MATH  MathSciNet  Google Scholar 

  28. Moore E H (1920) Abstract. Bull. Amer. Math. Soc. 26:394-395

    Google Scholar 

  29. Parzen E (1966) An approach to time series analysis. Annals of Math. Statistics 32:951-989

    Article  MathSciNet  Google Scholar 

  30. Penrose R (1955) A generalized inverse for matrices. Proc. Cambridge Philos. Soc. 51:406-413

    Article  MATH  MathSciNet  Google Scholar 

  31. Pinkus A (1985) n-width in Approximation Theory. Berlin, Springer-Verlag

    Google Scholar 

  32. Poggio T, Girosi F (1990) Networks for approximation and learning. Proceedings IEEE 78:1481-1497

    Article  Google Scholar 

  33. Poggio T, Smale S (2003) The mathematics of learning: dealing with data. Notices of the AMS 50:536-544

    MathSciNet  Google Scholar 

  34. Popper K (1968) The Logic of Scientific Discovery. New York, Harper Torch Book

    Google Scholar 

  35. Rummelhart D E, Hinton G E, Williams R J (1986) Learning internal representations by error propagation. In Parallel Distributed Processing (Eds. Rummelhart D E, McClelland J L), pp. 318-362. Cambridge, MIT Press

    Google Scholar 

  36. Russo L (2004) The Forgotten Revolution. Berlin, Springer-Verlag

    MATH  Google Scholar 

  37. Schölkopf B, Smola A J (2002) Learning with Kernels - Support Vector Machines, Regularization, Optimization and Beyond. Cambridge, MIT Press

    Google Scholar 

  38. Smith D W, McIntyre R (1982) Husserl and Intentionality: A Study of Mind, Meaning, and Language. Dordrecht and Boston, D. Reidel Publishing Co.

    Google Scholar 

  39. Strichartz R S (2003) A Quide to Distribution Theory and Fourier Transforms. Singapore, World Scientific

    Google Scholar 

  40. Tikhonov A N, Arsenin V Y (1977) Solutions of Ill-posed Problems. Washington, D.C., W.H. Winston

    MATH  Google Scholar 

  41. Wahba G (1990) Splines Models for Observational Data. Philadelphia, SIAM

    Google Scholar 

  42. Werbos P J (1995) Backpropagation: Basics and New Developments. In The Handbook of Brain Theory and Neural Networks (Ed. Arbib M), pp. 134-139. Cambridge, MIT Press

    Google Scholar 

Download references

Authors
  1. Věra Kůrková
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Informatics, Nicolaus Copernicus University, Ul. Grudziadzka 5, 87-100, Torun, Poland

    Włodzisław Duch

  2. Division of Computer Science, School of Computer Engineering, Nanyang Technological University, 639798, Singapore

    Włodzisław Duch

  3. Faculty of Mathematics and Information Sciences, Warsaw University of Technology, Plac Politechniki 1, 00-661, Warsaw, Poland

    Jacek Mańdziuk

Rights and permissions

Reprints and permissions

Copyright information

© 2007 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Kůrková, V. (2007). Generalization in Learning from Examples. In: Duch, W., Mańdziuk, J. (eds) Challenges for Computational Intelligence. Studies in Computational Intelligence, vol 63. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-71984-7_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-71984-7_13

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-71983-0

  • Online ISBN: 978-3-540-71984-7

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation