Abstract
Kernel–based methods are powerful for high dimensional function representation. The theory of such methods rests upon their attractive mathematical properties whose setting is in Hilbert spaces of functions. It is natural to consider what the corresponding circumstances would be in Banach spaces. Led by this question we provide theoretical justifications to enhance kernel–based methods with function composition. We explore regularization in Banach spaces and show how this function representation naturally arises in that problem. Furthermore, we provide circumstances in which these representations are dense relative to the uniform norm and discuss how the parameters in such representations may be used to fit data.
This work was supported by NSF Grant No. ITR-0312113.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Bennett, K., Bredensteiner: Duality and geometry in support vector machine classifiers. In: Langley, P. (ed.) Proc. of the 17–th Int. Conf. on Machine Learning, pp. 57–63. Morgan Kaufmann, San Francisco (2000)
Canu, S., Mary, X., Rakotomamonjy, A.: Functional learning through kernel. In: Suykens, J., et al. (eds.) In Advances in Learning Theory: Methods, Models and ApplicationsNATO Science Series III: Computer and Systems Sciences, vol. 190, pp. 89–110. IOS Press, Amsterdam (2003)
Carnicer, J.M., Bastero, J.: On best interpolation in Orlicz spaces. Approx. Theory and its Appl. 10(4), 72–83 (1994)
Dahmen, W., Micchelli, C.A.: Some remarks on ridge functions. Approx. Theory and its Appl. 3, 139–143 (1987)
Deutsch, F.: Best Approximation in inner Product Spaces CMS Books in Mathematics. Springer, Heidelberg (2001)
Hein, M., Bousquet, O.: Maximal Margin Classification for Metric Spaces. In: Proc. of the 16–th Annual Conference on Computational Learning Theory, COLT (2003)
Kimber, D., Long, P.M.: On-line learning of smooth functions of a single variable. Theoretical Computer Science 148(1), 141–156 (1995)
Lorenz, G.G.: Approximation of Functions, 2nd edn. Chelsea, New York (1986)
Gentile, C.: A new approach to maximal margin classification algorithms. Journal of Machine Learning Research 2, 213–242 (2001)
Leshno, M., Schocken, S.: Multilayer Feedforward Networks with a Non–Polynomial Activation Function can Approximate any Function. Neural Networks 6, 861–867 (1993)
Matousek, J.: Using the Borsuk-Ulam Theorem: Lectures on Topological Methods in Combinatorics and Geometry. Springer, Berlin (2003)
Mhaskar, H.N., Micchelli, C.A.: Approximation by superposition of sigmoidal functions. Advances in Applied Mathematics 13, 350–373 (1992)
Micchelli, C.A., Pontil, M.: A function representation for learning in Banach spaces. Research Note RN/04/05, Dept. of Computer Science, UCL (February 2004)
Micchelli, C.A. Pontil, M.: Regularization algorithms for learning theory. Working paper, Dept. of Computer Science, UCL (2004)
Micchelli, C.A., Utreras, F.I.: Smoothing and interpolation in a convex subset of a hilbert space. SIAM J. of Scientific and Statistical Computing 9, 728–746 (1988)
Morrison, T.J.: Functional Analysis: An Introduction to Banach Space Theory. John Wiley Inc. New York (2001)
Orlicz, W.: Linear Functional Analysis. World Scientific, Singapore (1990)
Pinkus, A.: n–Widths in Approximation Theory. Springer, Ergebnisse (1985)
Pinkus, A.: Approximation theory of the MLP model in neural networks. Acta Numerica 8, 143–196 (1999)
Rao, M.M., Ren, Z.D.R.: Theory of Orlicz Spaces. Marcel Dekker, Inc. New York (1992)
Royden, H.L.: Real Analysis, 3rd edn. Macmillan Publishing Company, New York (1988)
Schölkopf, B., Smola, A.J.: Learning with Kernels. The MIT Press, Cambridge (2002)
Vapnik, V.: The Nature of Statistical Learning Theory, 2nd edn. Springer, New York (1999)
von Luxburg, U., Bousquet, O.: Distance–based classification with Lipschitz functions. In: Proc. of the 16–th Annual Conference on Computational Learning Theory, COLT (2003)
Wahba, G.: Splines Models for Observational Data. Series in Applied Mathematics, SIAM, Philadelphia, vol. 59 (1990)
Zhang, T.: On the dual formulation of regularized linear systems with convex risks. Machine Learning 46, 91–129 (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Micchelli, C.A., Pontil, M. (2004). A Function Representation for Learning in Banach Spaces. In: Shawe-Taylor, J., Singer, Y. (eds) Learning Theory. COLT 2004. Lecture Notes in Computer Science(), vol 3120. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-27819-1_18
Download citation
DOI: https://doi.org/10.1007/978-3-540-27819-1_18
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-22282-8
Online ISBN: 978-3-540-27819-1
eBook Packages: Springer Book Archive