Abstract
The eigenvalues of the kernel matrix play an important role in a number of kernel methods. It is well known that these eigenvalues converge as the number of samples tends to infinity. We derive a probabilistic finite sample size bound on the approximation error of an individual eigenvalue, which has the important property that the bound scales with the dominate eigenvalue under consideration, reflecting the accurate behavior of the approximation error as predicted by asymptotic results and observed in numerical simulations. Under practical conditions, the bound presented here forms a significant improvement over existing non-scaling bound. Applications of this theoretical finding in kernel matrix selection and kernel target alignment are also presented.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge (2005)
Shawe-Taylor, J., Cristianini, N., Kandola, J.: On the concentration of spectral properties. In: Dietterich, T.G., Becker, S., Ghahramani, Z. (eds.) Advances in Neural Information Processing Systems 14, pp. 511–517. MIT Press, Cambridge (2002)
Schölkopf, B., Smola, A., Muller, K.: Kernel principal component analysis. In: Schölkopf, B., Burges, C., Smola, A. (eds.) Advances in Kernel Methods: Support Vector Machines, pp. 327–352. MIT Press, Cambridge (1998)
Saunders, C., Gammerman, A., Vovk, V.: Ridge regression learning algorithm in dual variables. In: Proceedings of the 15th International Conference on Machine Learning, pp. 24–27 (1998)
Trefethen, L., Bau, D.: Numerical Linear Algebra. SIAM, Philadelphia (1997)
Ng, A.Y., Zheng, A.X., Jordan, M.I.: Link analysis, eigenvectors and stability. In: Proceedings of the 7th International Joint Conference on Artificial Intelligence, pp. 903–910 (2001)
Martin, S.: The numerical stability of kernel methods. Technical report, Sandia National Laboratory, New Mexico (2005)
Jia, L., Liao, S.: Perturbation stability of computing spectral data on kernel matrix. In: The 26th International Conference on Machine Learning, Workshop on Numerical Mathematics on Machine Learning (2009)
Williams, C., Seeger, M.: The effect of the input density distribution on kernel-based classifiers. In: Langley, P. (ed.) Proceedings of the 17th International Conference on Machine Learning, pp. 1159–1166. Morgan Kaufmann, San Francisco (2000)
Bengio, Y., Vincent, P., Paiement, J., Delalleau, O., Ouimet, M., Le Roux, N.: Spectral clustering and kernel PCA are learning eigenfunctions. Technical Report TR 1239, University of Montreal (2003)
Shawe-Taylor, J., Williams, C.K., Cristianini, N., Kandola, J.: On the eigenspectrum of the gram matrix and the generalization error of kernel-PCA. IEEE Transactions on Information Theory 51(7), 2510–2522 (2005)
Zwald, L., Blanchard, G.: On the convergence of eigenspaces in kernel principal component analysis. In: Weiss, Y., Schölkopf, B., Platt, J. (eds.) Advances in Neural Information Processing Systems 18, pp. 1649–1656. MIT Press, Cambridge (2006)
Koltchinskii, V.: Asymptotics of spectral projections of some random matrices approximating integral operators. Progress in Probability 43, 191–227 (1998)
Koltchinskii, V., Giné, E.: Random matrix approximation of spectra of integral operators. Bernoulli 6(1), 113–167 (2000)
Horn, R.A., Johnson, C.R.: Matrix Analysis. Cambridge University Press, Cambridge (1985)
Braun, M.L.: Accurate error bounds for the eigenvalues of the kernel matrix. Journal of Machine Learning Research 7, 2303–2328 (2006)
Braun, M.L.: Spectral Properties of the Kernel Matrix and their Relation to Kernel Methods in Machine Learning. PhD thesis, Bonn University (2005)
Cristianini, N., Kandola, J., Elisseeff, A., Shawe-Taylor, J.: On kernel-target alignment. Journal of Machine Learning Research 1, 1–31 (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Jia, L., Liao, S. (2009). Accurate Probabilistic Error Bound for Eigenvalues of Kernel Matrix. In: Zhou, ZH., Washio, T. (eds) Advances in Machine Learning. ACML 2009. Lecture Notes in Computer Science(), vol 5828. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-05224-8_14
Download citation
DOI: https://doi.org/10.1007/978-3-642-05224-8_14
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-05223-1
Online ISBN: 978-3-642-05224-8
eBook Packages: Computer ScienceComputer Science (R0)