Abstract
In recent years manifold methods have attracted a considerable amount of attention in machine learning. However most algorithms in that class may be termed “manifold-motivated” as they lack any explicit theoretical guarantees. In this paper we take a step towards closing the gap between theory and practice for a class of Laplacian-based manifold methods. We show that under certain conditions the graph Laplacian of a point cloud converges to the Laplace-Beltrami operator on the underlying manifold. Theorem 1 contains the first result showing convergence of a random graph Laplacian to manifold Laplacian in the machine learning context.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Belkin, M.: Problems of Learning on Manifolds, The University of Chicago, Ph.D. Dissertation (2003)
Belkin, M., Niyogi, P.: Using Manifold Structure for Partially Labeled Classification. In: NIPS 2002 (2002)
Belkin, M., Niyogi, P.: Laplacian Eigenmaps for Dimensionality Reduction and Data Representation. Neural Computation 15(6), 1373–1396 (2003)
Belkin, M., Niyogi, P., Sindhwani, V.: On Manifold Regularization, AI Stats (2005)
Bengio, Y., Paiement, J.-F., Vincent, P., Delalleau, O., Le Roux, N., Ouimet, M.: Out-of-Sample Extensions for LLE, Isomap, MDS, Eigenmaps, and Spectral Clustering. In: NIPS 2003 (2003)
Bernstein, M., de Silva, V., Langford, J.C., Tenenbaum, J.B.: Graph approximations to geodesics on embedded manifolds, Technical Report (2000)
Bousquet, O., Chapelle, O., Hein, M.: Measure Based Regularization. In: NIPS 2003 (2003)
Bengio, Y., Paiement, J.-F., Vincent, P.: Out-of-Sample Extensions for LLE, Isomap, MDS, Eigenmaps, and Spectral Clustering. In: NIPS 2003 (2003)
Chapelle, O., Weston, J., Schoelkopf, B.: Cluster Kernels for Semi-Supervised Learning. In: NIPS 2002 (2002)
Chung, F.R.K.: Spectral Graph Theory. In: Regional Conference Series in Mathematics, vol. 92 (1997)
do Carmo, M.: Riemannian Geometry. Birkhäuser, Basel (1992)
Donoho, D.L., Grimes, C.E.: Hessian Eigenmaps: new locally linear embedding techniques for high-dimensional data. In: Proceedings of the National Academy of Arts and Sciences, vol. 100, pp. 5591–5596
Hein, M., Audibert, J.-Y., von Luxburg, U.: From Graphs to Manifolds – Weak and Strong Pointwise Consistency of Graph Laplacians. In: Auer, P., Meir, R. (eds.) COLT 2005. LNCS (LNAI), vol. 3559, pp. 470–485. Springer, Heidelberg (2005)
Meila, M., Shi, J.: Learning segmentation by random walks. In: NIPS 2000 (2000)
Memoli, F., Sapiro, G.: Comparing Point Clouds, IMA Technical Report (2004)
Lafon, S.: Diffusion Maps and Geodesic Harmonics, Ph. D. Thesis, Yale University (2004)
von Luxburg, U., Belkin, M., Bousquet, O.: Consistency of Spectral Clustering, Max Planck Institute for Biological Cybernetics Technical Report TR 134 (2004)
Ng, A., Jordan, M., Weiss, Y.: On Spectral Clustering: Analysis and an Algorithm. In: NIPS 2001 (2001)
Niyogi, P.: Estimating Functional Maps on Riemannian Submanifolds from Sampled Data presented at IPAM Workshop on Multiscale structures in the analysis of High-Dimensional Data (2004), http://www.ipam.ucla.edu/publications/mgaws3/mgaws3_5188.pdf
Niyogi, P., Smale, S., Weinberger, S.: Finding the Homology of Submanifolds with High Confidence from Random Samples, Univ. of Chicago Technical Report TR-2004-08 (2004)
Rosenberg, S.: The Laplacian on a Riemannian Manifold. Cambridge University Press, Cambridge (1997)
Roweis, S.T., Saul, L.K.: Nonlinear Dimensionality Reduction by Locally Linear Embedding. Science 290 (2000)
Smola, A., Kondor, R.: Kernels and Regularization on Graphs. In: COLT/KW 2003 (2003)
Shi, J., Malik, J.: Normalized Cuts and Image Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(8) (2000)
Spielman, D., Teng, S.: Spectral partitioning works: planar graphs and finite element meshes. In: FOCS 1996 (1996)
Szummer, M., Jaakkola, T.: Partially labeled classification with Markov random walks. In: NIPS 2001 (2001)
Tenenbaum, J.B., de Silva, V., Langford, J.C.: A Global Geometric Framework for Nonlinear Dimensionality Reduction. Science 290 (2000)
Kannan, R., Vempala, S., Vetta, A.: On Clusterings - Good, Bad and Spectral, Technical Report, Computer Science Department, Yale University (2000)
Zhou, D., Bousquet, O., Lal, T.N., Weston, J., Schoelkopf, B.: Learning with Local and Global Consistency. In: NIPS 2003 (2003)
Zhu, X., Lafferty, J., Ghahramani, Z.: Semi-supervised learning using Gaussian fields and harmonic functions. In: ICML 2003 (2003)
Zomorodian, A., Carlsson, G.: Computing persistent homology. In: 20th ACM Symposium on Computational Geometry (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Belkin, M., Niyogi, P. (2005). Towards a Theoretical Foundation for Laplacian-Based Manifold Methods. In: Auer, P., Meir, R. (eds) Learning Theory. COLT 2005. Lecture Notes in Computer Science(), vol 3559. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11503415_33
Download citation
DOI: https://doi.org/10.1007/11503415_33
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-26556-6
Online ISBN: 978-3-540-31892-7
eBook Packages: Computer ScienceComputer Science (R0)