Abstract
The local linear embedding algorithm (LLE) is a non-linear dimension-reducing technique that is widely used for its computational simplicity and intuitive approach. LLE first linearly reconstructs each input point from its nearest neighbors and then preserves these neighborhood relations in a low-dimensional embedding. We show that the reconstruction weights computed by LLE capture the high-dimensional structure of the neighborhoods, and not the low-dimensional manifold structure. Consequently, the weight vectors are highly sensitive to noise. Moreover, this causes LLE to converge to a linear projection of the input, as opposed to its non-linear embedding goal. To resolve both of these problems, we propose to compute the weight vectors using a low-dimensional neighborhood representation. We call this technique LDR-LLE. We present numerical examples of the perturbation and linear projection problems, and of the improved outputs resulting from the low-dimensional neighborhood representation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 290, 2323–2326 (2000)
Tenenbaum, J.B., de Silva, V., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Science 290, 2319–2323 (2000)
Belkin, M., Niyogi, P.: Laplacian Eigenmaps for dimensionality reduction and data representation. Neural. Comp. 15, 1373–1396 (2003)
Donoho, D.L., Grimes, C.: Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data. Proc. Natl. Acad. Sci. U.S.A. 100, 5591–5596 (2004)
Zhang, Z.Y., Zha, H.Y.: Principal manifolds and nonlinear dimensionality reduction via tangent space alignment. SIAM J. Sci. Comp. 26, 313–338 (2004)
Weinberger, K.Q., Saul, L.K.: Unsupervised learning of image manifolds by semidefinite programming. International Journal of Computer Vision 70(1), 77–90 (2006)
Xu, W., Lifang, X., Dan, Y., Zhiyan, H.: Speech visualization based on locally linear embedding (lle) for the hearing impaired. In: BMEI (2), pp. 502–505 (2008)
Shi, R., Shen, I.F., Chen, W.: Image denoising through locally linear embedding. In: CGIV 2005: Proceedings of the International Conference on Computer Graphics, Imaging and Visualization, pp. 147–152. IEEE Computer Society, Los Alamitos (2005)
Chen, J., Wang, R., Yan, S., Shan, S., Chen, X., Gao, W.: Enhancing human face detection by resampling examples through manifolds. IEEE Transactions on Systems, Man and Cybernetics, Part A 37, 1017–1028 (2007)
L’Heureux, P., Carreau, J., Bengio, Y., Delalleau, O., Yue, S.: Locally linear embedding for dimensionality reduction in qsar. J. Comput. Aided Mol. Des. 18, 475–482 (2004)
Wang, M., Yang, H., Xu, Z., Chou, K.: SLLE for predicting membrane protein types. J. Theor. Biol. 232, 7–15 (2005)
Xu, X., Wu, F., Hu, Z., Luo, A.: A novel method for the determination of redshifts of normal galaxies by non-linear dimensionality reduction. Spectroscopy and Spectral Analysis 26, 182–186 (2006)
Hadid, A., Pietikäinen, M.: Efficient locally linear embeddings of imperfect manifolds. Machine Learning and Data Mining in Pattern Recognition, 188–201 (2003)
Chang, H., Yeung, D.Y.: Robust locally linear embedding. Pattern Recognition 39, 1053–1065 (2006)
Varini, C., Degenhard, A., Nattkemper, T.W.: ISOLLE: LLE with geodesic distance. Neurocomputing 69, 1768–1771 (2006)
Wang, H., Zheng, J., Yao, Z., Li, L.: Improved locally linear embedding through new distance computing. In: Advances in Neural Networks - ISNN 2006, pp. 1326–1333 (2006)
Zhang, Z., Wang, J.: MLLE: Modified locally linear embedding using multiple weights. In: Schölkopf, B., Platt, J., Hoffman, T. (eds.) Advances in Neural Information Processing Systems, vol. 19, pp. 1593–1600. MIT Press, Cambridge (2007)
Saul, L.K., Roweis, S.T.: Think globally, fit locally: Unsupervised learning of low-dimensional manifolds. J. Mach. Learn. Res. 4, 119–155 (2003)
Lee, J.A., Verleysen, M.: Nonlinear Dimensionality Reduction. Springer, Berlin (2007)
Saul, L.K., Roweis, S.T.: Locally linear embedding (LLE) website, http://www.cs.toronto.edu/~roweis/lle/
Golub, G.H., Loan, C.F.V.: Matrix Computations. Johns Hopkins University Press, Baltimore (1983)
Wu, F., Hu, Z.: The LLE and a linear mapping. Pattern Recognition 39, 1799–1804 (2006)
Tenenbaum, J.B., de Silva, V., Langford, J.C.: Isomap website, http://isomap.stanford.edu/
Goldberg, Y., Zakai, A., Kushnir, D., Ritov, Y.: Manifold learning: The price of normalization. J. Mach. Learn. Res. 9, 1909–1939 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Goldberg, Y., Ritov, Y. (2008). LDR-LLE: LLE with Low-Dimensional Neighborhood Representation. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2008. Lecture Notes in Computer Science, vol 5359. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-89646-3_5
Download citation
DOI: https://doi.org/10.1007/978-3-540-89646-3_5
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-89645-6
Online ISBN: 978-3-540-89646-3
eBook Packages: Computer ScienceComputer Science (R0)