Abstract
In this paper, we explore the capabilities of a recently proposed method for non-linear dimensionality reduction and visualization called Locally Linear Embedding (LLE). LLE is proposed as an alternative to the traditional approaches. Its ability to deal with large sizes of high dimensional data and non-iterative way to find the embeddings make it more and more attractive to several researchers. All the studies which investigated and experimented this approach have concluded that LLE is a robust and efficient algorithm when the data lie on a smooth and well-sampled single manifold. None explored the behavior of the algorithm when the data include some noise (or outliers). Here, we show theoretically and empirically that LLE is significantly sensitive to the presence of a few outliers. Then we propose a robust extension to tackle this problem. Further, we investigate the behavior of the LLE algorithm in cases of disjoint manifolds, demonstrate the lack of single global coordinate system and discuss some alternatives.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Roweis, S., Saul, L.: Nonlinear Dimensionality Reduction by Locally Linear Embedding. Science v. (2000) 290, No. 5500, 2323–2326.
Saul, L., Roweis, S.: Think Globally, Fit Locally: Unsupervised Learning of Nonlinear Manifolds. Technical Report MS CIS-02-18, University of Pennsylvania. (2002).
Tenenbaum, J.B., De Silva, V., Langford, J.C.: A Global Geometric Framework for Nonlinear Dimensionality Reduction. Science v. (2000) 290, No. 5500, 2319–2323.
Jolliffe, I.T.: Principal Component Analysis. Springer-Verlag, New York (1986).
Cox, T., Cox, M.: Multidimensional Scaling. Chapman & Hall, 2nd edition (2000).
Kohonen, T.: Self-Organizing Maps. Springer Series in Information Sciences, 3rd edition (2000).
Hastie, T.J., Stuetzle, W.: Principal Curves and Surfaces. Journal of the American Statistical Association (1689), 84(406), 502–516.
Verbeek, J.J., Vlassis, N., Kröse, B.: A K-segments Algorithm for Finding Principal Curves. Pattern Recognition Letters (2002b), 23(8), 1009–1017.
DeMers, D., Cottrell, G.W.: Nonlinear Dimensionality Reduction. Advances in Neural Information Processing Systems, San Mateo, CA (1993) volume 5, 580–587.
Bishop, C., Svensen, M., Williams, C.: GTM: The Generative Topographic Mapping. Neural Computation (1998), 10(1), 215–234.
De Backer, S., Naud, A., Scheunders, P.: Non-Linear Dimensionality Reduction Techniques for Unsupervised Feature Extraction. Pattern Recognition Letters (1998) 19, 711–720.
Li, S., De Vel, O., Coomans, D.: Comparative Performance Analysis of Non-Linear Dimensionality Reduction Methods. Technical Report, James Cook University (1995).
Hadid, A., Kouropteva, O., Pietikäinen, M.: Unsupervised Learning Using Locally Linear Embedding: Experiments in Face Pose Analysis. Proc. 16th International Conference on Pattern Recognition, Quebec, Canada (2002), 1, 111–114.
DeCoste. D.: Visualizing Mercel Kernel Feature Spaces via Kernelized Locally Linear Embedding. 8th International Conference on Neural Information Processing. (2001).
Vlachos, M., Domeniconi, C., Gunopulos, D., Kollios, G., Koudas, N.: Non-Linear Dimensionality Reduction Techniques for Classification and Visualization. Knowledge Discovery and Data Mining. Edmonton, Canada (2002).
Perona, P., Polito, M.: Grouping and Dimensionality Reduction by Locally Linear Embedding. Neural Information Processing Systems NIPS (2001).
Pless, R., Simon, I.: Embedding Images in Non-Flat Spaces. Technical Report WU-CS-01-43, Washington University, December (2001).
Ghahramani, Z., Hinton, G.E.: The EM Algorithm for Mixtures of Factor Analyzers. Technical Report CRG-TR-96-1, Department of Computer Science, University of Toronto (May 1996).
Kambhatla, N., Leen, T. K.: Dimension Reduction by Local Principal Component Analysis. Neural Computation (1997) 9, 1493–1516.
Ojala, T., Mäenpää, T., Pietikäinen, M., Viertola, J., Kyllönen, J., Huovinen, S.: Outex — New Framework for Empirical Evaluation of Texture Analysis Algorithms. Proc. 16th International Conference on Pattern Recognition, Quebec, Canada (2002), 1, 701–706. (http://www.outex.oulu.fi/outex.php).
Ojala, T., Pietikäinen, M., Harwood, D.: A Comparative Study of Texture Measures with Classification Based on Feature Distributions. Pattern Recognition (1996) 29, 51–59.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hadid, A., Pietikäinen, M. (2003). Efficient Locally Linear Embeddings of Imperfect Manifolds. In: Perner, P., Rosenfeld, A. (eds) Machine Learning and Data Mining in Pattern Recognition. MLDM 2003. Lecture Notes in Computer Science, vol 2734. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45065-3_17
Download citation
DOI: https://doi.org/10.1007/3-540-45065-3_17
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40504-7
Online ISBN: 978-3-540-45065-8
eBook Packages: Springer Book Archive