Abstract
The amount of electronic information as well as the size and dimensionality of data sets have increased tremendously. Consequently, dimension reduction and visualization techniques have become increasingly popular in recent years. Dimension reduction is typically connected with loss of information. In supervised classification problems, class labels can be used to minimize the loss of information concerning the specific task. The aim is to preserve and potentially enhance the discrimination of classes in lower dimensions. Here we propose a prototype-based local relevance learning scheme, that results in an efficient nonlinear discriminative dimension reduction of labeled data sets. The method is introduced and discussed in terms of artificial and real world data sets.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Van der Maaten, L.J.P., Postma, E.O., Van den Herik, H.J.: Dimensionality Reduction: A Comparative Review (2007), http://ticc.uvt.nl/~lvdrmaaten/Laurens_van_der_Maaten/Matlab_Toolbox_for_Dimensionality_Reduction_files/Paper.pdf
Bunte, K., Schneider, P., Hammer, B., Schleif, F.-M., Villmann, T., Biehl, M.: Discriminative Visualization by Limited Rank Matrix Learning. Machine Learning Reports 2, 37–51 (2008), http://www.uni-leipzig.de/~compint/mlr/mlr_03_2008.pdf
Fukunaga, K.: Introduction to Statistical Pattern Recognition. Academic Press, New York (1990)
Villmann, T., Hammer, B., Schleif, F.M., Geweniger, T., Hermann, W.: Fuzzy classification by fuzzy labeled neural gas. Neural Networks 19(6-7), 772–779 (2006)
Kontkanen, P., Lahtinen, J., Myllymäki, P., Silander, T., Tirri, H.: Supervised model-based visualization of high-dimensional data. Intell. Data Anal. 4(3,4), 213–227 (2000)
Iwata, T., Saito, K., Ueda, N., Stromsten, S., Griffiths, T.L., Tenenbaum, J.B.: Parametric Embedding for Class Visualization. Neural Comp. 19(9), 2536–2556 (2007)
Kohonen, T.: Self-Organizing Maps, 2nd edn. Springer, Heidelberg (1997)
Hammer, B., Villmann, T.: Generalized relevance learning vector quantization. Neural Networks 15(8-9), 1059–1068 (2002)
Schneider, P., Biehl, M., Hammer, B.: Relevance Matrices in LVQ. In: Proc. of European Symposium on Artificial Neural Networks (ESANN), pp. 37–42 (2007)
Sato, A.S., Yamada, K.: Generalized learning vector quantization. In: NIPS, vol. 8, pp. 423–429 (1996)
Schneider, P., Bunte, K., Hammer, B., Villmann, T., Biehl, M.: Regularization in matrix relevance learning. Machine Learning Reports 2, 19–36 (2008), http://www.uni-leipzig.de/~compint/mlr/mlr_02_2008.pdf
Brand, M.: Charting a manifold. In: NIPS, vol. 15, pp. 961–968 (2003)
Tenenbaum, J.B., de Silva, V., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Science 290(5500), 2319–2323 (2000)
De Silva, V., Tenebaum, J.B.: Global versus local methods in nonlinear dimensionality reduction. In: Advances in Neural Information Processing System, pp. 705–712. MIT Press, Cambridge (2002)
Roweis, S.T., Saul, L.K.: Nonlinear Dimensionality Reduction by Locally Linear Embedding. Science 290(5500), 2323–2326 (2000)
Saul, L.K., Roweis, S.T.: Think globally, fit locally: unsupervised learning of nonlinear manifolds. Journal of Machine Learning Research 4, 119–155 (2003)
Hinton, G., Roweis, S.T.: Stochastic neighbor embedding. In: Advances in Neural Information Processing Systems, vol. 15, pp. 833–840 (2003)
Lee, J.A., Archambeau, C., Verleysen, M.: Locally linear embedding versus Isotop. In: 11th European Symposium on Artificial Neural Networks, pp. 527–534 (2003)
Van der Maaten, L.J.P., Hinton, G.E.: Visualizing High-Dimensional Data Using t-SNE. Journal of Machine Learning Research 9, 2579–2605 (2008)
Newman, D.J., Hettich, S., Blake, C.L., Merz, C.J.: UCI Repository of machine learning databases. University of California, Department of Information and Computer Science (1998), http://archive.ics.uci.edu/ml/ (last visit 19.04.2008)
Baudat, G., Anouar, F.: Generalized Discriminant Analysis Using a Kernel Approach. Neural Computation 12(10), 2385–2404 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bunte, K., Hammer, B., Biehl, M. (2009). Nonlinear Dimension Reduction and Visualization of Labeled Data. In: Jiang, X., Petkov, N. (eds) Computer Analysis of Images and Patterns. CAIP 2009. Lecture Notes in Computer Science, vol 5702. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03767-2_141
Download citation
DOI: https://doi.org/10.1007/978-3-642-03767-2_141
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-03766-5
Online ISBN: 978-3-642-03767-2
eBook Packages: Computer ScienceComputer Science (R0)