Abstract
Dimensionality reduction is the task of mapping high-dimensional patterns to low-dimensional spaces while maintaining important information. In this paper, we introduce a hybrid dimensionality reduction method that is based on the weighted average of the normalized distance matrices of two or more embeddings. Multi-dimensional scaling embeds the weighted average distance matrix in a low-dimensional space. The approach allows the hybridization of arbitrary point-wise embeddings. Instances of the hybrid algorithm template use principal component analysis, multi-dimensional scaling, and locally linear embedding. The variants are experimentally compared using three dimensionality reduction measures, i.e., the Shepard-Kruskal scaling, a co-ranking matrix measure, and the nearest neighbor regression error in presence of label information. The results show that the hybrid approaches outperform their native pendants in the majority of the experiments.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Dasgupta, S.: Experiments with random projection. In: Uncertainty in Artificial Intelligence (UAI), pp. 143–151. Morgan Kaufmann Publishers Inc. (2000)
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer, Berlin (2009)
Hotelling, H.: Analysis of a complex of statistical variables into principal components. J. Educ. Psychol. 24(6), 417–441 (1933)
Jolliffe, I.: Principal component analysis. In: Springer Series in Statistics. Springer, New York (1986)
Kramer, O.: Hybrid manifold clustering with evolutionary tuning. In: Applications of Evolutionary Computation, EvoApplications, pp. 481–490 (2015)
Kruskal, J.B.: Nonmetric multidimensional scaling: a numerical method. Psychometrika 29, 1–27 (1964)
Lee, J.A., Verleysen, M.: Nonlinear Dimensionality Reduction. Springer (2007)
Lee, J.A., Verleysen, M.: Quality assessment of dimensionality reduction: rank-based criteria. Neurocomputing 72(7–9), 1431–1443 (2009)
Lueks, W., Mokbel, B., Biehl, M., Hammer, B.: How to evaluate dimensionality reduction?—Improving the co-ranking matrix. CoRR (2011)
Moon, S., Qi, H.: Hybrid dimensionality reduction method based on support vector machine and independent component analysis. IEEE Trans. Neural Netw. Learn. Syst. 23(5), 749–761 (2012)
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12, 2825–2830 (2011)
Rossi, F.: How many dissimilarity/kernel self organizing map variants do we need? In: Advances in Self-Organizing Maps and Learning Vector Quantization, Workshop on Self-Organizing Maps (WSOM) (2014)
Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 290, 2323–2326 (2000)
StatLib-Datasets Archive. http://lib.stat.cmu.edu/datasets/ (2014)
Tenenbaum, J.B., Silva, V.D., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Science 290, 2319–2323 (2000)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendix A: Benchmark Problems
Appendix A: Benchmark Problems
The experiments in this paper are based on the following data sets:
-
The Swiss Roll is a simple artificial data set with \(d=3\) that allows a visualization of neighborhoods with colored patterns and label information based on pattern colors from scikit-learn [11].
-
The Housing data set, also known as California Housing from the StatLib repository [14] comprises 20,640 8-dimensional patterns and one label.
-
Friedman is the high-dimensional regression problem Friedman #1 generated with scikit-learn [11] with \(d=500\).
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Kramer, O. (2016). Dimensionality Reduction Hybridizations with Multi-dimensional Scaling. In: Merényi, E., Mendenhall, M., O'Driscoll, P. (eds) Advances in Self-Organizing Maps and Learning Vector Quantization. Advances in Intelligent Systems and Computing, vol 428. Springer, Cham. https://doi.org/10.1007/978-3-319-28518-4_13
Download citation
DOI: https://doi.org/10.1007/978-3-319-28518-4_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-28517-7
Online ISBN: 978-3-319-28518-4
eBook Packages: EngineeringEngineering (R0)