Abstract
Dimensionality reduction as the available method to overcome the “curses of dimensionality” has attracted wide attention. However, the pervious studies treat the visualization and the subsequent classification performance separately. In this case, we do not know whether there is a underlying relationship (i.e., direct proportion) between visualization and the followed classification performance.
In this paper we compare several dimensionality reduction techniques on three different types of data sets: 1) Benchmark, 2) Image, and 3) Text data. Specifically, to intuitively evaluate the quality of the dimension reduced data, the visualization analysis is carried out, in which we use a covariance matrix related criteria to quantify the information content of the data. Moreover, we also consider the classification accuracy in different latent spaces (i.e., dimensions) as another performance criterion to further analysis the relationship between the information content and the subsequent classification task. The experimental results show that there is no direct proportion relationship between the information content and the further classification performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Hotelling, H.: Analysis of a complex of statistical variables into principal components. J. Educ. Psych. 24 (1933)
Torgerson, W.: Multidimensional scaling: I. theory and method. Psychometrika 17(4), 401–419 (1952)
Belhumeur, P.N., Hespanha, J.P., Kriegman, D.J.: Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 711–720 (1997)
Tenenbaum, J.B., Silva, V., Langford, J.C.: A Global Geometric Framework for Nonlinear Dimensionality Reduction. Science 290, 2319–2323 (2000)
Weinberger, K.Q., Sha, F., Saul, L.K.: Learning a kernel matrix for nonlinear dimensionality reduction. In: Proceedings of the Twenty-First International Conference on Machine Learning, ICML 2004, pp. 839–846 (2004)
Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. SCIENCE 290, 2323–2326 (2000)
Hinton, G.E., Roweis, S.T.: Stochastic neighbor embedding. In: Advances in Neural Information Processing Systems 15 Neural Information Processing Systems, NIPS 2002, Vancouver, British Columbia, Canada, December 9-14, pp. 833–840 (2002)
Lawrence, N.: Probabilistic non-linear principal component analysis with gaussian process latent variable models. J. Mach. Learn. Res. 6, 1783–1816 (2005)
Fodor, I.K.: A survey of dimension reduction techniques (2002)
van der Maaten, L., Postma, E.O., van den Herik, H.J.: Dimensionality reduction: A comparative review (2008)
van der Maaten, L., Hinton, G.: Visualizing data using t-sne. J. Mach. Learn. Res. 9, 2579–2605 (2008)
Jiang, X., Gao, J., Hong, X., Cai, Z.: Gaussian processes autoencoder for dimensionality reduction. In: Tseng, V.S., Ho, T.B., Zhou, Z.-H., Chen, A.L.P., Kao, H.-Y. (eds.) PAKDD 2014, Part II. LNCS, vol. 8444, pp. 62–73. Springer, Heidelberg (2014)
Jiang, X., Gao, J., Wang, T., Zheng, L.: Supervised latent linear gaussian process latent variable model for dimensionality reduction. IEEE Transactions on Systems, Man, and Cybernetics, Part B 42(6), 1620–1632 (2012)
Arzuaga-Cruz, E., Jimenez-Rodriguez, L.O., Velez-Reyes, M., Kaeli, D., Rodriguez-Diaz, E., Velazquez-Santana, H.T., Castrodad-Carrau, A., Santos-Campis, L.E., Santiago, C.: A matlab toolbox for hyperspectral image analysis. In: Proceedings 2004 IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2004, vol. 7, pp. 4839–4842 (September 2004)
Hersh, W., Buckley, C., Leone, T.J., Hickam, D.: Ohsumed: An interactive retrieval evaluation and new large test collection for research. In: Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 1994, pp. 192–201. Springer-Verlag New York, Inc., New York (1994)
Chang, C.-C., Lin, C.-J.: Libsvm: A library for support vector machines. ACM Trans. Intell. Syst. Technol. 2(3), 27:1–27:27 (2011)
Wu, J., Zhu, X., Zhang, C., Yu, P.S.: Bag constrained structure pattern mining for multi-graph classification. IEEE Transactions on Knowledge and Data Engineering 26(10), 2382–2396 (2014)
Wu, J., Cai, Z., Zhu, X.: Self-adaptive probability estimation for naive bayes classification. In: IJCNN, pp. 1–8 (2013)
Wu, J., Cai, Z.: Attribute weighting via differential evolution algorithm for attribute weighted naive bayes (wnb). Journal of Computational Information Systems 7(5), 1672–1679 (2011)
Cook, J., Sutskever, I., Mnih, A., Hinton, G.: Visualizing similarity data with a mixture of maps. In: Society for Artificial Intelligence and Statistics AI and Statistics (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Song, X., Cai, Z., Xu, W. (2014). Dimensionally Reduction: An Experimental Study. In: Luo, X., Yu, J.X., Li, Z. (eds) Advanced Data Mining and Applications. ADMA 2014. Lecture Notes in Computer Science(), vol 8933. Springer, Cham. https://doi.org/10.1007/978-3-319-14717-8_30
Download citation
DOI: https://doi.org/10.1007/978-3-319-14717-8_30
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-14716-1
Online ISBN: 978-3-319-14717-8
eBook Packages: Computer ScienceComputer Science (R0)