Abstract
Traditional unsupervised dimensionality reduction techniques are widely used in many learning tasks, such as text classification and face recognition. However, in many applications, a few labeled examples are readily available. Thus, semi-supervised dimensionality reduction(SSDR), which could incorporate the label information, has aroused considerable research interests. In this paper, a novel SSDR approach, which employs the harmonic function in a gaussian random field to compute the states of all points, is proposed. It constructs a complete weighted graph, whose edge weights are assigned by the computed states. The linear projection matrix is then derived to maximize the separation of points in different classes. For illustration, we provide some deep theoretical analyses and promising classification results on different kinds of data sets. Compared with other dimensionality reduction approaches, it is more beneficial for classification. Comparing with the transductive harmonic function method, it is inductive and able to deal with new coming data directly.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Collins, M., Dasgupta, S., Schapire, R.: A generalization of principal component analysis to the exponential family. In: NIPS, vol. 13 (2001)
Wang, H., Wang, Z., et al.: Pca plus F-LDA: a new approach for face recognition. IJPRAI 21(6), 1059–1068 (2007)
Saul, L., Roweis, S.: Think globally, fit locally: unsupervised learning of low dimensional manifolds. Journal of Machine Learning Research 4, 119–155 (2003)
Weinberger, K., Saul, L.: Unsupervised learning of image manifolds by semidefinite programming. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2004), vol. 2, pp. 988–995 (2004)
Fukunaga, K.: Introduction to Statistical Pattern Recognition, 2nd edn. Academic Press, London (1990)
Stone, C.: The Dimensionality Reduction Principle for Generalized Additive Models. The Annals of Statistics 14(2), 590–606 (1986)
Yang, X., Fu, H., Zha, H., Barlow, J.L.: Semisupervised nonlinear dimensionality reduction. In: ICML-2006, Pittsburgh, PA, pp. 1065–1072 (2006)
Zhang, D., Zhou, Z., Chen, S.: Semi-supervised dimensionality reduction. In: SIAM Conference on Data Mining, SDM (2007)
Wagstaff, K., Cardie, C., Rogers, S., Schroedl, S.: Constrained k-means clustering with back- ground knowledge. In: Proc. 18th International Conference on Machine Learning (2001)
Cai, D., He, X., Han, J.: Semi-supervised discriminant analysis. In: Proceedings of the International Conference on Computer Vision (2007)
Song, Y., Nie, F., Zhang, C., Xiang, S.: A unified framework for semi-supervised dimensionality reduction. Pattern Recognition 41(9), 2789–2799 (2008)
Zhu, X.: Semi-supervised learning literature survey, Computer Sciences Technical report 1530, Univ. of Wisconsin, Madison (2007)
Zhu, X., Ghahramani, Z., Lafferty, J.: Semi-supervised learning using Gaussian fields and harmonic functions. In: Proceedings of the 20th International Conference on Machine Learning (2003)
Zhu, X.: Semi-Supervised Learning with Graphs. PhD thesis, Carnegie Mellon University, CMU-LTI-05-192 (2005)
Ye, J., Janardan, R., Li, Q., Park, H.: Feature Reduction via Generalized Uncorrelated Linear Discriminant Analysis. IEEE Transactions on Knowledge and Data Engineering 18(10), 1321–1322 (2006)
UMIST face database, http://images.ee.umist.ac.uk/danny/database.html
Yu, K., Bi, J., Tresp, V.: Active learning via transductive experimental design. In: ICML 2006, pp. 1081–1088 (2006)
USPS database, http://www.cs.toronto.edu/~roweis/data.html
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hou, C., Nie, F., Wu, Y. (2011). Semi-supervised Dimensionality Reduction via Harmonic Functions. In: Torra, V., Narakawa, Y., Yin, J., Long, J. (eds) Modeling Decision for Artificial Intelligence. MDAI 2011. Lecture Notes in Computer Science(), vol 6820. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-22589-5_10
Download citation
DOI: https://doi.org/10.1007/978-3-642-22589-5_10
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-22588-8
Online ISBN: 978-3-642-22589-5
eBook Packages: Computer ScienceComputer Science (R0)