Abstract
Dimension reduction is very important for applications in data mining and machine learning. Dependence maximization based supervised feature extraction (SDMFE) is an effective dimension reduction method proposed recently. A shortcoming of SDMFE is that it can only use labeled data, and does not work well when labeled data are limited. However, in many applications, it is a common case. In this paper, we propose a novel feature extraction method, called Semi-Supervised Dependence Maximization Feature Extraction (SSDMFE), which can utilize simultaneously both labeled and unlabeled data to perform feature extraction. The labeled data are used to maximize the dependence and the unlabeled data are used as regulations with respect to the intrinsic geometric structure of the data. Experiments on several datasets are presented and the results demonstrate that SSDMFE achieves much higher classification accuracy than SDMFE when the amount of labeled data are limited.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Jolliffe, I.: Principal component analysis. Springer, New York (2002)
Fisher, R.A.: The use of multiple measurements in taxonomic problems. Ann. of Eugenics 7, 179–188 (1936)
Liu, X., Wang, Z., Feng, Z., Tang, J.: A Pairwise Covariance-Preserving Projection Method for Dimension Reduction. In: Seventh IEEE International Conference on Data Mining, pp. 223–231 (2007)
He, X., Cai, D., Yan, S., Zhang, H.: Neighborhood preserving embedding. In: Tenth IEEE International Conference on Computer Vision, pp. 1208–1213 (2005)
Liu, X., Yin, J., Feng, Z., Dong, J., Wang, L.: Orthogonal Neighborhood Preserving Embedding for Face Recognition. In: IEEE International Conference on Image Processing, pp. 133–136 (2007)
Cristianini, N., Shawe-Taylor, J.: An introduction to support Vector Machines: and other kernel-based learning methods. Cambridge Univ. Pr., Cambridge (2000)
Mika, S., Ratsch, G., Muller, K.: A mathematical programming approach to the kernel fisher algorithm. Advances in neural information processing systems, 591–597 (2001)
Rosipal, R., Trejo, L.: Kernel partial least squares regression in reproducing kernel hilbert space. The Journal of Machine Learning Research 2, 97–123 (2002)
Gretton, A., Bousquet, O., Smola, A., Scholkopf, B.: Measuring statistical dependence with Hilbert-Schmidt norms. In: Jain, S., Simon, H.U., Tomita, E. (eds.) ALT 2005. LNCS, vol. 3734, pp. 63–77. Springer, Heidelberg (2005)
Smola, A., Gretton, A., Borgwardt, K., Bedo, J.: Supervised feature selection via dependence estimation. In: Proceedings of the 24th International Conference on Machine Learning, pp. 823–830. ACM, New York (2007)
Chen, J., Ji, S., Ceran, B., Li, Q., Wu, M., Ye, J.: Learning subspace kernels for classification. In: KDD, pp. 106–114 (2008)
Zhu, X.: Semi-supervised learning literature survey. Department of Computer Sciences, University of Wisconsin at Madison, Madison (2006)
Zhang, Y., Yeung, D.: Semi-supervised discriminant analysis using robust path-based similarity. In: CVPR, pp. 1–8 (2008)
Sindhwani, V., Selvaraj, S.: Large scale semi-supervised linear support vector machines. In: Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval, pp. 477–484 (2006)
Chen, L., Liao, H., Ko, M., Lin, J., Yu, G.: A new LDA-based face recognition system which can solve the small sample size problem. Pattern recognition 33(10), 1713–1726 (2000)
Smola, A., Gretton, A., Borgwardt, K.: A dependence maximization view of clustering. In: Proceedings of the 24th International Conference on Machine Learning, pp. 815–822. ACM, New York (2007)
Cai, D., He, X., Han, J.: Semi-supervised discriminant analysis. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1–7 (2007)
Chung, F.: Spectral graph theory: American Mathematical Society (1997)
Sim, T., Baker, S., Bsat, M.: The CMU pose, illumination, and expression database. IEEE Transactions on Pattern Analysis and Machine Intelligence 25(12), 1615–1618 (2002)
Georghiades, A., Belhumeur, P., Kriegman, D.: From few to many: illumination cone models for face recognitionunder variable lighting and pose. IEEE Transactions on Pattern Analysis and Machine Intelligence 23(6), 643–660 (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Liu, X., Tang, J., Liu, J., Feng, Z., Wang, Z. (2009). Semi-supervised Discriminant Analysis Based on Dependence Estimation. In: Huang, R., Yang, Q., Pei, J., Gama, J., Meng, X., Li, X. (eds) Advanced Data Mining and Applications. ADMA 2009. Lecture Notes in Computer Science(), vol 5678. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03348-3_24
Download citation
DOI: https://doi.org/10.1007/978-3-642-03348-3_24
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-03347-6
Online ISBN: 978-3-642-03348-3
eBook Packages: Computer ScienceComputer Science (R0)