Abstract
In this paper, we propose a novel nonlinear discriminative dimensionality reduction method for clustering high dimensional data. The proposed method first represents the desired low dimensional nonlinear embedding as linear combinations of predefined smooth vectors with respect to data manifold, which are characterized by a weighted graph. Then the optimal combination coefficients and optimal cluster assignment matrix are computed by maximizing between-cluster scatter and minimizing within-cluster scatter simultaneously as well as preserving smoothness of the cluster assignment matrix with respect to the data manifold. We solve the optimization problem in an iterative algorithm, which is proved to be convergent. The contributions of the proposed method are two folds: 1) obtained nonlinear embedding can recover intrinsic manifold structure as well as enhance the cluster structure of the original data; 2) the cluster results can also be obtained in dimensionality reduction procedure. Extensive experiments conducted on UCI data sets and real world data sets have shown the effectiveness of the proposed method for both clustering and visualization high dimensional data set.
This work is supported by National Natural Science Foundation of China (Grant No. 60970034, 60603015), and the Foundation for Author of National Excellent Doctoral Dissertation(Grant No. 2007B4).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Jolliffe, I.: Principal component analysis. Springer, Heidelberg (2002)
Tenenbaum, J., Silva, V., Langford, J.: A global geometric framework for nonlinear dimensionality reduction. Science 290, 2319–2323 (2000)
Roweis, S., Saul, L.: Nonlinear dimensionality reduction by locally linear embedding. Science 290, 2323–2326 (2000)
Belkin, M., Niyogi, P.: Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation 15(6), 1373–1396 (2003)
Duda, R., Hart, P., Stork, D.: Pattern classification. John Wiley & Sons, Chichester (2001)
von Luxburg, U.: A tutorial on spectral clustering. Stat. Comput. 17, 395–416 (2007)
Yan, S., Xu, D., Zhang, B., Zhang, H.J., Yang, Q., Lin, S.: Graph embedding and extensions: A general framework for dimensionality reduction. IEEE Transactions on Pattern Analysis and Machine Intelligence 29(1), 40–51 (2007)
Nie, F., Xu, D., Tsang, I.W., Zhang, C.: Spectral embedded clustering. In: Proceedings of the 21st International Jont Conference on Artifical Intelligence, pp. 1181–1186. Morgan Kaufmann Publishers Inc., San Francisco (2009)
Belkin, M., Niyogi, P.: Semi-supervised learning on riemannian manifolds. Machine Learning 56(1), 209–239 (2004)
Li, Z., Liu, J., Tang, X.: Constrained clustering via spectral regularization. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009. IEEE, Los Alamitos (2009)
Zha, H., He, X., Ding, C., Simon, H.: Spectral relaxation for k-means clustering. In: Advances in Neural Information Processing Systems, vol. 2, pp. 1057–1064. MIT Press, Cambridge (2002)
Jianbo, S.Y., Yu, S.X., Shi, J.: Multiclass spectral clustering. In: Proceedings of 9th IEEE International Conference on Computer Vision 2003, pp. 313–319 (2003)
Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(8), 888–905 (2000)
Ye, J., Zhao, Z., Wu, M.: Discriminative k-means for clustering. In: Platt, J., Koller, D., Singer, Y., Roweis, S. (eds.) Advances in Neural Information Processing Systems, vol. 20, pp. 1649–1656. MIT Press, Cambridge (2008)
Nene, S.A., Nayar, S.K., Murase, H.: Columbia object image library (coil-20). Technical report, Dept. Comput. Sci., Columbia Univ., New York (1996)
Oliva, A., Torralba, A.: Modeling the shape of the scene: A holistic representation of the spatial envelope. International Journal of Computer Vision 42(3), 145–175 (2001)
Zelnik-Manor, L., Perona, P.: Self-tuning spectral clustering. In: Saul, L.K., Weiss, Y., Bottou, L. (eds.) Advances in Neural Information Processing Systems, vol. 17, pp. 1601–1608. MIT Press, Cambridge (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zhan, Y., Yin, J. (2011). Nonlinear Discriminative Embedding for Clustering via Spectral Regularization. In: Huang, J.Z., Cao, L., Srivastava, J. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2011. Lecture Notes in Computer Science(), vol 6634. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-20841-6_20
Download citation
DOI: https://doi.org/10.1007/978-3-642-20841-6_20
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-20840-9
Online ISBN: 978-3-642-20841-6
eBook Packages: Computer ScienceComputer Science (R0)