Abstract
Spectral methods are strong tools that can be used for extraction of the data’s structure based on eigenvectors of constructed affinity matrices. In this paper, we aim to propose some new measurement functions to evaluate the ability of each eigenvector of affinity matrix in data clustering. In the proposed strategy, each eigenvector’s elements are clustered by traditional fuzzy c-means algorithm and then informative eigenvectors selection is performed by optimization of an objective function which defined based on three criterions. These criterions are the compactness of clusters, distance between clusters and stability of clustering to evaluate each eigenvector based on considering the structure of clusters which placed on. Finally, Lagrange multipliers method is used to minimize the proposed objective function and extract the most informative eigenvectors. To indicate the merits of our algorithm, we consider UCI Machine Learning Repository databases, COIL20, YALE-B and PicasaWeb as benchmark data sets. Our simulation’s results confirm the superior performance of the proposed strategy in developing spectral clustering compared to conventional clustering methods and recent eigenvector selection based algorithms.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bach, F. R., & Jordan, M. I. (2006). Learning spectral clustering, with application to speech separation. J. Mach. Learn., 7, 1963–2001.
Bezdek, J. C. (1976). A physical interpretation of Fuzzy ISODATA. IEEE Trans. SMC, 6, 387–390.
Costa, J., & Hero, A. (2004). Geodesic entropic graphs for dimension and entropy estimation in manifold learning. IEEE Trans. Signal Process., 52, 2210–2221.
Driessche, R. V., & Roose, D. (1995). An improved spectral bisection algorithm and its application to dynamic load balancing. Parallel Comput., 21, 29–48.
Fiedler, M. (1973). Algebraic connectivity of graphs. Czechoslov. Math. J., 23, 298–305.
Fukunaga, K. (1990). Introduction to statistical pattern recognition. San Diego: Academic Press Professional Inc.
Georghiades, A., Belhumeur, P., & Kriegman, D. (2001). From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Trans. Pattern Anal. Mach. Intell., 23(6), 643–660.
He, X., & Niyogi, P. (2003). Locality preserving projections. In Proceedings of Advances in Neural Information Processing Systems, 16 (p. 153).
Hendrickson, B., & Leland, R. (1995). An improved spectral graph partitioning algorithm for mapping parallel computations. SIAM J. Sci. Comput., 16, 452–469.
Higham, D. J. (2007). Spectral clustering and its use in bioinformatics. J. Comput. Appl. Math., 204, 25–37.
Jimenez, L. O., & Landgrebe, D. A. (1997). Supervised classification in high-dimensional space: Geometrical, statistical, and asymptotical properties of multivariate data. IEEE Trans. Syst. Man Cybern., 28(1), 39–54.
Jolliffe, I. T. (2002). Principal component analysis. New York: Springer.
Lee, J. A., & Verleysen, M. (2007). Nonlinear dimensionality reduction. New York, NY: Springer.
Munkres, J. (1957). Algorithms for the assignment and transportation problems. J. Soc. Ind. Appl. Math., 5(1), 32–38.
Nene, S. A., Nayar, S. K. & Murase, H. (1996). Columbia object image library (COIL-20). Department of Computer Science, Columbia University, New York, Technical Report CUCS-005-96.
Ng, A. Y., Jordan, I., & Weiss, Y. (2002). On spectral clustering: Analysis and an algorithm. In Proceedings of Advances in Neural Information Processing Systems (pp. 849–856).
Nie, F., Zeng, Z., Tsang, I. W., & Xu, D. (2011). Spectral embedded clustering: A framework for in-sample and out-of-sample spectral clustering. IEEE Trans. Neural Netw., 22(11), 1796–1809.
Rebagliati, N., & Verri, A. (2011). Spectral clustering with more than K eigenvectors. Neurocomputing, 74, 1391–1401.
Saul, L. K., Weinberger, K. Q., Ham, J. H., Sha, F., & Lee. D. D. (2006). Spectral methods for dimensionality reduction. In O. Chapelle, B. Schoelkopf, & A. Zien (Eds.), Semisupervised learning. Cambridge, MA: The MIT Press.
Schölkopf, B. (1996). Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput., 10, 1299–1319.
Scott, G. L., & Longuet-Higgins, H. C. (1990). Feature grouping by relocalisation of eigenvectors of the proxmity matrix. In Presented at the British Machine Vision Conference.
Shi, J., & Malik, J. (2000). Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell., 22, 888–905.
van der Maaten, L. J. P., & Hinton, G. E. (2008). Visualizing data using t-SNE. J. Mach. Learn. Res., 9, 2431–2456.
Weiss, Y. (1999). Segmentation using eigenvectors: A unifying view. In International Conference on Computer Vision.
Windham, M. P. (1982). Cluster validity for the fuzzy c-means clustering algorithm. IEEE Trans. PAMI, 4, 357–363.
Xiang, T., & Gong, S. (2008). Spectral clustering with eigenvector selection. Pattern Recognit., 41, 1012–1029.
Ye, J., Zhao, Z., & Wu, M. (2007). Discriminative K-means for clustering. In Proceedings of Neural Information Processing Systems, Vancouver, BC, Canada, (pp. 1649–1656).
Zhang, Z., & Jordan, M. I. (2008). Multiway spectral clustering: A margin-based perspective. Stat. Sci., 23, 383–403.
Zhao, F. (2010). Spectral clustering with eigenvector selection based on entropy ranking. Neurocomputing, 73, 1704–1717.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Hosseini, M., Azar, F.T. A new eigenvector selection strategy applied to develop spectral clustering. Multidim Syst Sign Process 28, 1227–1248 (2017). https://doi.org/10.1007/s11045-016-0391-6
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11045-016-0391-6