Abstract
Spectral clustering is a clustering method based on algebraic graph theory. It has aroused extensive attention of academia in recent years, due to its solid theoretical foundation, as well as the good performance of clustering. This paper introduces the basic concepts of graph theory and reviews main matrix representations of the graph, then compares the objective functions of typical graph cut methods and explores the nature of spectral clustering algorithm. We also summarize the latest research achievements of spectral clustering and discuss several key issues in spectral clustering, such as how to construct similarity matrix and Laplacian matrix, how to select eigenvectors, how to determine cluster number, and the applications of spectral clustering. At last, we propose several valuable research directions in light of the deficiencies of spectral clustering algorithms.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Adefioye AA, Liu XH, Moor BD (2013) Multi-view spectral clustering and its chemical application. Int J Comput Biol Drug Des 6(1–2):32–49
Alpert CJ, Kahng AB (1995) Multi-way partitioning via geometric embeddings, orderings and dynamic programming. IEEE Trans Comput-Aaid Des Integr Circuits Syst 14(11):1342–1358
Alpert CJ, Yao SZ (1995) Spectral partitioning: the more eigenvectors, the better. In: Proceedings of the 32nd annual ACM/IEEE design automation conference. ACM, New York, pp 195–200
Alzate C, Suykens JAK (2012) Hierarchical kernel spectral clustering. Neural Netw 35:21–30
Bach FR, Jordan MI (2006) Learning spectral clustering, with application to speech separation. J Mach Learn Res 7:1963–2001
Bames ER (1982) An algorithm for partitioning the nodes of a graph. SIAM J Algebraic Discrete Methods 17(5):541–550
Blekas K, Lagaris IE (2013) A spectral clustering approach based on Newton’s equations of motion. Int J Intell Syst 28(4):394–410
Cai XY, Dai GZ, Yang LB (2008) Survey on spectral clustering algorithms. Comput Sci 35(7):14–18
Chasanis VT, Likas AC, Galatsanos NP (2009) Scene detection in videos using shot clustering and sequence alignment. IEEE Trans Multimed 11(1):89–100
Chen WF, Feng GC (2012) Spectral clustering with discriminant cuts. Knowl-Based Syst 28:27–37
Chen WF, Feng GC (2012) Spectral clustering: a semi-supervised approach. Neurocomputing 77(1):229–242
Chen WY, Song YQ, Bai HJ et al (2011) Parallel spectral clustering in distributed systems. IEEE Trans Patt Anal Mach Intell 33(3):568–586
Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc Ser B Stat Methodol 39(1):1–38
Ding CHQ, He X, Zha H et al (2001) A min-max cut algorithm for graph partitioning and data clustering. In: Proceedings of IEEE international conference on data mining (ICDM’ 2001), pp 107–114
Ding L, Gonzalez-Longatt FM, Wall P, Terzija V (2013) Two-step spectral clustering controlled islanding algorithm. IEEE Trans Power Syst 28(1):75–84
Ding SF, Jia HJ, Zhang LW et al (2012) Research of semi-supervised spectral clustering algorithm based on pairwise constraints. Neural Comput Appl. doi:10.1007/s00521-012-1207-8
Ding SF, Qi BJ, Jia HJ et al (2013) Research of semi-supervised spectral clustering based on constraints expansion. Neural Comput Appl 22(Suppl 1):S405–S410
Donath WE, Hoffman AJ (1973) Lower bounds for the partitioning of graph. IBM J Res Dev 17(5):420–425
Dong XW, Frossard P, Vandergheynst P, Nefedov N (2012) Clustering with multi-layer graphs: a spectral perspective. IEEE Trans Sig Process 60(11):5820–5831
Driessche RV, Roose D (1995) An improved spectral bisection algorithm and its application to dynamic load balancing. Parallel Comput 21(1):29–48
Dunn JC (1974) Well-separated clusters and the optimal fuzzy partitions. J Cybern 4(1):95–104
Fang YX, Wang JH (2012) Selection of the number of clusters via the bootstrap method. Comput Stat Data Anal 56(3):468–477
Fiedler M (1973) Algebraic connectivity of graphs. Czechoslov Math J 23(2):298–305
Frederix K, Van Barel M (2013) Sparse spectral clustering method based on the incomplete Cholesky decomposition. J Comput Appl Math 237(1):145–161
Hagen L, Kahng AB (1992) New spectral methods for radio cut partitioning and clustering. IEEE Trans Comput-aid Des Integr Circuits Syst 11(9):1074–1085
Hamad D, Biela P (2008) Introduction to spectral clustering. In: 3rd International conference on information and communication technologies: from theory to applications, 1–5, pp 490–495
Hendrickson B, Leland R (1995) An improved spectral graph partitioning algorithm for mapping parallel computations. SIAM J Sci Comput 16(2):452–459
Higham DJ, Kibble M (2004) A unified view of spectral clustering. In: University of Strathclyde Mathematics Research Report 02
Huang Z (1997) A fast clustering algorithm to cluster very large categorical data sets in data mining. In: Proceedings of the SIGMOD workshop on research issues on data mining and knowledge discovery. Tucson, pp 146–151
Huang Z (1998) Extensions to the k-means algorithm for clustering large data sets with categorical values. Data Min Knowl Discov 2(3):283–304
Jia JH, Xiao X, Liu BX, Jiao LC (2011) Bagging-based spectral clustering ensemble selection. Patt Recogn Lett 32(10):1456–1467
Jiao LC, Shang FH, Wang F, Liu YY (2012) Fast semi-supervised clustering with enhanced spectral embedding. Patt Recogn 45(12):4358–4369
Kluger Y, Basri R, Chang JT et al (2003) Spectral biclustering of microarray data: coclustering genes and conditions. Genome Res 13(4):703–716
Leicht EA, Newman MEJ (2008) Community structure in directed networks. Phys Rev Lett 100(11):118703
Li JY, Zhou JG, Guan JH et al (2011) A survey of clustering algorithms based on spectra of graphs. CAAI Trans Intell Syst 6(5):405–414
Li XY, Guo LJ (2012) Constructing affinity matrix in spectral clustering based on neighbor propagation. Neurocomputing 97:125–130
Liu HQ, Jiao LC, Zhao F (2010) Non-local spatial spectral clustering for image segmentation. Neurocomputing 74(1–3):461–471
Liu HQ, Zhao F, Jiao LC (2012) Fuzzy spectral clustering with robust spatial information for image segmentation. Appl Soft Comput 12(11):3636–3647
Luo DJ, Huang H, Ding C, Nie FP (2010) On the eigenvectors of p-Laplacian. Mach Learn 81(1):37–51
Luxburg U, Belkin M, Bousquet O (2008) Consistency of spectral clustering. Ann Stat 36(2):555–586
MacQueen J (1967) Some methods for classification and analysis of multivariate observations. In: Proceedings of 5th Berkeley symposium on mathematical statistics, 1, pp 281–297
Malik J, Belongie S, Leung T et al (2001) Contour and texture analysis for image segmentation. Int J Comput Vis 43(1):7–27
Meila M, Shi JB (2001) Learning segmentation by random walks. Advances in neural information processing systems. MIT Press, Cambridge, pp 873–879
Michoel T, Nachtergaele B (2012) Alignment and integration of complex networks by hypergraph-based spectral clustering. Phys Rev E 86(5):056111
Mirkin B, Nascimento S (2012) Additive spectral method for fuzzy cluster analysis of similarity data including community structure and affinity matrices. Inf Sci 183(1):16–34
Mohar B (1997) Some applications of Laplace eigenvalues of graphs. Graph Symmetry Algebraic Methods Appl 497(22):227–275
Nascimento MCV, de Carvalho ACPLF (2011) Spectral methods for graph clustering: a survey. Eur J Oper Res 211(2):221–231
Newman MEJ (2004) Analysis of weighted networks. Phys Rev E 70(5):056131
Newman MEJ (2006) Finding community structure in networks using the eigenvectors of matrices. Phys Rev E 74(3):036104
Newman MEJ (2006) Modularity and community structure in networks. Proc Nat Acad Sci US 103(23):8577–8582
Ng AY, Jordan MI, Weiss Y (2002) On spectral clustering: analysis and an algorithm. Adv Neural Inf Process Syst 14:849–856
Paccanaro A, Chennubhotla C, Casbon JA (2006) Spectral clustering of protein sequences. Nucl Acids Res 34(5):1571–1580
Rebagliati N, Verri A (2011) Spectral clustering with more than K eigenvectors. Neurocomputing 74(9):1391–1401
Sarkar S, Soundararajan P (2000) Supervised learning of large perceptual organization: graph spectral partitioning and learning automata. IEEE Trans Patt Anal Mach Intell 22(5):504–525
Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Patt Anal Mach Intell 22(8):888–905
Sun JG, Liu J, Zhao LY (2008) Clustering algorithms research. J Softw 19(1):48–61
Tepper M, Muse P, Almansa A, Mejail M (2011) Automatically finding clusters in normalized cuts. Patt Recogn 44(7):1372–1386
Tung F, Wong A, Clausi DA (2010) Enabling scalable spectral clustering for image segmentation. Patt Recogn 43(12):4069–4076
Urquhart R (1982) Graph theoretical clustering based on limited neighborhood sets. Pattern Recogn 15(3):173–187
von Luxburg U (2007) A tutorial on spectral clustering. Stat Comput 17(4):395–416
Wang JH (2010) Consistent selection of the number of clusters via cross validation. Biometrika 97(4):893–904
Wang L, Bo LF, Jiao LC (2007) Density-sensitive spectral clustering. Acta Electronica Sinica 35(8):1577–1581
Wang LJ, Dong M (2012) Multi-level low-rank approximation-based spectral clustering for image segmentation. Patt Recogn Lett 33(16):2206–2215
Wang Y, Jiang Y, Wu Y, Zhou ZH (2011) Spectral clustering on multiple manifolds. IEEE Trans Neural Netw 22(7):1149–1161
Wei YC, Cheng CK (1989) Toward efficient hierarchical designs by ratio cut partitioning. In: IEEE international conference on CAD. New York, pp 298–301
Wu Z, Leahy R (1993) An optimal graph theoretic approach to data clustering: theory and its application to image segmentation. IEEE Trans Patt Anal Mach Intell 15(11):1101–1113
Xiang T, Gong S (2008) Spectral clustering with eigenvector selection. Patt Recogn 41(3):1012–1029
Xie B, Wang M, Tao DC (2011) Toward the optimization of normalized graph Laplacian. IEEE Trans Neural Netw 22(4):660–666
Xie YK, Zhou YQ, Huang XJ (2009) A spectral clustering based conference resolution method. J Chin Inf Process 23(3):10–16
Yang P, Zhu QS, Huang B (2011) Spectral clustering with density sensitive similarity function. Knowl-Based Syst 24(5):621–628
Yang Y, Xu D, Nie FP, Yan SC, Zhuang YT (2010) Image clustering using local discriminant models and global integration. IEEE Trans Image Process 19(10):2761–2773
Zahn CT (1971) Graph-theoretic methods for detecting and describing gestalt clusters. IEEE Trans Comput 20(1):68–86
Zeng S, Sang N, Tong XJ (2011) Hand-written numeral recognition based on spectrum clustering. In: MIPPR 2011: pattern recognition and computer vision, Proceedings of SPIE, p 8004
Zhang XC, Li JW, Yu H (2011) Local density adaptive similarity measurement for spectral clustering. Patt Recogn Lett 32(2):352–358
Zhang XC, You QZ (2011) An improved spectral clustering algorithm based on random walk. Frontiers Comput Sci China 5(3):268–278
Zhang XR, Jiao LC, Liu F (2008) Spectral clustering ensemble applied to SAR image segmentation. IEEE Trans Geosci Rem Sens 46(7):2126–2136
Zhao F, Jiao LC, Liu HQ et al (2010) Spectral clustering with eigenvector selection based on entropy ranking. Neurocomputing 73(10–12):1704–1717
Acknowledgments
This work is supported by the National Key Basic Research Program of China (No.2013CB329502), and the Fundamental Research Funds for the Central Universities (No.2013XK10).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Jia, H., Ding, S., Xu, X. et al. The latest research progress on spectral clustering. Neural Comput & Applic 24, 1477–1486 (2014). https://doi.org/10.1007/s00521-013-1439-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-013-1439-2