Abstract
K-Means users usually have to decide on the number of clusters and the initial state by themselves. Evolutionary K-Means (EKM), a hybrid algorithm of K-Means and genetic algorithm, solves the problem by choosing the two parameters automatically through partition evolution; however, the final partition obtained often doesn’t meet users’ expectations. As a solution to this problem, we suggest using background knowledge for enhancing clustering quality and propose a semi-supervised approach that incorporates instance level constraints into the objective function of EKM. Firstly, we define Constrained Silhouette Index (CS) for data instances, which decreases the silhouette index of the instance having violated constraints. Then, we present two weighted approaches to extend the influence of constraints beyond the level of instance for evaluating the quality of a cluster or a partition. To evaluate the performance of CS in guiding EKM algorithms, we combine the two types of CS with F-EAC algorithm, and get two constrained EKM algorithms, which are named as CEAC1 and CEAC2, respectively. Experimental results on two artificial datasets and eight UCI datasets suggest a few constraints are often powerful enough to improve the accuracy of labelling instances and choosing K, and more constraints may improve the performance even more.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Alves V, Campello RJGB, Hruschka ER (2006) Towards a fast evolutionary algorithm for clustering. In: Proceedings of IEEE Congress on Evolutionary Computation, pp 1776–1783
Arbelaitz O, Gurrutxaga I, Muguerza J, Perez JM, Perona I (2013) An extensive comparative study of cluster validity indices. Pattern Recognit. 46:243–256
Arthur D, Vassilvitskii S (2007) K-means++: the advantages of careful seeding. In: Proceedings of 18th annual ACM-SIAM symposium on discrete algorithms (SODA), pp 1027–1035
Bache K, Lichman M (2013) UCI machine learning repository. University of California, School of Information and Computer Science, Irvine. http://archive.ics.uci.edu/ml
Basu S, Banerjee A, Mooney R (2003) Comparing and unifying search-based and similarity-based approaches to semi-supervised clustering. In: Proceedings of the 20th international conference on machine learning (ICML 2003) workshop on the continuum from labeled to unlabeled data in machine learning and data mining, pp 42–49
Bellet A, Habrard A, Sebban M (2013) Survey on metric learning for feature vectors and structured data, technical report. arXiv:1306.6709
Brunsch T, Roglin H (2013) A bad instance for K-Means++. Theor Comput Sci 505:19–26
Cano JR, Cordon O, Herrera F, Sanchez F (2002) A greedy randomized adaptive search procedure applied to the clustering problem as an initialization process using K-Means as a local search procedure. J Intell Fuzzy Syst 12:235–242
Chen S, Chao Y, Wang H, Fu H (2006) A prototypes-embedded genetic K-Means algorithm. In: Proceedings of 18th international conference on pattern recognition (ICPR 06), pp 724–727
Demiriz A, Bennett K, Embrechts M (1999) Semi-supervised clustering using genetic algorithms. Intell Eng Syst Artif Neural Netw 9:809–814
Duarte JMM, Fred ALN, Duarte JF (2013) Data clustering validation using constraints. In: Proceedings of international conference on knowledge discovery and information retrieval and the international conference on knowledge management and information sharing (KDIR/KMIS), pp 17–27
Dutta H, Passonneau RJ, Lee A, Radeva A, Xie B, Waltz D, Taranto B (2011) Learning parameters of the K-Means algorithm from subjective human annotation. In: Proceedings of 24th international Florida artificial intelligence research society conference (FLAIRS 11)
He Z (2013) Constrained silhouette based evolutionary K-Means. In: Proceedings of Chinese intelligent automation conference, lecture notes in electrical engineering, vol 256. pp 615–622
Hong Y, Kwong S, Wang H, Ren Q, Chang Y (2008) Probabilistic and graphical model based genetic algorithm driven clustering with instance-level constraints. In: Proceedings of IEEE congress on evolutionary computation (CEC 08), pp 322–329
Hong Y, Kwong S, Xiong H, Ren Q (2008) Genetic-guided semi-supervised clustering algorithm with instance-level constraints. In: Proceedings of genetic and evolutionary computation conference (GECCO 08), pp 1381–1388
Hong Y, Kwong S (2009) Learning assignment order of instances for constrained K-means clustering algorithm. IEEE Trans Syst Man Cybern Part B Cybern 39(2):568–574
Hruschka ER, Campello RJGB, Freitas AA, de Carvalho ACPLF (2009) A survey of evolutionary algorithms for clustering. IEEE Trans Syst Man Cybern Part C Appl Rev 39(2):133–155
Hruschka ER, Campello RJGB, de Castro LN (2006) Evolving clusters in gene-expression data. Inf Sci 176:1898–1927
Hwang C, Chang T (2012) Genetic K-Means collaborative filtering for multi-criteria recommendation. J Comput Inf Syst 8(1):293–303
Jain AK (2010) Data clustering: 50 years beyond K-means. Pattern Recognit Lett 31(8):651–666
Krishna K, Murty MN (1999) Genetic K-Means algorithm. IEEE Trans Syst Man Cybern Part B Cybern 29(3):433–439
Kryszczuk K, Hurley P (2010) Estimation of the number of clusters using multiple clustering validity indices. In: Proceedings of 9th international workshop on multiple classifier systems (MCS 2010). Lecture notes in computer science, vol 5997. pp 114–123
Liu H, Huang S (2003) Evolutionary semi-supervised fuzzy clustering. Pattern Recognit Lett 24:3105–3113
Naldi MC, Campello RJGB, Hruschka ER, Carvalho ACPLF (2011) Efficiency issues of evolutionary K-Means. Appl Soft Comput 11:1938–1952
Naldi MC, Carvalho ACPLF, Campello RJGB (2013) Cluster ensemble selection based on relative validity indexes. Data Min Knowl Disc 27(2):259–289
Rousseeuw PJ (1987) Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. J Comput Appl Math 20:53–65
Shamir O, Tishby N (2010) Stability and model selection in K-Means clustering. Mach Learn 80(2–3):213–243
Tseng M, Chiang C, Tang P, Wu H (2010) A study on cluster validity using intelligent evolutionary K-Means approach. In: Proceedings of 9th international conference on machine learning and cybernetics (ICMLC 10), pp 2510–2515
Vendramin L, Campello RJGB, Hruschka ER (2010) Relative clustering validity criteria: a comparative overview. Stat Anal Data Min 3(4):243–256
Wagstaff K, Cardie C, Rogers S, Schroedl S (2001) Constrained K-Means clustering with background knowledge. In: Proceedings of 18th international conference on machine learning (ICML 01), pp 577–584
Wagstaff K (2007) Value, cost, and sharing: open issues in constrained clustering. In: Proceedings of 5th international workshop on knowledge discovery in inductive databases. Lecture notes in computer science, vol 4747. pp 1–10
Xu R, Wunsch D (2005) Survey of clustering algorithms. IEEE Trans Neural Netw 16(3):645–678
Yang L, Jin R (2006) Distance metric learning: a comprehensive survey. Technical report, Department of Computer Science and Engineering, Michigan State University
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by V. Loia.
Rights and permissions
About this article
Cite this article
He, Z. Evolutionary K-Means with pair-wise constraints. Soft Comput 20, 287–301 (2016). https://doi.org/10.1007/s00500-014-1503-6
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-014-1503-6