A Tabu search approach to the clustering problem

doi:10.1016/0031-3203(95)00022-R

Pattern Recognition

Volume 28, Issue 9, September 1995, Pages 1443-1451

https://doi.org/10.1016/0031-3203(95)00022-R Get rights and content

Abstract

In this paper we consider the problem of clustering m objects into c clusters. The objects are represented by points in an n-dimensional Euclidean space, and the objective is to classify these m points into c clusters such that the distance between points within a cluster and its center (which is to be found) is minimized. The problem is a nonconvex program that has many local minima. It has been studied by many researchers and the most well-known algorithm for solving it is the k-means algorithm. In this paper, we develop a new algorithm for solving this problem based on a tabu search technique. Preliminary computational experience on the developed algorithm are encouraging and compare favorably with both the k-means and the simulated annealing algorithms.

References (16)

R.W. Klein et al.
Experiments in projection and clustering by simulated annealing
Pattern Recognition
(1989)
S.Z. Selim et al.
A simulated annealing algoroithm for the clustering problem
Pattern Recognition
(1991)
K.S. Al-Sultan
Efficient global algorithms for Hand and Fuzzy Clustering
E. Forgy
Cluster analysis of multivariate data: efficiency versus interpretability of classifications
Biometrics
(1965)
J.B. McQueen, Some methods of classification and analysis of multivariate observations, Proc. 5th Berkeley Symp....
S.Z. Selim et al.
K-Means-type algorithm: generalized convergence theorem and characterization of local optimality
IEEE Trans. Pattern Anal. Mach. Intell.
(1984)
R.E. Jensen
A dynamic programming algorithm for cluster analysis
Appl. Statist.
(1969)
H.D. Vindo
Integer Programming and the theory of grouping
J. Am. Statist. Soc.
(1969)

There are more references available in the full text version of this article.

Cited by (243)

A novel optimization approach towards improving separability of clusters
2023, Computers and Operations Research
The objective functions in optimization models of the sum-of-squares clustering problem reflect intra-cluster similarity and inter-cluster dissimilarities and in general, optimal values of these functions can be considered as appropriate measures for compactness of clusters. However, the use of the objective function alone may not lead to the finding of separable clusters. To address this shortcoming in existing models for clustering, we develop a new optimization model where the objective function is represented as a sum of two terms reflecting the compactness and separability of clusters. Based on this model we develop a two-phase incremental clustering algorithm. In the first phase, the clustering function is minimized to find compact clusters and in the second phase, a new model is applied to improve the separability of clusters. The Davies–Bouldin cluster validity index is applied as an additional measure to compare the compactness of clusters and silhouette coefficients are used to estimate the separability of clusters. The performance of the proposed algorithm is demonstrated and compared with that of four other algorithms using synthetic and real-world data sets. Numerical results clearly show that in comparison with other algorithms the new algorithm is able to find clusters with better separability and similar compactness.
Collaborative annealing power k-means++ clustering
2022, Knowledge-Based Systems
Clustering is the most fundamental technique for data processing. This paper presents a collaborative annealing power $k$ -means++ clustering algorithm by integrating the $k$ -means++ and power $k$ -means algorithms in a collaborative neurodynamic optimization framework. The proposed algorithm starts with $k$ -means++ to select initial cluster centers, then leverages the power $k$ -means to find multiple sets of centers as alternatives and a particle swarm optimization rule to reinitialize the centers in the subsequential iterations for improving clustering performance. Experimental results on twelve benchmark datasets are elaborated to demonstrate the superior performance of the proposed algorithm to seven mainstream clustering algorithms in terms of 21 internal and external indices.
Memetic differential evolution methods for clustering problems
2021, Pattern Recognition
The Euclidean Minimum Sum-of-Squares Clustering(MSSC) is one of the most important models for the clustering problem. Due to its NP-hardness, the problem continues to receive much attention in the scientific literature and several heuristic procedures have been proposed. Recent research has been devoted to the improvement of the classical K-MEANS algorithm, either by suitably selecting its starting configuration or by using it as a local search method within a global optimization algorithm. This paper follows this last approach by proposing a new implementation of a Memetic Differential Evolution (MDE) algorithm specifically designed for the MSSC problem and based on the repeated execution of K-MEANS from selected configurations. In this paper we describe how to adapt MDE to the clustering problem and we show, through a vast set of numerical experiments, that the proposed method has very good quality, measured in terms of the minimization of the objective function, as well as a very good efficiency, measured in the number of calls to the local optimization routine, with respect to state of the art methods.
A simulated annealing algorithm with a dual perturbation method for clustering
2021, Pattern Recognition
Clustering is a powerful tool in exploratory data analysis that partitions a set of objects into clusters with the goal of maximizing the similarity of objects within each cluster. Due to the tendency of clustering algorithms to find suboptimal partitions of data, the approximation method Simulated Annealing (SA) has been used to search for near-optimal partitions. However, existing SA-based partitional clustering algorithms still settle to local optima. We propose a new SA-based clustering algorithm, the Simulated Annealing with Gaussian Mutation and Distortion Equalization algorithm (SAGMDE), which uses two perturbation methods to allow for both large and small perturbations in solutions. Our experiments on a diverse collection of data sets show that SAGMDE performs more consistently and yields better results than existing SA clustering algorithms in terms of cluster quality while maintaining a reasonable runtime. Finally, we use generative art as a visualization tool to compare various partitional clustering algorithms.
A novel data clustering algorithm based on gravity center methodology
2020, Expert Systems with Applications
The concept of clustering is to separate clusters based on the similarity which is greater within cluster than among clusters. The similarity consists of two principles, namely, connectivity and cohesion. However, in partitional clustering, while some algorithms such as K-means and K-medians divides the dataset points according to the first principle (connectivity) based on centroid clusters without any regard to the second principle (cohesion), some others like K-medoids partially consider cohesion in addition to connectivity. This prevents to discover clusters with convex shape and results are affected negatively by outliers. In this paper a new Gravity Center Clustering (GCC) algorithm is proposed which depends on critical distance (λ) to define threshold among clusters. The algorithm falls under partition clustering and is based on gravity center which is a point within cluster that verifies both the connectivity and cohesion in determining the similarity of each point in the dataset. Therefore, the proposed algorithm deals with any shape of data better than K-means, K-medians and K-medoids. Furthermore, GCC algorithm does not need any parameters beforehand to perform clustering but can help user improving the control over clustering results and deal with overlapping and outliers providing two coefficients and an indicator. In this study, 22 experiments are conducted using different types of synthetic, and real healthcare datasets. The results show that the proposed algorithm satisfies the concept of clustering and provides great flexibility to get the optimal solution especially since clustering is considered as an optimization problem.
2.25 - Common Clustering Algorithms
2020, Comprehensive Chemometrics: Chemical and Biochemical Data Analysis, Second Edition: Four Volume Set
The major steps of an overall clustering task are preclustering, clustering, and postclustering. Preclustering involves data preparation, including feature extraction, selection, transformation normalization, cleaning, and data reduction, whereas postclustering involves cluster usability encompassing cluster validity, reasoning, interpretation, and visualization. This article focuses on the second step, “clustering,” which is further divided into two key modules: clustering criterion and clustering method. This clustering step takes a set X = {x₁, x₂, …, x_n} of preprocessed points (synonymously elements, objects, instances, cases or patterns) as an input and produces a clustered result as an output (either partitioning or hierarchical) for postclustering. It first requires a clustering criterion to be built and needs a clustering algorithm to optimize the clustering criterion.

View all citing articles on Scopus

View full text

A Tabu search approach to the clustering problem

Abstract

Pattern Recognition

Pattern Recognition

Efficient global algorithms for Hand and Fuzzy Clustering

Cluster analysis of multivariate data: efficiency versus interpretability of classifications

Biometrics

K-Means-type algorithm: generalized convergence theorem and characterization of local optimality

IEEE Trans. Pattern Anal. Mach. Intell.

A dynamic programming algorithm for cluster analysis

Appl. Statist.

Integer Programming and the theory of grouping

J. Am. Statist. Soc.