Design of hybrids for the minimum sum-of-squares clustering problem

Abstract

A series of metaheuristic algorithms is proposed and analyzed for the non-hierarchical clustering problem under the criterion of minimum sum-of-squares clustering. These algorithms incorporate genetic operators and local search and tabu search procedures. The aim is to obtain quality solutions with short computation times. A series of computational experiments has been performed. The proposed algorithms obtain better results than previously reported methods, especially with a small number of clusters.

Introduction

Consider a set X={x₁,x₂,…,x_N} of N points in R^q and let m be a predetermined positive integer. The minimum sum-of-squares clustering (MSSC) problem is to find a partition of X into m disjoint subsets (clusters) so that the sum of squared distances from each point to the centroid of its cluster is minimum. Specifically, let P_m denote the set of all the partitions of X in m sets, where each partition P∈P_m is defined as P=(C₁,C₂,…,C_m) and where C_i denotes each of the clusters that forms P. Thus, the problem can be expressed as

$$\text{min}\text{P∈P}{}_{\mathrm{m}}\text{∑}\text{i=1}\text{m}\text{∑}\text{x}{}_{\mathrm{l}}\text{∈C}{}_{\mathrm{i}}\text{||x}{}_{\mathrm{l}}\text{−}\text{x}\text{̄}{}_{\mathrm{i}}\text{||}{}^2\text{,}$$

where the centroid $\text{x}\text{̄}{}_{\mathrm{i}}$ is defined as

$$\text{x}\text{̄}{}_{\mathrm{i}}\text{=}\text{1}\text{n}{}_{\mathrm{i}}\text{∑}\text{x}{}_{\mathrm{l}}\text{∈C}{}_{\mathrm{i}}\text{x}{}_{\mathrm{l}}\text{with}\text{n}{}_{\mathrm{i}}\text{=|C}{}_{\mathrm{i}}\text{|.}$$

Equivalently the problem can be written as

$$\text{min}\text{∑}\text{l=1}\text{N}\text{||x}{}_{\mathrm{l}}\text{−}\text{x}\text{̄}{}_{\mathrm{c(l)}}\text{||}{}^2\text{,}$$

where c(l) is the cluster to which point x_l belongs.

The design of clusters is a well-known exploratory data analysis issue called pattern recognition. The aim is to find whether a given set of cases X has some structure and, in if so, to display it in the form of a partition. This problem belongs to the area of non-hierarchical cluster design, which has many applications in Economics, Social and Natural Sciences. It is known to be NP-hard (Brucker, 1978).

Various exact methods for MSSC can be found in the literature (see, for example, Koontz et al., 1975; Diehr, 1985), some of which, such as the method proposed by du Merle et al. (2000), have succeeded in resolving problems with up to 150 points. For larger-sized problems the use of heuristic algorithms is still necessary. The most popular are those based on local search methods, such as the well-known K-means (Jancey, 1966) and H-means (Howard, 1966) procedures. In a recent work, Hansen and Mladenovic (2001) propose a new local search procedure, J-means, along with variants H-means+ or HK-means. In recent years algorithms using metaheuristic strategies have been designed, such as simulated annealing (Klein and Dubes, 1989), tabu search (TS) (Al-Sultan, 1995), genetic algorithms (Babu and Murty, 1993) or most recently variable neighborhood search or VNS (du Merle et al., 2000; Hansen and Mladenovic, 2001).

A series of algorithms that is able to obtain good solutions in short times is proposed for this problem. Initially, a genetic algorithm is designed using local search methods, thus becoming a memetic algorithm. A simple procedure based on a TS method using binary trees is also suggested. This method demonstrates its capacity to improve solutions in very few iterations. The incorporation of this procedure into memetic algorithms yields hybrid algorithms. Finally, these memetic and hybrid algorithms are analyzed and compared with other techniques. In all cases, the proposed techniques give adequate solutions, compared with other techniques, in reasonable time, especially with small values of m.

This paper is structured as follows: in the next section some of the already existing local search algorithms are described. In Section 3, the TS method is presented. Section 4 considers the genetic algorithm in detail. In Section 5 Memetic and hybrid algorithms are described. Section 6 presents the results of different computational experiments and compares the effectiveness and efficiency of the proposed algorithms with other recent techniques. Finally, Section 7 summarizes the contribution.