A swarm-inspired projection algorithm

Abstract

In this paper a new data projection algorithm which was inspired by the foraging behaviors of doves is proposed. We name the new data projection the swarm-inspired projection (SIP) algorithm. The algorithm allows us to visually estimate the number of clusters existing in a data set. Based on the projection result, we may then partition the data set into the corresponding number of clusters. The SIP algorithm regards each data pattern in a data set as a piece of crumb which will be sequentially tossed to a flock of doves on the ground. The doves will adjust their physical positions to compete for crumbs. Gradually, the flock of doves will be divided into several groups according to the distributions of the crumbs. The formed groups will naturally correspond to the underlying data structures in the data set. By viewing the scatter plot of the final positions of the doves we can estimate the number of clusters existing in the data set. Several data sets were used to demonstrate the effectiveness of the proposed SIP algorithm.

Introduction

Recently, many properties of social insect (or animals) collective behaviors have attracted a great amount of attention from researchers [1], [2]. Social insects (or animals) provide us with a powerful concept to create decentralized systems of simple interacting, and often mobile, agents (e.g., ants, bees, birds). A rich source of mechanisms in social insect collective behaviors may serve as metaphors for designing the so-called swarm-intelligence-based systems. Swarm intelligence, a form of artificial intelligence, is the emergent collective intelligence of groups of simple agents. Different swarm-intelligence-based systems are inspired by different subsets of the available metaphors. Ant colony systems (ACS) [3], [4] and particle swarm optimization (PSO) [5], [6] are two well-known kinds of swarm intelligence. The applications of these systems are widely spread from optimization, communications networks, to robots [7], [8], [9], [10], [11].

Cluster analysis is one of the basic tools for exploring the underlying structure of a given data set and is being applied in a wide variety of engineering and scientific disciplines such as medicine, psychology, biology, society, pattern recognition, and image processing. The primary objective of cluster analysis is to partition a given data set into so-called homogeneous clusters such that patterns within a cluster are more similar to each other than patterns belonging to different clusters. There are two major difficulties encountered in clustering data: (1) cluster geometric shapes are full of variability and (2) the number of clusters is not always known a priori. Different distance measures lead to different types of clusters (e.g., compact hyper-spheres, compact hyper-ellipsoids, lines, shells, etc.). Recently, several clustering algorithms with different distance measures have been developed for clustering data sets with different geometric shapes [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22]. These algorithms are used to detect compact clusters [12], [13], [14], [15], straight lines [14], [15], [16], shells [17], [18], [19], contours with polygonal boundaries [19], [20], or clusters of different geometrical structures [21], [22].

In fact, if cluster analysis is to make a significant contribution to engineering applications, much more attention must be paid to the determination of the optimal number of clusters. Basically, there are four different approaches to the determination of the cluster numbers. The first approach is to use a certain global validity measure to validate clustering results for a range of cluster numbers. Generally, the determination of the optimum cluster number using global validity measures is very expensive since clustering has to be carried out for a variety of possible cluster numbers. Therefore, the idea of performing progressive clustering was proposed [23], [24], [25], [26]. Since the optimal partition cannot be found in a single run, one could filter the presumably good clusters, eliminate spurious clusters, and merge compatible clusters into larger clusters. This process is continued until the final partition contains good clusters only. The third approach is to perform competitive clustering [27], [28]. By the mechanism of competing for data points only good clusters can survive. Another approach is the use of the family of projection algorithms. Projection algorithms can project high-dimensional data onto a low-dimensional space to facilitate visual inspection of the data. They allow us to visualize high-dimensional data as a two-dimensional scatter plot. As it turns out, this can provide better insight into the data since clustering tendencies in the data may become apparent from the projection. The Sammon's nonlinear mapping is one of the most popular projection algorithms [29]. Most of the conventional projection algorithms utilize the gradient descent method to optimize some kind of objective function; therefore, they are susceptible to premature convergence on a local optimum and usually are computationally expensive.

Recently, many different methods have been proposed to utilize the topology preserving property of the self-organizing feature map (SOM) algorithm to project high-dimensional data onto a low-dimensional space to facilitate visual inspection on the data [30], [31], [32], [33], [34]. Several swarm-intelligence-based data clustering algorithms have also been proposed in recent several years [35], [36], [37], [38], [39], [40], [41], [42], [43], [44], [45], [46], [47], [48], [49]. For example, while some researchers adopted the PSO algorithm rather than the gradient-based method to optimize some kind of cluster validity function (e.g., the quantization error, intra-cluster distance, or the inter-cluster distance) to cluster data [35], [36], [37], [38], Edwards et al. proposed to use the particle swarm optimization (PSO) algorithm to effectively map high-dimensional data points onto a lower-dimensional space so as to identify clusters inherent in the data set [39]. The advantage of these approaches [35], [36], [37], [38], [39] is that they may avoid the local minimization problem incurred from the use of gradient-based optimization tools; however, they still need to pre-specify the number of clusters before they can proceed to the optimization procedure. In addition to those PSO-based approaches [35], [36], [37], [38], [39], a new PSO-based dynamic clustering approach (DCPSO) was proposed to automatically determine the optimum number of clusters and simultaneously cluster the data set [40]. Many different approaches based on the collective behaviors of ants have also been proposed for data clustering and data visualization [41], [42], [43], [44], [45]. The general idea of these researches is that isolated items will be picked up by artificial ants and then dropped at some other location where more items of that type are present. It is a kind of positive feedback that leads to the formation of large clusters. In addition, a novel data visualization method based on the schooling behavior of fish was proposed to allow the user to see complex correlations between data items through the amount of time each fish spends near others [46], [47], [48], [49].

In this paper, a new data projection algorithm, which was inspired by collective behaviors of doves, is proposed. We name the new clustering algorithm the swarm-inspired projection (SIP) algorithm. The paper is organized as follows. In Section 2 the motivations of the new data projection algorithm will be given and then detailed descriptions of the method are discussed. In Section 3, we try to explore the similarities and differences between the proposed SIP algorithm and some existing swarm-based algorithms. In Section 4 a number of computer simulations are given. Some discussions about the SIP algorithm are given in Section 5 and the conclusions of this study are presented in Section 6.