A new swarm-based efficient data clustering approach using KHM and fuzzy logic

Abstract

Clustering is a useful technique to create different groups of objects on the basis of their nature. Objects of same group are of similar in nature and differ to the objects of other groups. Clustering has proved its importance in various fields such as information retrieval, bioinformatics, image processing and many others. In this paper, particle swarm optimization (PSO) technique is used with K-harmonic means (KHM) for clustering. PSO overcomes the limitations of KHM like local optimum problem. Fuzzy logic is also employed in this paper to make PSO adaptive in nature by controlling various parameters. The performance of the proposed approach is validated on five benchmark datasets in terms of inter-clustering distance, intra-clustering distance, F-measure and fitness value. The results of proposed approach are compared with well-known conventional clustering techniques such as K-means, KHM and fuzzy C-means along with different state-of-the-art clustering approaches. Two text-based benchmark datasets such as CACM and CISI are also used to test the performance of all clustering approaches. The proposed clustering approach gives better results in comparison with other clustering approaches as clear from both the experimental and statistical analyses.

Appendices

Appendix A: K-harmonic means clustering algorithm

This algorithm was proposed by Zhang et al. (1999, 2000). The variants of KHM were also proposed by Hammerly and Elkan (2002). KHM gives dynamic weight to each data point by averaging of the harmonic means of the distance from each data point to all centers. The harmonic average assigns a large weight to a data point that is not close to any centers and a small weight to the data point that is close to one or more centers. Therefore, KHM is less sensitive to the initialization than the K-means. Some notations are described in Table 12 as used in KHM algorithm, before discussing it:

Table 12 Notations and descriptions used in KHM

The detail of KHM clustering algorithm is given as follows:

• 1. Initially, select the centers randomly.

• 2. Determine objective function value by (A.1) as defined below

  $$ \text{KHM}\,\left( {X, C} \right) = \sum\limits_{i = 1}^{n} {\frac{k}{{\sum\nolimits_{j = 1}^{k} {\frac{1}{{\left\| {x_{i} - c_{j} } \right\|^{p} }}} }}} $$

  (A.1)

  where p is an input parameter and typically p ≥ 2.

• 3. Compute membership value m (c_j/x_i) in each center c_j for each data point x_i by (A.2) as defined below

  $$ m\,\left( {c_{j} , x_{i} } \right) = \frac{{\left\| {x_{i} - c_{j} } \right\|^{ - p - 2} }}{{\sum\nolimits_{j = 1}^{k} {\left\| {x_{i} - c_{j} } \right\|^{ - p - 2} } }}. $$

  (A.2)

• 4. In this step, compute weights W (x_i) for each data point x_i by (A.3) as follows

  $$ W\,\left( {x_{i} } \right) = \frac{{\sum\nolimits_{j = 1}^{k} {\left\| {x_{i} - c_{j} } \right\|^{ - p - 2} } }}{{\left( {\sum\nolimits_{j = 1}^{k} {\left\| {x_{i} - c_{j} } \right\|^{ - p - 2} } } \right)^{2} }}. $$

  (A.3)

• 5. Now re-compute the locations of each center c_j from all the data points x_i according to their memberships and weights define by (A.4).

  $$ c_{j} = \frac{{\sum\nolimits_{i = 1}^{n} {m\left( {c_{j} /x_{i} } \right) w\left( {x_{i} } \right) x_{i} } }}{{\sum\nolimits_{i = 1}^{n} {m\left( {c_{j} /x_{i} } \right) w\left( {x_{i} } \right) } }} $$

  (A.4)

• 6. Repeat steps 2–5 for predefined number of iterations or until KHM(X, C) does not change significantly.

• 7. Assign data point x_i to cluster j with the biggest m(c_j/x_i).

It is demonstrated that KHM is essentially insensitive to the initialization of the centers (Zhang et al. 1999), while it tends to converge to local optima (Güngör and Ünler 2008).

Appendix B: Particle swarm optimization

PSO is an evolutionary approach which has been successfully applied to science and many practical fields (Aupetit et al. 2007; Liao et al. 2007). It is a sociologically inspired optimization algorithm which is based on population. Each particle in PSO represents an individual, and all the particles form a swarm. The solution space for any problem is formulated as a search space in PSO. Each position of search space represents a correlated solution of the problem. Each particle moves according to its velocity. The movement of a particle is computed as (B.1) and (B.2)

$$ x_{i} \left( {t + 1} \right) \leftarrow \, x_{i} \left( {t + 1} \right) + v_{i} \left( t \right) $$

(B.1)

$$ \begin{aligned} v_{i} \left( {t + 1} \right) \, &\leftarrow \omega v_{i} \left( t \right) + c_{1} {\text{rand}}_{1} \left( {pbest_{i} \left( t \right) \, - \, x_{i} \left( t \right)} \right) \\&\quad+ c_{2} {\text{rand}}_{2} \left( {gbest\left( t \right) - \, x_{i} \left( t \right)} \right) \end{aligned} $$

(B.2)

where x_i(t) is the position of particle i at time t, v_i(t) is the velocity of particle i at time t, ω is an inertia weight scaling the previous velocity, rand₁ and rand₂ are random variables between 0 and 1, pbest_i(t) is the best position found by particle i so far, gbest(t) is the best position of whole swarm so far, c₁ and c₂ are two acceleration coefficients that scale the influence of pbest_i(t) and gbest(t), respectively.