A New Robust Fuzzy Clustering Approach: DBKIFCM

Abstract

A clustering algorithm—Distance Based Gaussian Kernelized Intuitionistic Fuzzy C Means (DBKIFCM) is proposed. This algorithm is based on Gaussian kernel, outlier identification, and intuitionist fuzzy sets. It is intended to resolve the issue of presence of outliers, problem of sensitivity to initialization (STI) and is motivated by good performance of Radial Based Kernelized Intuitionistic Fuzzy C Means (KIFCM-RBF). Experiments are performed on standard 2D data sets such as Diamond (D12 and D15), and Dunn and real-world high dimension data sets such as Fisheriris, Wisconsin breast cancer, and Wine. DBKIFCM outcomes are studied in relation to Fuzzy C Means (FCM), Intuitionistic Fuzzy C Means (IFCM), KIFCM-RBF, Density Oriented Fuzzy C Means (DOFCM). It is observed that proposed approach significantly outperforms the earlier proposed algorithms with respect to outlier identification, effect of noise, issue of STI, and clustering error.

Appendix A: Distance Based Approach of Gaussian Kernelized Intuitionistic Fuzzy C Means

1.1 Proof of Distance-Based Approach of Gaussian Kernelized Intuitionistic Fuzzy C Means

Proof of DBKIFCM which is distance-based approach of KIFCM is given in this appendix. Minimization of Eq. (15), subjected to constraint in Eq. (23) is done by minimizing the following function:

$$ J_{DBKIFCM} = \sum\limits_{i = 1}^{nc + 1} {\sum\limits_{j = 1}^{n} {md_{ji}^{*m} \left\| {\Phi \left( {d_{j} } \right) - \Phi \left( {C_{i} } \right)} \right\|^{2} } } + \sum\limits_{i = 1}^{nc + 1} {\pi_{i}^{*} e^{{1 - \pi_{i}^{*} }} } + \sum\limits_{i = 1}^{nc + 1} {\lambda_{i} \left( {\sum\limits_{j = 1}^{n} {md_{ji}^{*} - 1} } \right)} $$

(29)

where λ_i (i = 1, 2, 3,…, nc) are Langrangian multipliers.

So, to solve this problem, we need to specify $\left\| {\Phi \left( {d_{j} } \right) - \Phi \left( {C_{i} } \right)} \right\|^{2}$ as follows:

$$ \begin{aligned} \left\| {\Phi \left( {d_{j} } \right) - \Phi \left( {C_{i} } \right)} \right\|^{2} & = \left( {\Phi \left( {d_{j} } \right) - \Phi \left( {C_{i} } \right)} \right).\left( {\Phi \left( {d_{j} } \right) - \Phi \left( {C_{i} } \right)} \right) \\ & = \Phi \left( {d_{j} } \right).\Phi \left( {d_{j} } \right) - 2\Phi \left( {d_{j} } \right).\Phi \left( {C_{i} } \right) + \Phi \left( {C_{i} } \right).\Phi \left( {C_{i} } \right) \\ & = K\left( {d_{j} ,d_{j} } \right) - 2K\left( {d_{j} ,C_{i} } \right) + K\left( {C_{i} ,C_{i} } \right) \\ \end{aligned} $$

As in the proposed technique, we adopted Gaussian Kernel, that is:

$$ K(x,y) = \exp \left( { - \tfrac{{\sum {\left| {x_{i}^{a} - y_{i}^{a} } \right|^{b} } }}{{h^{2} }}} \right) $$

(30)

where h denotes kernel width, and a, b are data specific positive number, then K(x,x) = 1. Hence,

$$ \left\| {\Phi \left( {d_{j} } \right) - \Phi \left( {C_{i} } \right)} \right\|^{2} = 2\left( {1 - K\left( {d_{j} ,C_{i} } \right)} \right) $$

(31)

So, Eq. (26) modifies to:

$$ J_{DBKIFCM} = 2\sum\limits_{i = 1}^{nc + 1} {\sum\limits_{j = 1}^{n} {md_{ji}^{*m} \left( {1 - K\left( {d_{j} ,C_{i} } \right)} \right)} } + \sum\limits_{i = 1}^{nc + 1} {\pi_{i}^{*} e^{{1 - \pi_{i}^{*} }} } + \sum\limits_{i = 1}^{nc + 1} {\lambda_{i} \left( {\sum\limits_{j = 1}^{n} {md_{ji}^{*} - 1} } \right)} $$

(32)

Or

$$ J_{DBKIFCM} = 2\sum\limits_{i = 1}^{nc + 1} {\sum\limits_{j = 1}^{n} {md_{ji}^{*m} \left( {1 - \exp \left( { - \tfrac{{\sum {\left| {d_{j}^{a} - C_{i}^{a} } \right|^{b} } }}{{h^{2} }}} \right)} \right)} } + \sum\limits_{i = 1}^{nc + 1} {\pi_{i}^{*} e^{{1 - \pi_{i}^{*} }} } + \sum\limits_{i = 1}^{nc + 1} {\lambda_{i} \left( {\sum\limits_{j = 1}^{n} {md_{ji}^{*} - 1} } \right)} $$

(33)

1.1.1 Partial Derivative of J_DBKIFCM (Given in Eq. (30)) with Respect to $md_{ji}^{*}$

The partial derivative of J_DBKIFCM wrt $md_{ji}^{*}$ is:

$$ \frac{\partial J}{{\partial md_{ji}^{*} }} = 2m \cdot md_{ji}^{*(m - 1)} \left( {1 - K\left( {d_{j} ,C_{i} } \right)} \right) + \lambda_{i} $$

(34)

Equate (31) to zero to solve for $md_{ji}^{*}$:

$$ \frac{\partial J}{{\partial md_{ji}^{*} }} = 0 $$

(35)

$$ \Rightarrow 2m \cdot md_{ji}^{*(m - 1)} \left( {1 - K\left( {d_{j} ,C_{i} } \right)} \right) + \lambda_{i} = 0 $$

(36)

$$ \Rightarrow md_{ji}^{*} = \left( { - \tfrac{{\lambda_{i} }}{{2m.\left( {1 - K\left( {d_{j} ,C_{i} } \right)} \right)}}} \right)^{{\tfrac{1}{{\left( {m - 1} \right)}}}} $$

(37)

To fulfil the following constraint:

$$ 0 \le md_{ji}^{*} \le 1 $$

and $\sum\nolimits_{i = 1}^{nc + 1} {md_{ji}^{*} } = 1$ for $\forall j \in \left[ {1,n} \right]$.

$md_{(nc + 1)j}^{*} = 1$ when j denoting an outlier and $\sum\nolimits_{i = 1}^{nc} {md_{ji}^{*} } = 1$ for j denoting any data point.

$$ md_{ji}^{*} = \tfrac{{md_{ji}^{*} }}{{\sum\nolimits_{i = 1}^{nc + 1} {md_{ji}^{*} } }} $$

(38)

using Eq. (34), Eq. (35) can be written as:

$$ md_{ji}^{*} = \frac{{\left( { - \tfrac{{\lambda_{i} }}{{2m.\left( {1 - K\left( {d_{j} ,C_{i} } \right)} \right)}}} \right)^{{\tfrac{1}{{\left( {m - 1} \right)}}}} }}{{\sum\nolimits_{i = 1}^{nc + 1} {\left( {\left( { - \tfrac{{\lambda_{i} }}{{2m.\left( {1 - K\left( {d_{j} ,C_{i} } \right)} \right)}}} \right)^{{\tfrac{1}{{\left( {m - 1} \right)}}}} } \right)} }} $$

(39)

$$ \Rightarrow md_{ji}^{*} = \frac{{\mathop {\left( { - \tfrac{1}{{\left( {1 - K\left( {d_{j} ,C_{i} } \right)} \right)}}} \right)}\nolimits^{{\tfrac{1}{{\left( {m - 1} \right)}}}} }}{{\sum\nolimits_{i = 1}^{nc + 1} {\left( {\left( { - \tfrac{1}{{\left( {1 - K\left( {d_{j} ,C_{i} } \right)} \right)}}} \right)^{{\tfrac{1}{{\left( {m - 1} \right)}}}} } \right)} }} $$

(40)

Using following equation:

$$ \Phi_{{d_{ji}^{2} }} = \left\| {\Phi \left( {d_{j} } \right) - \Phi \left( {C_{i} } \right)} \right\|^{2} $$

(41)

Equation (43) can be re written as:

$$ md_{ji}^{*} = \frac{1}{{\sum\nolimits_{j = 1}^{nc + 1} {\left( {\frac{{\Phi_{{d_{ic}^{2} }} }}{{\Phi_{{d_{ij}^{2} }} }}} \right)^{{\tfrac{1}{m - 1}}} } }} $$

(42)

1.1.2 Partial Derivation of J_DBKIFCM with Respect to C_i

The partial derivation of J_DBKIFCM with respect to C_i is:

$$ \frac{\partial J}{{\partial C_{i} }} = 2\sum\limits_{j = 1}^{n} {md_{ji}^{*m} \left( { - \exp \left( { - \tfrac{{\sum {\left| {d_{j}^{a} - C_{i}^{a} } \right|^{b} } }}{{h^{2} }}} \right)} \right)\left( {\tfrac{{b\left( {d_{j} - C_{i} } \right)^{(b - 1)} \left( { - 1} \right)(aC_{i}^{(a - 1)} )}}{{h^{2} }}} \right)} $$

(43)

$$ \Rightarrow \frac{\partial J}{{\partial C_{i} }} = 2ab\sum\limits_{j = 1}^{n} {md_{ji}^{*m} \left( {\exp \left( { - \tfrac{{\sum {\left| {d_{j}^{a} - C_{i}^{a} } \right|^{b} } }}{{h^{2} }}} \right)} \right)\left( {\tfrac{{\left( {d_{j}^{a} - C_{i}^{a} } \right)^{(b - 1)} (C_{i}^{(a - 1)} )}}{{h^{2} }}} \right)} $$

(44)

$$ \Rightarrow \frac{\partial J}{{\partial C_{i} }} = 2ab\sum\limits_{j = 1}^{n} {md_{ji}^{*m} \left( {K\left( {d_{j} ,C_{i} } \right)} \right)\left( {\tfrac{{\left( {d_{j} - C_{i} } \right)^{(b - 1)} (C_{i}^{(a - 1)} )}}{{h^{2} }}} \right)} $$

(45)

To solve for C_i Equate Eq. (44) to zero:

$$ \frac{\partial J}{{\partial C_{i} }} = 0 $$

(46)

$$ \Rightarrow 2ab\sum\limits_{j = 1}^{n} {md_{ji}^{*m} \left( {K\left( {d_{j} ,C_{i} } \right)} \right)\left( {\tfrac{{\left( {d_{j} - C_{i} } \right)^{(b - 1)} (C_{i}^{(a - 1)} )}}{{h^{2} }}} \right)} = 0 $$

(47)

$$ \Rightarrow \sum\limits_{j = 1}^{n} {md_{ji}^{*m} \left( {K\left( {d_{j} ,C_{i} } \right)} \right)\left( {\tfrac{{\left( {d_{j} - C_{i} } \right)^{(b - 1)} (C_{i}^{(a - 1)} )}}{{h^{2} }}} \right)} = 0 $$

(48)

From above, either C_i = 0 or C_i = d_j or $md_{ji}^{*} = 0$; $md_{ji}^{*}$ is 0 when d_j is an outlier. C_i = 0 or C_i = d_j is not giving updation of C_i. So, update C_i using following equation:

$$ C_{i} = \frac{{\sum\nolimits_{j = 1}^{n} {md_{ji}^{*m} \left( {d_{j} } \right)} }}{{\sum\nolimits_{j = 1}^{n} {md_{ji}^{*m} } }} $$

(49)