Clustering data stream with uncertainty using belief function theory and fading function

Abstract

Data stream clustering faces major challenges such as lack of memory and time. Therefore, traditional clustering methods are not suitable for this kind of data. On the other hand, most data stream clustering methods do not consider the problems of uncertainty and ambiguity in the data. So, in this case, where an object is close to a set of clusters, this object cannot be correctly and simply categorized. The aim of this study is to provide a new method for clustering data stream, called clustering data stream using belief function, with regard to the problem of uncertain and ambiguous data. In the proposed method, the belief function theory is used to cluster objects into single clusters or a set of clusters and determines the structure of data. In addition, using window, weighted centers, and the fading function overcomes the restrictions of data stream. The results of the experiments have been compared with state-of-the-art methods, which show the superiority of the proposed method in terms of purity, error rate, and ambiguity rate measures.

Appendices

Appendix 1

By taking the derivative of Eq. (20) with respect to $\lambda_{i}$, we have:

$$ \frac{{\partial {\mathcal{L}}}}{{\partial \lambda_{i} }} = \mathop \sum \limits_{{j\left| { c_{j} } \right\rangle 1}} m_{ij} + \mathop \sum \limits_{{l| c_{l} = 1}} m_{il} + m_{i\emptyset } - 1 = 0 $$

(31)

By adjusting and taking the derivative of Eq. (20) with respect to $m_{ij}$, $m_{il}$, and $m_{i\emptyset }$, we have:

$$ m_{ij} = \left( {\frac{{\lambda_{i} }}{{(\beta ){\text{ow}}_{i} 2^{{ - \lambda \Delta t_{i} }} }}} \right)^{{\frac{1}{\beta - 1}}} \cdot \left( {\frac{1}{{\frac{{\mathop \sum \nolimits_{{\omega_{k} \in A_{j} }} d_{ik}^{2} + d_{ij}^{2} }}{{c_{j} + \gamma }}}}} \right)^{{\frac{1}{\beta - 1}}} $$

(32)

$$ m_{il} = \left( {\frac{{\lambda_{i} }}{{(\beta ){\text{ow}}_{i} 2^{{ - \lambda \Delta t_{i} }} }}} \right)^{{\frac{1}{\beta - 1}}} \cdot \left( {\frac{1}{{d_{il}^{2} }}} \right)^{{\frac{1}{\beta - 1}}} $$

(33)

$$ m_{i\emptyset } = \left( {\frac{{\lambda_{i} }}{{(\beta ){\text{ow}}_{i} 2^{{ - \lambda \Delta t_{i} }} }}} \right)^{{\frac{1}{\beta - 1}}} \cdot \left( {\frac{1}{{\delta^{2} }}} \right)^{{\frac{1}{\beta - 1}}} $$

(34)

And by using these equations in Eq. (31), we have:

$$ \begin{aligned} & \left( {\frac{{\lambda_{i} }}{{(\beta ){\text{ow}}_{i} 2^{{ - \lambda \Delta t_{i} }} }}} \right)^{{\frac{1}{\beta - 1}}} \\ & \quad = \left( {\mathop \sum \limits_{{j\left| { c_{j} } \right\rangle 1}} \frac{1}{{ \left( {\frac{{\mathop \sum \nolimits_{{\omega_{k} \in A_{j} }} d_{ik}^{2} + d_{ij}^{2} }}{{c_{j} + \gamma }}} \right)^{{\frac{1}{\beta - 1}}} }} + \mathop \sum \limits_{{l| c_{l} = 1}} \frac{1}{{ d_{il}^{{\frac{2}{\beta - 1}}} }} + \mathop \sum \limits_{{j| c_{j} = 0}} \frac{1}{{ \delta^{{\frac{2}{\beta - 1}}} }} } \right)^{ - 1} \\ \end{aligned} $$

(35)

Then, the equations of masses are obtained using Eq. (35) in Eqs. (32), (33), and (34).

Appendix 2

Taking the derivative of the objective function with respect to v, we have:

$$ \frac{{\partial J_{\text{DSCBF}} }}{{\partial v_{l} }} = \mathop \sum \limits_{i = 1}^{n} \mathop \sum \limits_{{j\left| { c_{j} } \right\rangle 1}} 2^{{ - \lambda \Delta t_{i} }} {\text{ow}}_{i} m_{ij}^{\beta } \frac{{\frac{{\partial d_{il}^{2} }}{{\partial v_{l} }} + \frac{{\partial d_{ij}^{2} }}{{\partial v_{l} }}}}{{c_{j} + \gamma }} + \mathop \sum \limits_{i = 1}^{n} \mathop \sum \limits_{{l| c_{l} = 1}} 2^{{ - \lambda \Delta t_{i} }} {\text{ow}}_{i} m_{il}^{\beta } \frac{{\partial d_{il}^{2} }}{{\partial v_{l} }} $$

(36)

The partial derivatives of the distances with respect to v are given by

$$ \frac{{\partial d_{il}^{2} }}{{\partial v_{l} }} = 2\left( {v_{l} - x_{i} } \right)\quad c_{l} = 1 $$

(37)

$$ \begin{aligned} \frac{{\partial d_{ij}^{2} }}{{\partial v_{l} }} & = \frac{{2\left( {v_{l} - x_{i} } \right) + \frac{2}{{c_{j} }}\left( {\frac{{\mathop \sum \nolimits_{{ \omega_{g} \in A_{j} }} v_{g} }}{{c_{j} }} - x_{i} } \right)}}{{c_{j} + \gamma }} \\ & \quad \omega_{l} \in A_{j} \quad c_{j} > 1 \\ \end{aligned} $$

(38)

Using Eqs. (37) and (38) in Eq. (36), we have:

$$ \begin{aligned} & \left( {\mathop \sum \limits_{i = 1}^{n} 2^{{ - \lambda \Delta t_{i} }} {\text{ow}}_{i} m_{il}^{\beta } + \mathop \sum \limits_{i = 1}^{n} \mathop \sum \limits_{{\omega_{l} \in A_{j} }} 2^{{ - \lambda \Delta t_{i} }} {\text{ow}}_{i} m_{ij}^{\beta } \frac{{1 + \frac{1}{{c_{j} }}}}{{c_{j} + \gamma }}} \right)x_{i} \\ & \quad = \mathop \sum \limits_{i = 1}^{n} 2^{{ - \lambda \Delta t_{i} }} {\text{ow}}_{i} m_{il}^{\beta } v_{l} + \mathop \sum \limits_{i = 1}^{n} \mathop \sum \limits_{{\omega_{l} \in A_{j} }} 2^{{ - \lambda \Delta t_{i} }} {\text{ow}}_{i} m_{ij}^{\beta } \frac{{\left( {\frac{{v_{l} + \mathop \sum \nolimits_{{ \omega_{g} \in A_{j} }} v_{g} }}{{c_{j}^{2} }}} \right)}}{{c_{j} + \gamma }} \\ \end{aligned} $$

(39)

To simplify the calculations and obtain the centers, the linear equation (Eq. 23) is presented.