CEVCLUS: evidential clustering with instance-level constraints for relational data

Abstract

Recent advances in clustering consider incorporating background knowledge in the partitioning algorithm, using, e.g., pairwise constraints between objects. As a matter of fact, prior information, when available, often makes it possible to better retrieve meaningful clusters in data. Here, this approach is investigated in the framework of belief functions, which allows us to handle the imprecision and the uncertainty of the clustering process. In this context, the EVCLUS algorithm was proposed for partitioning objects described by a dissimilarity matrix. It is extended here so as to take pairwise constraints into account, by adding a term to its objective function. This term corresponds to a penalty term that expresses pairwise constraints in the belief function framework. Various synthetic and real datasets are considered to demonstrate the interest of the proposed method, called CEVCLUS, and two applications are presented. The performances of CEVCLUS are also compared to those of other constrained clustering algorithms.

Appendix A: Optimization algorithm

The minimization of $J_{CEVCLUS}$ can be performed using any unconstrained nonlinear programming algorithm. In the experiments reported in Sect. 4, we used the same gradient-based optimization as in Denœux and Masson (2004). This method is briefly sketched below.

Let $\varvec{w}$ be the vector of parameters and $J(\varvec{w})$ the objective function to be minimized. The algorithm is a variant of gradient descent in which each parameter $w_i$ has its own step size $\eta _j$, and the step sizes are adapted during the optimization process, depending on the evolution of the objective function and on the sign of the derivatives at successive iterations. Let $t$ be the iteration counter. Let us first assume that the objective function has decreased between iterations $t-1$ and $t$. Then the following rule is applied to update each step size $\eta _j$:

$$\begin{aligned} \eta _j(t)= \left\{ \begin{array}{l@{\quad }l} \beta \; \eta _j(t-1) &{} \text{ if } \frac{\partial J}{\partial w_j} (t-1) \cdot \frac{\partial J}{\partial w_j} (t) >0\\ \gamma \; \eta _j(t-1) &{} \mathrm {otherwise, } \end{array} \right. \end{aligned}$$

(24)

where $\beta >1$ and $\gamma <1$ are two coefficients. Hence, the step size is increased if the derivatives have kept the same sign during two iterations, and it is increased if the sign of the derivative has changed, which indicates that we have “jumped over” a minimum. The parameters are then updated by:

$$\begin{aligned} w_j(t+1) = w_j(t) - \eta _j(t) \frac{\partial J}{\partial w_j} (t). \end{aligned}$$

(25)

If now the objective function has increased between iterations $t-1$ and $t$, all step sizes are decreased simultaneously:

$$\begin{aligned} \eta _j(t)=\delta \; \eta _j(t-1) \quad \forall j \end{aligned}$$

(26)

with $\delta <1$, and the parameters are updated starting from where they were at the previous iteration:

$$\begin{aligned} w_j(t+1) = w_j(t-1) - \eta _j(t) \frac{\partial J}{\partial w_j} (t-1). \end{aligned}$$

(27)

As in Denœux and Masson (2004), we set the parameters $\beta$, $\gamma$ and $\delta$ to $1.2$, $0.8$ and $0.5$ in our experiments.