Robust Optimization for Clustering

Abstract

In this paper, we investigate the robust optimization for the minimum sum-of squares clustering (MSSC) problem. Each data point is assumed to belong to a box-type uncertainty set. Following the robust optimization paradigm, we obtain a robust formulation that can be interpreted as a combination of MSSC and k-median clustering criteria. A DCA-based algorithm is developed to solve the resulting robust problem. Preliminary numerical results on real datasets show that the proposed robust optimization approach is superior than MSSC and k-median clustering approaches.

A Appendix

Given $a, a_i \in \mathbb {R}$, $b_i \in \mathbb {R}_{++}$ ($i=1,\dots ,m$), consider the problem

$$\begin{aligned} \min \left\{ f(x) := \frac{1}{2} (x - a)^2 + \sum _{i=1}^m b_i |x - a_i| : x \in \mathbb {R}\right\} . \end{aligned}$$

(13)

Assume that $\{a_i\}_{i=1}^m$ is in ascending order $a_1 \le a_2 \le \dots \le a_m$. Denote by $f^-(x)$ (resp. $f^+(x)$) the left (resp. right) derivative of f at x. We have

$$\begin{aligned} f^-(x) = x - a + \sum _{i=1}^m b_i \delta _i, \quad f^+(x) = x - a + \sum _{i=1}^m b_i \sigma _i, \end{aligned}$$

(14)

where $\delta _i = -1$ if $x \le a_i$ and 1 otherwise, $\sigma _i = -1$ if $x < a_i$ and 1 otherwise ($\forall i = 1,\dots ,m$). For convenience, let $a_0 = -\infty$ and $a_{m+1} = +\infty$. Note that (13) is strongly convex, so the solution $x^*$ is unique. We can find out the place where $x^*$ is by using the following property.

Proposition 1

Let $\bar{a} = {{\mathrm{arg\,min}}}\left\{ \sum _{i=1}^m b_i |x - a_i| : x \in \mathbb {R}\right\}$, $l_b = \min (a,\bar{a})$, and $u_b = \max (a,\bar{a})$. We have the following assertions:

• (i) $x^* \in [l_b,u_b]$.

• (ii) If $f^-(a_i) > 0$, then $x^* \in (a_0,a_i)$. If $f^+(a_i) < 0$, then $x^* \in (a_i,a_{m+1})$ . If $f^-(a_i) \le 0$ and $f^+(a_i) \ge 0$, then $x^* = a_i$.

Proof

Given g is any finite convex function on $\mathbb {R}$. We have $\partial g(x) = [f^-(x),f^+(x)]$ for any $x \in \mathbb {R}$, and $\tilde{x} \in \mathop {{{\mathrm{arg\,min}}}}_{x \in \mathbb {R}} g(x) \Leftrightarrow 0 \in \partial g(\tilde{x}) \Leftrightarrow f^-(\tilde{x}) \le 0 \le f^+(\tilde{x})$. Moreover, if $\tilde{x}$ is the unique optimum of g on $\mathbb {R}$, then for any $x \ne \tilde{x}$,

$$\begin{aligned} 0 > g(\tilde{x}) - g(x) \ge y (\tilde{x}-x), \quad \forall y \in \partial g(x). \end{aligned}$$

Therefore, $g^+(x) < 0$ if $x < \tilde{x}$, and $g^-(x) > 0$ if $x>\tilde{x}$. We have ii) is proved.

Let $f_1(x) = \frac{1}{2}(x-a)^2$ and $f_2(x) = \sum _{i=1}^m b_i |x - a_i|$. We have $f_1$ and $f_2$ are finite convex functions on $\mathbb {R}$, and a (resp. $\bar{a}$) is optimum of $f_1$ (resp. $f_2$) on $\mathbb {R}$. Without loss of generality we assume that $a \le \bar{a}$. Then $f^+(a) = f_2^+(a) \le 0$ and $f^-(\bar{a}) = f_1^-(\bar{a}) \ge 0$. This implies that $a \le x^* \le \bar{a}$. Thus, i) is proved.

Once we find out the interval where $x^*$ is and f is differentiable, $x^*$ is easily determined by solving the equation $f'(x) = 0$. The specific procedure for finding the solution $x^*$ of problem (13) is given in Algorithm 1. It is clear that Algorithm 1 terminates after at most m steps.