Subspace Clustering by Capped $l_1$ Norm

Abstract

Subspace clustering, as an important clustering problem, has drawn much attention in recent years. State-of-the-art methods generally try to design an efficient model to regularize the coefficient matrix while ignore the influence of the noise model on subspace clustering. However, the real data are always contaminated by the noise and the corresponding subspace structures are likely to be corrupted. In order to solve this problem, we propose a novel subspace clustering algorithm by employing capped $l_1$ norm to deal with the noise. Consequently, the noise term with large error can be penalized by the proposed method. So it is more robust to the noise. Furthermore, the grouping effect of our method is theoretically proved, which means highly correlated points can be grouped together. Finally, the experimental results on two real databases show that our method outperforms state-of-the-art methods.

Appendix

1.1 Proof of Theorem 2

Proof

Let

$$\begin{aligned} L(z)=\left\| {x - Xz} \right\| _2^2 + \frac{\lambda }{{{s}}}\left\| z \right\| _2^2{}. \end{aligned}$$

(23)

Since ${z^*} = \arg \mathop {\min }\limits _z L(z)$. We have

$$\begin{aligned} {\left. {\frac{{\partial L(z)}}{{\partial z}}} \right| _{z = {z^*}}=0}{}. \end{aligned}$$

(24)

It gives

$$\begin{aligned} - 2x_i^T(x - X{z^*}) + 2\frac{\lambda }{s}z_i^* = 0,\end{aligned}$$

(25)

$$\begin{aligned} - 2x_j^T(x - X{z^*}) + 2\frac{\lambda }{s}z_j^* = 0. \end{aligned}$$

(26)

Equations (25) and (26) give

$$\begin{aligned} z_i^* - z_j^* = \frac{s}{\lambda }(x_i^T - x_j^T)(x - X{z^*}){}. \end{aligned}$$

(27)

Note that each column of the data X is normalized, we get ${\left\| {x_i^T - x_j^T} \right\| _2} = 2\sqrt{1 - r}$, where $r = x_i^T{x_j}$. Since $z^*$ is the optimal solution of the problem (21), we have

$$\begin{aligned} \left\| {x - X{z^*}} \right\| _2^2 + \frac{\lambda }{s}\left\| {{z^*}} \right\| _2^2 = L({z^*}) \le L(0) = \left\| x \right\| _2^2{}. \end{aligned}$$

(28)

Thus $\left\| {x - X{z^*}} \right\| _2^2 \le \left\| x \right\| _2^2$. Finally, we get

$$\begin{aligned} \frac{{\left| {z_i^* - z_j^*} \right| }}{{{{\left\| x \right\| }_2}}} \le \frac{{{s}}}{\lambda }\sqrt{2(1 - r)}{}. \end{aligned}$$

(29)