Low-Rank Online Metric Learning

Abstract

Image classification is a key problem in computer vision community. Most of the conventional visual recognition systems usually train an image classifier in an offline batch mode with all training data provided in advance. Unfortunately in many practical applications, usually only a small amount of training samples are available in the initialization stage and many more would come sequentially during the online process. Because the image data characteristics could dramatically change over time, it is important for the classifier to adapt to the new data incrementally. In this chapter, we present an online metric learning model to address the online image classification/scene recognition problem via adaptive similarity measurement. Given a number of labeled samples followed by a sequential input of unseen testing samples, the similarity metric is learned to maximize the margin of the distance among different classes of samples. By considering the low-rank constraint, our online metric learning model not only provides competitive performance compared with the state-of-the-art methods, but also guarantees to converge. A bi-linear graph is also applied to model the pair-wise similarity, and an unseen sample is labeled depending on the graph-based label propagation, while the model can also self-update using the new samples that are more confident labeled. With the ability of online learning, our methodology can well handle the large-scale streaming video data with the ability of incremental self-update. We also demonstrate that the low-rank property widely exists in natural data. In the experiments, we evaluate our model to online scene categorization and experiments on various benchmark datasets and comparisons with state-of-the-art methods demonstrate the effectiveness and efficiency of our algorithm.

$\copyright$ [2013] IEEE. Reprinted, with permission, from Yang Cong, Ji Liu, Junsong Yuan, Jiebo Luo “Self-supervised online metric learning with low rank constraint for scene categorization”, IEEE Transactions on Image Processing, Vol. 22, No. 8, August 2013, pp. 3179–3191.

Appendix

1.1 Proof of Theorem 1

Proof

Since $W$ is a PSD matrix, it can be decomposed as $W=UU^T$ where $U\in \mathbb {R}^{d\times d}$. Consider the following equation $X^TV = X^TU$ with respect to $V$. Define $B\in \mathbb {R}^{d\times (d-r)}$ with linear dependent columns $B_{.i}$’s in the null space of $X^T$. One can obtain the solution as $V=U+BZ$ where $Z\in \mathbb {R}^{(d-r)\times d}$. Split $U$ and $B$ into two parts $U={U1 \atopwithdelims ()U2}$ and $B={B1\atopwithdelims ()B2}$ where $U1\in \mathbb {R}^{(d-r)\times d}$, $U2\in \mathbb {R}^{r\times d}$, $B1\in \mathbb {R}^{(d-r)\times (d-r)}$, and $B2\in \mathbb {R}^{r\times r}$. Define $Z=-B1^{-1}U1$. One verifies that $V={0 \atopwithdelims ()U2-B2B1^{-1}U1}$ and its rank is at most $r$. Since $X^TU=X^TV$, we obtain $X^TWX=X^TQX$ and the rank of $Q$ is r by letting $Q=VV^T$. $\blacksquare$

1.2 Proof of Theorem 2

Proof

Decompose $C$ into the symmetric space and the skew symmetric space, i.e., $C = C_y + C_k$ where $C_y = {1 \over 2}(C+C^T)$ and $C_k = {1\over 2}(C-C^T)$. Note that $\langle C_y, C_k\rangle = 0$. Consider $W\succeq 0$ ($W$ must be symmetric) in the following

$$\begin{aligned} \Vert W-C\Vert ^2_F&=\Vert W-C_y-C_k\Vert ^2_F \nonumber \\&=\Vert W-C_y\Vert ^2_F + \Vert C_k\Vert ^2_F + 2\langle W-C_y, C_k \rangle \nonumber \\&=\Vert W-C_y\Vert ^2_F + \Vert C_k\Vert ^2_F. \end{aligned}$$

(26)

Thus, we obtain $prox_{\gamma P,\Omega }(C)=prox_{\gamma P,\Omega }(C_y)$.

$$\begin{aligned}&\min _{W\succeq 0}~{1\over 2}\Vert W-C_y\Vert ^2_F + \gamma \Vert W\Vert _* \nonumber \\&\quad =\min _{W\succeq 0}~{1\over 2}\Vert W-C_y\Vert ^2_F + \max _{\Vert Z\Vert \le \gamma , Z\in S\mathbb {R}^{d\times d}} \langle W,Z \rangle \nonumber \\&\quad =\max _{\Vert Z\Vert \le \gamma , Z\in S\mathbb {R}^{d\times d}}\min _{W\succeq 0}~{1\over 2}\Vert W-C_y\Vert ^2_F + \langle W,Z \rangle \nonumber \\&\quad =\max _{\Vert Z\Vert \le \gamma , Z\in S\mathbb {R}^{d\times d}}\min _{W\succeq 0}~{1\over 2}\Vert W-C_y+Z\Vert ^2_F + \langle C_y,Z \rangle - {1\over 2}\Vert Z\Vert ^2_F\nonumber \\&\quad =\max _{\Vert Z\Vert \le \gamma , Z\in S\mathbb {R}^{d\times d}}~{1\over 2}\Vert (C_y-Z)^-\Vert ^2_F + \langle C_y,Z \rangle - {1\over 2}\Vert Z\Vert ^2_F \end{aligned}$$

(27)

The first equality uses the dual form of the trace norm of a PSD matrix, where $S\mathbb {R}$ denotes the symmetric space. The second equality is due to Von Neumann theorem. The last equality uses the result that the projection from a symmetric matrix $X$ onto the SDP cone is $X^+$, which also implies that $W=(C_y-Z)^+$.

It follows that

$$\begin{aligned}&\max _{\Vert Z\Vert \le \gamma , Z\in S\mathbb {R}^{d\times d}}~{1\over 2}\Vert (C_y+Z)^-\Vert ^2_F + \langle C_y,Z \rangle - {1\over 2}\Vert Z\Vert ^2_F\nonumber \\&\quad =\max _{\Vert Z\Vert \le \gamma , Z\in S\mathbb {R}^{d\times d}}~{1\over 2}\Vert (C_y-Z)^-\Vert ^2_F - {1\over 2}\Vert C_y-Z\Vert ^2_F + {1\over 2}\Vert C_y\Vert ^2_F\nonumber \\&\quad =\max _{\Vert Z\Vert \le \gamma , Z\in S\mathbb {R}^{d\times d}}~-{1\over 2}\Vert (C_y-Z)^+\Vert ^2_F + {1\over 2}\Vert C_y\Vert ^2_F \end{aligned}$$

(28)

From the last formulation, we obtain the optimal $Z^*=\mathcal {T}_\gamma (C_y)$ and the optimal $W^*=(C_y-Z^*)^+=(C_y-\mathcal {T}_\gamma (C_y))^+=\mathcal {D}_\gamma (C_y)^+=D_\gamma (C_y)$. It completes our proof. $\blacksquare$