Exemplar-Guided Similarity Learning on Polynomial Kernel Feature Map for Person Re-identification

Abstract

Person re-identification is a crucial problem for video surveillance, aiming to discover the correct matches for a probe person image from a set of gallery person images. To directly describe the image pair, we present a novel organization of polynomial kernel feature map in a high dimensional feature space to break down the variability of positive person pairs. An exemplar-guided similarity function is built on the map, which consists of multiple sub-functions. Each sub-function is associated with an “exemplar” image being responsible for a particular type of image pair, thus excels at separating the persons with similar appearance. We formulate a unified learning problem including a relaxed loss term as well as two kinds of regularization strategies particularly designed for the feature map. The corresponding optimization algorithm jointly optimizes the coefficients of all the sub-functions and selects the proper exemplars for a better discrimination. The proposed method is extensively evaluated on six public datasets, where we thoroughly analyze the contribution of each component and verify the generalizability of our approach by cross-dataset experiments. Results show that the new method can achieve consistent improvements over state-of-the-art methods.

Appendices

Dual Form Deviation for Updating ${\mathbf{U}}_{1}$

In this section, we will give the deviation from Eqs. (31) to (35). As $g_{1}({\mathbf{U}}_{1})=L({\mathbf{U}}_{1})=\frac{1}{N}\sum _{n=1}^{N}[1-\langle \varOmega ({\mathbf{x}}_{n}),{\mathbf{U}}_{1}\rangle ]_{+}$, the problem of Eq. (31) equals:

$$\begin{aligned} \begin{aligned}&\min _{{\mathbf{U}}_{1}, \varvec{\xi }} \quad \frac{\rho }{2}\Vert {\mathbf{U}}_{1}-\left( {\mathbf{U}}_{3}^{l}-\varvec{\varLambda }_{1}^{l}\right) \Vert _{F}^{2}+ \frac{1}{N}\sum _{n=1}^{N} \xi _{n}\\&\text {s.t.} \quad 1 - \langle \varOmega ({\mathbf{x}}_{n}),{\mathbf{U}}_{1}\rangle \le \xi _{n}, \quad \xi _{n} \ge 0, \quad \forall n. \end{aligned} \end{aligned}$$

(40)

The Lagrangian associated with Eq. (40) is:

$$\begin{aligned}&La_{1}({\mathbf{U}}_{1},\varvec{\xi }, \varvec{\alpha }, \varvec{\beta }) = \frac{\rho }{2}\Vert {\mathbf{U}}_{1}-\left( {\mathbf{U}}_{3}^{l}-\varvec{\varLambda }_{1}^{l} \right) \Vert _{F}^{2}+\frac{1}{N}{\mathbf{1}}^{\top }\varvec{\xi }\nonumber \\&\quad +\sum _{n=1}^{N}(1- \langle \varOmega ({\mathbf{x}}_{n}), {\mathbf{U}}_{1}\rangle _{F})\alpha _{n}- \varvec{\alpha }^{\top } \varvec{\xi }- \varvec{\beta }^{\top }\varvec{\xi }, \end{aligned}$$

(41)

where $\varvec{\alpha }$ and $\varvec{\beta }$ are dual variable vectors with non-negative elements, and $\alpha _{n}$ is the n-th element of $\varvec{\alpha }$. The Lagrangian dual is $h_{1}(\varvec{\alpha }, \varvec{\beta })= \inf _{{\mathbf{U}}_{1}, \varvec{\xi }} La_{1}({\mathbf{U}}_{1},\varvec{\xi },\varvec{\alpha }, \varvec{\beta })$, with $\varvec{\alpha }\succeq 0$ and $\varvec{\beta }\succeq 0$ being the dual feasible. We solve ${\mathbf{U}}_{1}$ and $\varvec{\xi }$ from the optimality condition:

$$\begin{aligned} \begin{aligned}&\frac{\partial La_{1}}{\partial {\mathbf{U}}_{1}} = \rho \left( {\mathbf{U}}_{1}-{\mathbf{U}}_{3}^{l}+\varvec{\varLambda }_{1}^{l}\right) -\sum _{n=1}^{N} \varOmega ({\mathbf{x}}_{n})\alpha _{n} = {\mathbf{0}} \\&\frac{\partial La_{1}}{\partial \varvec{\xi }} = \frac{1}{N}{\mathbf{1}}^{\top }-\varvec{\alpha }^{\top }-\varvec{\beta }^{\top } = {\mathbf{0}}. \end{aligned} \end{aligned}$$

(42)

Taking Eq. (42) into Eq. (41), $h_{1}(\varvec{\alpha }, \varvec{\beta })$ becomes:

$$\begin{aligned} \begin{aligned} h_{1}(\varvec{\alpha }, \varvec{\beta })=&-\frac{1}{2\rho }\sum _{i=1}^{N}\sum _{j=1}^{N}\alpha _{i}\alpha _{j}\langle \varOmega ({\mathbf{x}}_{i}), \varOmega ({\mathbf{x}}_{j})\rangle _{F}\\&+{\mathbf{1}}^{\top }\varvec{\alpha }- \sum _{n=1}^{N}\left\langle \varOmega ({\mathbf{x}}_{n}), {\mathbf{U}}_{3}^{l}-\varvec{\varLambda }_{1}^{l} \right\rangle _{F}\alpha _{n}. \end{aligned} \end{aligned}$$

(43)

The dual variable vector $\varvec{\beta }$ is eliminated. We define the kernel matrix ${\mathbf{H}}$ with each element $H_{i,j}=\langle \varOmega ({\mathbf{x}}_{i}), \varOmega ({\mathbf{x}}_{j})\rangle _{F}$ and define a vector ${\mathbf{b}}$ with $b_{n} = \langle \varOmega ({\mathbf{x}}_{n}), {\mathbf{U}}_{3}^{l}-\varvec{\varLambda }_{1}^{l}\rangle _{F}-1$, the dual function becomes:

$$\begin{aligned} h(\varvec{\alpha })=-\frac{1}{2\rho }\varvec{\alpha }^{\top }{\mathbf{H}}\varvec{\alpha }-{\mathbf{b}}^{\top } \varvec{\alpha }\end{aligned}$$

(44)

Also because $\frac{1}{N}{\mathbf{1}}-\varvec{\alpha }- \varvec{\beta }= {\mathbf{0}}$ and $\varvec{\alpha }\succeq 0$, $\alpha _{n}$ belongs to the domain $[0,\frac{1}{N}]$. We therefore obtain the standard quadratic programming problem of Eq. (35). With optimal $\varvec{\alpha }^{*}$, Eq. (36) is obtained according to Eq. (42).

A Separated ADMM Algorithm for Updating ${\mathbf{U}}_{3}$

In this section, we present a separated ADMM algorithm for the update of ${\mathbf{U}}_{3}$. Directly projecting a non-symmetric matrix onto ${\mathbb {S}}_{-}^{d}$ is difficult, we therefore introduce an auxiliary set ${\mathcal {C}}'=\{{\mathbf{U}}| {\mathbf{W}}_{M}^{c,k}\in {\mathbb {S}}^{d}, \forall c, \forall k\}$, and define $g_{3}'({\mathbf{E}}) = \infty \delta [{\mathbf{E}}\in {\mathcal {C}}']$. As ${\mathcal {C}}\subset {\mathcal {C}}'$, the sub-problem of Eq. (33) is equivalent to:

$$\begin{aligned} \begin{aligned}&\min _{{\mathbf{E}}, {\mathbf{F}}} \quad g_{3}'({\mathbf{E}})+ \rho \Vert {\mathbf{E}}-{\mathbf{B}}\Vert _{F}^{2}+g_{3}({\mathbf{F}}) \\&\quad \text {s.t.} \quad {\mathbf{E}} = {\mathbf{F}}, \end{aligned} \end{aligned}$$

(45)

where ${\mathbf{B}}= \frac{1}{2}({\mathbf{U}}_{1}^{l+1}+{\mathbf{U}}_{2}^{l+1}+\varvec{\varLambda }_{1}^{l}+\varvec{\varLambda }_{2}^{l})$, and ${\mathbf{U}}_{3}^{l+1}$ equals the optimal ${\mathbf{E}}$ or ${\mathbf{F}}$. By introducing Lagrange multipliers ${\mathbf{G}}$, we obtain the augmented Lagrangian:

$$\begin{aligned} \begin{aligned} La_2( {\mathbf{E}},{\mathbf{F}}, {\mathbf{G}})&= g_{3}'({\mathbf{E}})+ \rho \Vert {\mathbf{E}}-{\mathbf{B}}\Vert _{F}^{2}+g_{3}({\mathbf{F}})\\ +\,\rho '\langle {\mathbf{G}},&{\mathbf{E}}-{\mathbf{F}}\rangle _{F}+\frac{\rho '}{2}\Vert {\mathbf{E}}-{\mathbf{F}} \Vert _{F}^{2}, \end{aligned} \end{aligned}$$

(46)

where $\rho '$ is a scaling parameter for this ADMM. The ADMM algorithm consists of the following iterations.

$$\begin{aligned}&{\mathbf{E}}^{k+1} = \arg \min _{{\mathbf{E}}} g_{3}'({\mathbf{E}})+ \rho \Vert {\mathbf{E}}-{\mathbf{B}}\Vert _{F}^{2}+\frac{\rho '}{2}\Vert {\mathbf{E}}-{\mathbf{F}}^{k}+{\mathbf{G}}^{k} \Vert _{F}^{2}, \end{aligned}$$

(47)

$$\begin{aligned}&{\mathbf{F}}^{k+1} = \arg \min _{{\mathbf{F}}} g_{3}({\mathbf{F}})+ \frac{\rho '}{2}\Vert {\mathbf{F}}-{\mathbf{E}}^{k+1}-{\mathbf{G}}^{k}\Vert _{F}^{2}, \end{aligned}$$

(48)

$$\begin{aligned}&{\mathbf{G}}^{k+1} = {\mathbf{G}}^{k}+{\mathbf{E}}^{k+1} - {\mathbf{F}}^{k+1}. \end{aligned}$$

(49)

The problem of Eq. (47) is equivalent to:

$$\begin{aligned} \begin{aligned}&{\mathbf{E}}^{k+1} = \arg \min _{{\mathbf{E}}}g_{3}'({\mathbf{E}})+ \\&\frac{2\rho +\rho '}{2}\Vert {\mathbf{E}}-\left( \frac{2\rho }{2\rho +\rho '}{\mathbf{B}}+\frac{\rho '}{2\rho +\rho '}({\mathbf{F}}^{k}-{\mathbf{G}}^{k}) \right) \Vert _{F}^{2}. \end{aligned} \end{aligned}$$

(50)

Equation (50) indicates projecting $\big ( \frac{2\rho }{2\rho +\rho '}{\mathbf{B}}+\frac{\rho '}{2\rho +\rho '}({\mathbf{F}}^{k}- {\mathbf{G}}^{k}) \big )$ onto the set ${\mathcal {C}}'$. More specifically, we project the sub-matrices corresponding to ${\mathbf{W}}_{M}^{c,k}$ to be symmetric by the function $f({\mathbf{W}}):=\frac{1}{2}({\mathbf{W}}+{\mathbf{W}}^{\top })$. ${\mathbf{G}}^{k}$ is initialized as zero matrix and it stays in ${\mathcal {C}}'$ during the updating of Eq. (49), therefore, $({\mathbf{E}}^{k+1}+{\mathbf{G}}^{k})\in {\mathcal {C}}'$. Equation (48) indicates projecting $\big ( {\mathbf{E}}^{k+1}+ {\mathbf{G}}^{k}\big )$ onto set ${\mathcal {C}}$. It needs to project the corresponding sub-matrices from ${\mathbb {S}}^{d}$ to ${\mathbb {S}}^{d}_{-}$. The projection can be efficiently obtained by cropping the positive eigenvalues to be zero (Boyd and Vandenberghe 2004).

We operate $K=10$ iterations for the separated ADMM, and ${\mathbf{U}}_{3}^{l+1}$ is obtained by ${\mathbf{F}}^{K}$.