Inter-frame constrained coding based on superpixel for tracking

Abstract

A video tracking method based on superpixel with inter-frame constrained coding is proposed in this paper. The 3-D CIE Lab color feature is extracted at each superpixel to characterize local image information. Based on the color feature, the superpixel-based coding model is achieved between the adjacent frames for correctly tracking object. The proposed tracking method considers the interaction of corresponding superpixels between the adjacent frames of complex scenes, which enhances the stability of encoding. Due to the update of codebook and classifier parameter, the proposed method is robust for long-term object tracking. We test the proposed method on 15 challenging sequences involving drastic illumination change, partial or full occlusion, and large pose variation. The proposed method shows excellent performance in comparison with eight previously proposed trackers.

Appendix A

In this appendix, we provide the derivation of Eq. (1).

$$\begin{aligned} \begin{array}{l} \mathop {\min }\limits _{\mathrm{{\mathbf {c}}}_i^t \in \mathrm{{\mathbf {C}}}} \displaystyle \sum \limits _{i=1}^N {\left\| {\mathrm{{\mathbf {x}}}_i^t -\mathrm{{\mathbf {B}}}^{t-1}\mathrm{{\mathbf {c}}}_i^t } \right\| } ^2+\lambda \left\| {\mathrm{{\mathbf {c}}}_i^{t-1} -\mathrm{{\mathbf {c}}}_i^t } \right\| ^2 \\ \mathrm{s.t.}\;\;\mathrm{{\mathbf {1}}}^T\mathrm{{\mathbf {c}}}_i^t =1,\;\forall i \\ \end{array} \end{aligned}$$

Given the sample feature $\mathrm{{\mathbf {x}}}_i^t$, according to Eq. (2), we can estimate the $\mathrm{{\mathbf {c}}}_i^{t-1}$. Furthermore, for the objective function, the two terms, $\mathrm{{\mathbf {B}}}^{t-1}$ and $\mathrm{{\mathbf {c}}}_i^{t-1}$, are constants respectively for each frame, so the $\mathrm{{\mathbf {c}}}_i^t$ can be achieved by solving a convex optimization problem.

Let $J( \mathrm{{\mathbf {C}}})=\sum \nolimits _{i=1}^N {\left\| {\mathrm{{\mathbf {x}}}_i^t -\mathrm{{\mathbf {B}}}^{t-1}\mathrm{{\mathbf {c}}}_i^t } \right\| } ^2+\lambda \left\| {\mathrm{{\mathbf {c}}}_i^{t-1} -\mathrm{{\mathbf {c}}}_i^t } \right\| ^2$

Introduce a Lagrange multiplier function:

$$\begin{aligned} J( {\mathrm{{\mathbf {C}}},\eta })&= \sum \limits _{i=1}^N {\left\| {\mathrm{{\mathbf {x}}}_i^t \mathrm{{\mathbf {1}}}^T\mathrm{{\mathbf {c}}}_i^t -\mathrm{{\mathbf {B}}}^{t-1}\mathrm{{\mathbf {c}}}_i^t } \right\| } ^2\\&+\lambda \left\| {\mathrm{{\mathbf {c}}}_i^{t-1} \mathrm{{\mathbf {1}}}^T\mathrm{{\mathbf {c}}}_i^t -\mathrm{{\mathbf {c}}}_i^t } \right\| ^2+2\eta ( {1-\mathrm{{\mathbf {1}}}^T\mathrm{{\mathbf {c}}}_i^t }) \end{aligned}$$

Let $\partial J\left( {{\mathbf {C}},\eta } \right) /\partial {\mathbf {c}}_{i}^{t} = 0 .$

$$\begin{aligned}&\left[ \left( {{\mathbf {B}}^{{t - 1^{T} }} - {\mathbf {1x}}_{i}^{{t^{T} }} } \right) \left( {{\mathbf {B}}^{{t - 1^{T} }} - {\mathbf {1x}}_{i}^{{t^{T} }} } \right) ^{T}\right. \\&\quad \left. +\, \lambda \left( {{\mathbf {c}}_{i}^{{t - 1}} {\mathbf {1}}^{T} - {\mathbf {E}}} \right) ^{T} \left( {{\mathbf {c}}_{i}^{{t - 1}} {\mathbf {1}}^{T} - {\mathbf {E}}} \right) \right] {\mathbf {c}}_{i}^{t} = \eta {\mathbf {1}} \end{aligned}$$

where $\mathrm{{\mathbf {E}}}$ is identity matrix.

$$\begin{aligned}&\widehat{{{\mathbf {c}}_{i}^{t} }} = \eta \left[ \left( {{\mathbf {B}}^{{t - 1^{T} }} - {\mathbf {1x}}_{i}^{{t^{T} }} } \right) \left( {{\mathbf {B}}^{{t - 1^{T} }} - {\mathbf {1x}}_{i}^{{t^{T} }} } \right) ^{T}\right. \nonumber \\&\qquad \qquad \quad \left. +\, \lambda \left( {{\mathbf {c}}_{i}^{{t - 1}} {\mathbf {1}}^{T} - {\mathbf {E}}} \right) ^{T} \left( {{\mathbf {c}}_{i}^{{t - 1}} {\mathbf {1}}^{T} - {\mathbf {E}}} \right) \right] /{\mathbf {1}} \end{aligned}$$

(9)

Because $\mathrm{{\mathbf {1}}}^T\mathrm{{\mathbf {c}}}_i^t =1$, we normalize the $\widehat{\mathrm{{\mathbf {c}}}_i^t }$,

$$\begin{aligned} {\mathbf {c}}_{i}^{t} = \widehat{{{\mathbf {c}}_{i}^{t} }}/\left( {{\mathbf {1}}^{T} \widehat{{{\mathbf {c}}_{i}^{t} }}} \right) \end{aligned}$$

(10)

Furthermore, for achieving the normalizing result of $\mathrm{{\mathbf {c}}}_i^t$, the $\eta$ should be:

$$\begin{aligned} \eta \!=\! \frac{1}{{{\mathbf {1}}^{T} \left[ {\left( {{\mathbf {B}}^{{t \!-\! 1^{T} }} \!-\! {\mathbf {1x}}_{i}^{{t^{T} }} } \right) \left( {{\mathbf {B}}^{{t \!-\! 1^{T} }} \!-\! {\mathbf {1x}}_{i}^{{t^{T} }} } \right) ^{T} \!+\! \lambda \left( {{\mathbf {c}}_{i}^{{t \!-\! 1}} {\mathbf {1}}^{T} \!-\! {\mathbf {E}}} \right) ^{T} \left( {{\mathbf {c}}_{i}^{{t \!-\! 1}} {\mathbf {1}}^{T} \!-\! {\mathbf {E}}} \right) } \right] ^{{ \!-\! 1}} {\mathbf {1}}}} \end{aligned}$$