Robust object tracking via online Principal Component–Canonical Correlation Analysis (P3CA)

Abstract

Effective object representation plays a significant role in object tracking. To fulfill the requirements of tracking robustness and effectiveness, in this paper, we propose an adaptive appearance model called Principal Component–Canonical Correlation Analysis (P3CA). P3CA is a compact association of principal component analysis (PCA) and canonical correlation analysis (CCA), which results in robust tracking along with low computation cost. CCA is incorporated into P3CA appearance model for its effectiveness in handling occlusion due to the introduction of canonical correlation score instead of holistic information to evaluate the target goodness. However, it is time consuming and often suffers from Small Sample Size (3S) problem. To address these issues, PCA is incorporated and we obtain our P3CA subspace by performing CCA on the low-dimensional data gained by projecting the high-dimensional observations to the PCA subspaces. In addition, to account for appearance variations, we propose a novel online updating algorithm for P3CA subspace, which updates the PCA and CCA subspaces cooperatively and synchronously. Finally, we incorporate the dynamic P3CA appearance model with the particle filter framework in a probabilistic manner and select the candidate object with the largest weight as the final tracking result. Comparative results on several challenging sequences demonstrate that our tracker performs better than a number of state-of-the-art methods proposed recently in handling partial occlusion and various appearance variations.

Appendix: Proofs of Eqs. (11)–(14)

According to the definition of the covariance matrix, Eq. (11) can be obtained as:

$$\begin{aligned} {\varvec{\Sigma }}_{xy}^\mathrm{new}&= \frac{1}{(t+m)}(\mathbf{x}-\mathbf{m}_x^\mathrm{new})(\mathbf{y}-\mathbf{m}_y^\mathrm{new})^T\nonumber \\&= \frac{1}{(t+m)}\sum _{i=1}^{t+m}(\mathbf{x}_i-\mathbf{m}_x^\mathrm{new})(\mathbf{y}_i-\mathbf{m}_y^\mathrm{new})^T\nonumber \\&= \frac{1}{(t+m)}\sum _{i=1}^{t}\left(\mathbf{x}_i-\mathbf{m}_x^{\prime }\right)\left(\mathbf{y}_i-\mathbf{m}_y^{\prime }\right)^T\nonumber \\&+\frac{1}{(t+m)}\sum _{i=t+1}^{t+m}\left(\mathbf{x}_i-\mathbf{m}_x^{\prime }\right)\left(\mathbf{y}_i-\mathbf{m}_y^{\prime }\right)^T\nonumber \\&+\frac{m}{(t+m)^2}\sum _{i=1}^{t+m}\left(\mathbf{x}_i-\mathbf{m}_x^{\prime }\right)(\mathbf{m}_y^{\prime }-\mathbf{m}_y^{\prime \prime })^T\nonumber \\&+\frac{m}{(t+m)^2}(\mathbf{m}_x^{\prime }-\mathbf{m}_x^{\prime \prime })\sum _{i=1}^{t+m}\left(\mathbf{y}_i-\mathbf{m}_y^{\prime }\right)^T\nonumber \\&+\frac{m^2}{(t+m)^3}\sum _{i=1}^{t+m}(\mathbf{m}_x^{\prime }-\mathbf{m}_x^{\prime \prime })(\mathbf{m}_y^{\prime }-\mathbf{m}_y^{\prime \prime })^T \end{aligned}$$

(15)

The second term in (15) can be figured out by (16)

$$\begin{aligned}&{\sum _{i=t+1}^{t+m}\left(\mathbf{x}_i-\mathbf{m}_x^{\prime }\right)\left(\mathbf{y}_i-\mathbf{m}_y^{\prime }\right)^T}\nonumber \\&\quad =m\left({\varvec{\Sigma }}_{xy}^{\prime \prime }+\mathbf{m}_x^{\prime \prime }\mathbf{m}_y^{^{\prime \prime }T}\right)-m\mathbf{m}_x^{\prime \prime }\mathbf{m}_y^{^{\prime }T}-m\mathbf{m}_x^{\prime }\mathbf{m}_y^{^{\prime \prime }T}\nonumber \\&\quad + m\mathbf{m}_x^{\prime }\mathbf{m}_y^{^{\prime }T}\nonumber \\&\quad =m\left({\varvec{\Sigma }}_{xy}^{\prime \prime }+(\mathbf{m}_x^{\prime }-\mathbf{m}_x^{\prime \prime })(\mathbf{m}_y^{\prime }-\mathbf{m}_y^{\prime \prime })^T\right) \end{aligned}$$

(16)

The third term in (15) can be figured out by (17)

$$\begin{aligned}&{\sum _{i=1}^{t+m}\left(\mathbf{x}_i-\mathbf{m}_x^{\prime } \right)(\mathbf{m}_y^{\prime }-\mathbf{m}_y^{\prime \prime })^T}\nonumber \\&\quad =\left(\sum _{i=1}^{t}\mathbf{x}_i+\sum _{i=t+1}^{t+m} \mathbf{x}_i-\sum _{i=1}^{t+m}\mathbf{m}_x^{\prime }\right) (\mathbf{m}_y^{\prime }-\mathbf{m}_y^{\prime \prime })^T\nonumber \\&\quad =-m(\mathbf{m}_x^{\prime }-\mathbf{m}_x^{\prime \prime })(\mathbf{m}_y^{\prime } -\mathbf{m}_y^{\prime \prime })^T \end{aligned}$$

(17)

Similar as (17), the fourth term in (15) can be gained by (18)

$$\begin{aligned}&{(\mathbf{m}_x^{\prime }-\mathbf{m}_x^{\prime \prime })\sum _{i=1}^{t+m} \left(\mathbf{y}_i-\mathbf{m}_y^{\prime }\right)^T}\nonumber \\&\quad =-m(\mathbf{m}_x^{\prime }-\mathbf{m}_x^{\prime \prime })(\mathbf{m}_y^{\prime } -\mathbf{m}_y^{\prime \prime })^T \end{aligned}$$

(18)

Finally, (15)can be simplified using (16)–(18) as (19):

$$\begin{aligned} {\varvec{\Sigma }}_{xy}^\mathrm{new}&= \frac{t}{(t+m)}{\varvec{\Sigma }}_{xy}^{\prime } +\frac{m}{(t+m)}{\varvec{\Sigma }}_{xy}^{\prime \prime }\nonumber \\&+\frac{tm}{(t+m)^2}(\mathbf{m}_x^{\prime }-\mathbf{m}_x^{\prime \prime }) (\mathbf{m}_y^{\prime }-\mathbf{m}_y^{\prime \prime })^T \end{aligned}$$

(19)

In order to get the updated inverse intra-class matrix $({\varvec{\Sigma }}_{xx}^{-1})^\mathrm{new}$ and $({\varvec{\Sigma }}_{yy}^{-1})^\mathrm{new}$ in Eq. (12), we need obtain ${\varvec{\Sigma }}_{xx}^\mathrm{new}$ and ${\varvec{\Sigma }}_{yy}^\mathrm{new}$ firstly, which can obtained using the same method as shown in (15)–(19) and simply by replacing $\mathbf{y}$ with $\mathbf{x}$ and $\mathbf{x}$ with $\mathbf{y}$, respectively. Therefore ${\varvec{\Sigma }}_{xx}^\mathrm{new}$ and ${\varvec{\Sigma }}_{yy}^\mathrm{new}$ can be expressed as (20) and (21):

$$\begin{aligned} {\varvec{\Sigma }}_{xx}^\mathrm{new}&= \frac{t}{(t+m)}{\varvec{\Sigma }}_{xx}^{\prime } +\frac{m}{(t+m)}{\varvec{\Sigma }}_{xx}^{\prime \prime }\nonumber \\&+\frac{tm}{(t+m)^2}(\mathbf{m}_x^{\prime }-\mathbf{m}_x^{\prime \prime }) (\mathbf{m}_x^{\prime }-\mathbf{m}_x^{\prime \prime })^T \end{aligned}$$

(20)

$$\begin{aligned} {\varvec{\Sigma }}_{yy}^\mathrm{new}&= \frac{t}{(t+m)}{\varvec{\Sigma }}_{yy}^{\prime } +\frac{m}{(t+m)}{\varvec{\Sigma }}_{yy}^{\prime \prime }\nonumber \\&+\frac{tm}{(t+m)^2}(\mathbf{m}_y^{\prime }-\mathbf{m}_y^{\prime \prime }) (\mathbf{m}_y^{\prime }-\mathbf{m}_y^{\prime \prime })^T \end{aligned}$$

(21)

Since we have obtained (20), then we can obtain $({\varvec{\Sigma }}_{xx}^{-1})^\mathrm{new}$ in Eq. (12) using Sherman-Morrison formula, $(A+uv^T)^{-1}$ $=A^{-1}-(A^{-1}uv^TA^{-1})/(1+v^TA^{-1}u)$, by letting $A=\mathbf{D}_x=t/(t+m){\varvec{\Sigma }}_{xx}^{\prime }+m/(t+m){\varvec{\Sigma }}_{xx}^{\prime \prime }$ and $u=v=a=\sqrt{mt}/(t+m)(\mathbf{m}_x^{\prime }-\mathbf{m}_x^{\prime \prime })$. $({\varvec{\Sigma }}_{yy}^{-1})^\mathrm{new}$ in Eq. (12) can be obtained using the same method.