Visual Tracking via Subspace Learning: A Discriminative Approach

Abstract

Good tracking performance is in general attributed to accurate representation over previously obtained targets and/or reliable discrimination between the target and the surrounding background. In this work, a robust tracker is proposed by integrating the advantages of both approaches. A subspace is constructed to represent the target and the neighboring background, and their class labels are propagated simultaneously via the learned subspace. In addition, a novel criterion is proposed, by taking account of both the reliability of discrimination and the accuracy of representation, to identify the target from numerous target candidates in each frame. Thus, the ambiguity in the class labels of neighboring background samples, which influences the reliability of the discriminative tracking model, is effectively alleviated, while the training set still remains small. Extensive experiments demonstrate that the proposed approach outperforms most state-of-the-art trackers.

Appendices

Appendix A: Derivation of the Iterative Algorithm

This section presents the detailed solutions of all variables in Eq. (9) and the deviation of the iterative algorithm to solve the problem of the discriminative low-rank learning.

Solving ${\mathbf {A}}$

By fixing other variables, minimizing the IALM function $\mathcal {L}\left( {\mathbf {A}},{\mathbf {E}},{\mathbf {M}},{\mathbf {w}},{\mathbf {v}},b\right)$ with respect to ${\mathbf {A}}$ is equivalent to

$$\begin{aligned} \min _{\mathbf {A}}\frac{1}{2}\left\| {\mathbf {A}}-\frac{1}{2}\left( {\mathbf {X}}-{\mathbf {E}}+{\mathbf {M}}\!+\!\frac{1}{\tau }\left( {\mathbf {J}}_1-{\mathbf {J}}_3\right) \right) \right\| _F^2 \!\!+\frac{1}{2\tau }\left\| {\mathbf {A}}\right\| _*, \end{aligned}$$

(20)

which is derived from completing the squares. This minimization can be solved by using the singular values thresholding method (Cai et al. 2010). Thus, ${\mathbf {A}}$ is found by

$$\begin{aligned} {\mathbf {A}}={\mathbf {U}}\varphi \left( {\mathbf {S}},\frac{1}{2\tau }\right) {\mathbf {V}}^T, \end{aligned}$$

(21)

where ${\mathbf {U}}{\mathbf {S}}{\mathbf {V}}^T=\frac{1}{2}\left( {\mathbf {X}}-{\mathbf {E}}+{\mathbf {M}}+\frac{1}{\tau }\left( {\mathbf {J}}_1-{\mathbf {J}}_3\right) \right)$ and

$$\begin{aligned} \varphi \left( x,\varepsilon \right) =sign\left( x\right) \max \left( 0,\left| x\right| -\varepsilon \right) \end{aligned}$$

(22)

denotes the shrinkage operator and independently applies to each entries of x.

Solving ${\mathbf {M}}$

Minimizing $\mathcal {L}\left( {\mathbf {A}},{\mathbf {E}},{\mathbf {M}},{\mathbf {w}},{\mathbf {v}},b\right)$ with respect to ${\mathbf {M}}$ is equivalent to

$$\begin{aligned} \min _{\mathbf {M}}\left\| {\mathbf {z}}^T-{\mathbf {w}}^T{\mathbf {M}}-b\mathbf {1}^T+\frac{1}{\tau }{\mathbf {J}}_2\right\| _2^2 +\left\| {\mathbf {A}}-{\mathbf {M}}+\frac{1}{\tau }{\mathbf {J}}_3\right\| _F^2, \end{aligned}$$

(23)

which is a least squares problem. This minimization has a closed-form solution. Thus, ${\mathbf {M}}$ is found by

$$\begin{aligned} {\mathbf {M}}=\left( {\mathbf {w}}{\mathbf {w}}^T\!+\,\!\mathbf {I}\right) ^{-1}\left( {\mathbf {w}}\left( {\mathbf {z}}^T-b\mathbf {1}^T+\frac{1}{\tau }{\mathbf {J}}_2\right) \!+\!{\mathbf {A}}+\frac{1}{\tau }{\mathbf {J}}_3\right) . \end{aligned}$$

(24)

Solving ${\mathbf {E}}$

Minimizing $\mathcal {L}\left( {\mathbf {A}},{\mathbf {E}},{\mathbf {M}},{\mathbf {w}},{\mathbf {v}},b\right)$ with respect to ${\mathbf {E}}$ is equivalent to

$$\begin{aligned} \min _{\mathbf {E}}\frac{1}{2}\left\| {\mathbf {X}}-{\mathbf {A}}+\frac{1}{\tau }{\mathbf {J}}_1-{\mathbf {E}}\right\| _F^2 +\frac{\alpha }{\tau }\left\| {\mathbf {E}}\right\| _1, \end{aligned}$$

(25)

which is derived from completing the squares. This minimization can be solved by using the iterative shrinkage thresholding method (Beck and Teboulle 2009). Thus, ${\mathbf {E}}$ is found by

$$\begin{aligned} {\mathbf {E}}=\varphi \left( {\mathbf {X}}-{\mathbf {A}}+\frac{1}{\tau }{\mathbf {J}}_1,\frac{\alpha }{\tau }\right) . \end{aligned}$$

(26)

Solving ${\mathbf {w}}$ and b

Minimizing $\mathcal {L}\left( {\mathbf {A}},{\mathbf {E}},{\mathbf {M}},{\mathbf {w}},{\mathbf {v}},b\right)$ with respect to ${\mathbf {w}}$ and b are respectively equivalent to

$$\begin{aligned} \begin{aligned}&\min _{\mathbf {w}}\frac{1}{2}\left\| {\mathbf {z}}^T-{\mathbf {w}}^T{\mathbf {M}}-b\mathbf {1}^T+\frac{1}{\tau }{\mathbf {J}}_2\right\| _2^2 +\frac{1}{2}\left\| {\mathbf {v}}-{\mathbf {w}}+\frac{1}{\tau }{\mathbf {J}}_4\right\| _2^2 \\&\quad +\frac{\beta }{\tau }\left\| {\mathbf {w}}\right\| _2^2, \\&\min _b \left\| {\mathbf {z}}^T-{\mathbf {w}}^T{\mathbf {M}}-b\mathbf {1}^T+\frac{1}{\tau }{\mathbf {J}}_2\right\| _2^2, \end{aligned} \end{aligned}$$

(27)

both of which can be solved via least squares with the closed-form solutions

$$\begin{aligned} \begin{aligned} {\mathbf {w}}&=\left( {\mathbf {M}}{\mathbf {M}}^T+\left( 1+\frac{2\beta }{\tau }\right) \mathbf {I}\right) ^{-1} \\&\quad \cdot \left( {\mathbf {M}}\left( {\mathbf {z}}^T-b\mathbf {1}^T+\frac{1}{\tau }{\mathbf {J}}_2\right) ^T+{\mathbf {v}}+\frac{1}{\tau }{\mathbf {J}}_4\right) , \\&b=\frac{1}{N}\left\langle \mathbf {1}^T,{\mathbf {z}}^T-{\mathbf {w}}^T{\mathbf {M}}+\frac{1}{\tau }{\mathbf {J}}_2 \right\rangle , \end{aligned} \end{aligned}$$

(28)

where N denotes the number of the training samples.

Solving ${\mathbf {v}}$

Minimizing $\mathcal {L}\left( {\mathbf {A}},{\mathbf {E}},{\mathbf {M}},{\mathbf {w}},{\mathbf {v}},b\right)$ with respect to ${\mathbf {v}}$ is equivalent to

$$\begin{aligned} \min _{\mathbf {v}}\frac{1}{2}\left\| {\mathbf {v}}-{\mathbf {w}}+\frac{1}{\tau }{\mathbf {J}}_4\right\| _2^2+\frac{\gamma }{\tau }\left\| {\mathbf {v}}\right\| _1, \end{aligned}$$

(29)

which is derived from completing the squares. This minimization can be solved by the iterative shrinkage thresholding method (Beck and Teboulle 2009). Thus, ${\mathbf {v}}$ is found by

$$\begin{aligned} {\mathbf {v}}=\varphi \left( {\mathbf {w}}-\frac{1}{\tau }{\mathbf {J}}_4,\frac{\gamma }{\tau }\right) . \end{aligned}$$

(30)

The main steps of the iterative algorithm are depicted in Algorithm 2. The algorithm stops when the values of the IALM function $\mathcal {L}\left( {\mathbf {A}},{\mathbf {E}},{\mathbf {M}},{\mathbf {w}},{\mathbf {v}},b\right)$ between two consecutive iterations have small difference. Note that we set the parameters $\alpha =\frac{1}{\sqrt{\max \left( d,N\right) }}$, $\tau =\frac{1.25}{\max \left( svd\left( {\mathbf {X}}\right) \right) }$ and $\kappa =1.6$ following the recommendations in Lin et al. (2010), and empirically set $\beta =1-\alpha$ and $\gamma =\alpha$ in Algorithm 2.

Appendix B: Evaluations on Different Situations

Fig. 21

Performance of our tracker and the top ten ranking trackers in Wu et al. (2013) in different challenging cases. In the caption of each sub-figure, the number in parentheses denotes the number of the video sequences in the corresponding case

For more thorough evaluation of our tracker, we also analyze the performance of our tracker in different challenging situations, such as illumination variation and occlusion. The evaluation results in representative situations are reported as follows, as shown in Fig. 21.

Occlusion In the situation of occlusion, the target is occluded by other objects. Occlusion may easily lead to tracking failure because the target disappears partially or entirely for a period. From the results shown in Fig. 21a, it can be seen that our tracker is robust against occlusion and obtains good tracking results. It benefits from the facts that (1) the sparse reconstruction errors can absorb the occlusion during our subspace learning, such that the learned subspace only acquires the non-occluded information of the target; and (2) the good discriminative capability of the learned subspace can reliably separate the target from the background. The competing trackers using sparse reconstruction errors for occlusion handling, such as SCM and LSK, and the competing trackers using discriminative tracking model, such as Struck, also achieve good tracking results on some video sequences in this case.

Non-Rigid Deformation The motion of the target may cause non-rigid deformations in the appearance. From the results shown in Fig. 21b, it is evident that our tracker obtains supreme performance in this case. This is attributed to the facts that (1) the small deformation, which causes small reconstruction errors, is effectively processed by the subspace learning; and (2) the large deformation, which causes large reconstruction errors, is compensated by using the sparsity constraint on the reconstruction errors.

Illumination Variation In this case, the illumination of the scene changes drastically, leading to significant changes in the appearance of the target. From the results shown in Fig. 21c, it can be seen that our tracker obtains the most excellent results in this case. This is attributed to that the subspace learning is effective to handle illumination change. Note that the adaptive dimension reduction of our subspace learning also makes our tracker more stable in this case. It can also be seen that some subspace learning based trackers, such as LLR and SSL, also obtain good tracking performance in this case.

Background Clutter In this situation, the tracker is distracted by the cluttered background. Thus, the trackers that consider the difference between the target and the background information may be more effective in this case. From the results shown in Fig. 21d, it can be seen that our tracker performs favorably in this case. This is attributed to the good discriminative capability of our tracker, which can reliably distinguish the target from the background. As analyzed above, the competing trackers that considers the background, such as SET, also obtain good tracking results in this case.

Out-of-Plane Rotation The motion of either the target or the camera may cause out-of-plane rotations in the appearance of the target. From the results shown in Fig. 21e, it is evident that our tracker performs the best in this case. On one hand, the temporal locality of our subspace (only using the recently obtained targets) is effective to describe the appearance changes of the target with out-of-plane rotations. On the other hand, the linear classifier can successfully separate the target with out-of-plane rotations from the background.

Scale Variation In this case, the scale of the appearance of the target on successive frames varies over time, such that the tracker may result in inaccurate tracking results. Because we take account of the scale change of the target in the motion state, as shown in Eq. (10), it can be seen from the results shown in Fig. 21f that our tracker is insensitive to scale change and obtains favourable performance in this case.