Robust Tracking via Locally Structured Representation

Abstract

Representation method is critical to visual tracking. A robust representation describes the target accurately, leading to good tracking performance. In this work, a novel representation is proposed, which is designed to be simultaneously low-rank and joint sparse for the local patches within a target region. In this representation, the subspace structure is exploited by the low-rank constraint to reflect the global information of all the patches, and the sparsity structure is captured by the joint sparsity restriction to describe the locally intimate relationship between the neighboring patches. Importantly, to make the representation computationally applicable to visual tracking, a novel fast algorithm based on greedy strategy is proposed, and the performance analysis of this algorithm is also presented. Thus, the tracking in this work is formulated as a locally low-rank and joint sparse matching problem within particle filtering framework. A large number of experimental results show that the tracking drift problem is effectively alleviated in various challenging situations by using the proposed representation method. Both qualitative and quantitative evaluations demonstrate that the proposed tracker performs favorably against many other state-of-the-art trackers. Benefitting from the good adaptive capability of the representation, all the parameters of the proposed tracking algorithm are fixed in all the experiments.

Appendices

Appendix 1: Proof of the Theorems

Theorem 1

(orthogonality) Algorithm 1 ensures that the active columns of the dictionary ${\mathbf {D}}$ will not be chosen again for the support in the next iteration and afterwards.

Proof

At the i-th iteration, the term $\left\| {\mathbf {D}}{\mathbf {Z}}-{\mathbf {X}}\right\| _F^2$ is minimized with respect to ${\mathbf {Z}}$ in step 5, such that the support of each column of ${\mathbf {Z}}$ is in $\mathcal {S}^i$. Let ${\mathbf {D}}_{\mathcal {S}^i}$ denote the matrix with size $d\times |\mathcal {S}^i|$ that contains the columns from ${\mathbf {D}}$ belonging to this support. Thus, to minimize $\left\| {\mathbf {D}}{\mathbf {Z}}-{\mathbf {X}}\right\| _F^2$ is equivalent to minimize $\left\| {\mathbf {D}}_{\mathcal {S}^i}{\mathbf {Z}}_{\mathcal {S}^i}-{\mathbf {X}}\right\| _F^2$, where ${\mathbf {Z}}_{\mathcal {S}^i}$ consists of the non-zero rows of the matrix ${\mathbf {Z}}$. The solution is given by zeroing the derivative of the quadratic form

$$\begin{aligned} {\mathbf {D}}_{\mathcal {S}^i}^T\left( {\mathbf {D}}_{\mathcal {S}^i}{\mathbf {Z}}_{\mathcal {S}^i}-{\mathbf {X}}\right) =-{\mathbf {D}}_{\mathcal {S}^i}^T{\mathbf {R}}^{(i)}=\mathbf {0}. \end{aligned}$$

Here the following formula of the residual is used, i.e., ${\mathbf {R}}^{(i)}={\mathbf {X}}-{\mathbf {D}}{\mathbf {Z}}^{(i)}={\mathbf {X}}-{\mathbf {D}}_{\mathcal {S}^i}{\mathbf {Z}}_{\mathcal {S}^i}$. Thus, the above relation indicates that the columns in ${\mathbf {D}}$ that are part of the support $\mathcal {S}^i$ are necessarily orthogonal to the residual ${\mathbf {R}}^{(i)}$. It implies that in the next iteration and afterwards, these columns will not be chosen again as the active columns. $\square$

Theorem 2

(normalization invariance) Algorithm 1 produces the same solution when using either the original dictionary ${\mathbf {D}}$ or its normalized version $\widetilde{{\mathbf {D}}}$.

Proof

At the i-th iteration, the errors in step 3 are computed from

$$\begin{aligned} \epsilon \left( j\right)= & {} \min _{\tilde{{\mathbf {z}}}_j}\left\| {\mathbf {d}}_j\tilde{{\mathbf {z}}}_j-{\mathbf {R}}^{(i-1)}\right\| _F^2 \\= & {} \left\| {\mathbf {d}}_j\frac{{\mathbf {d}}_j^T{\mathbf {R}}^{(i-1)}}{\left\| {\mathbf {d}}_j\right\| _2^2}-{\mathbf {R}}^{(i-1)}\right\| _F^2 \\= & {} \left\| {\mathbf {R}}^{(i-1)}\right\| _F^2-2\left\langle {\mathbf {d}}_j\frac{{\mathbf {d}}_j^T{\mathbf {R}}^{(i-1)}}{\left\| {\mathbf {d}}_j\right\| _2^2}, {\mathbf {R}}^{(i-1)} \right\rangle \\&+\left\| {\mathbf {d}}_j\frac{{\mathbf {d}}_j^T{\mathbf {R}}^{(i-1)}}{\left\| {\mathbf {d}}_j\right\| _2^2}\right\| _F^2 \\= & {} \left\| {\mathbf {R}}^{(i-1)}\right\| _F^2-\frac{\left\| {\mathbf {d}}_j^T{\mathbf {R}}^{(i-1)}\right\| _2^2}{\left\| {\mathbf {d}}_j\right\| _2^2}. \end{aligned}$$

Denoting $\widetilde{{\mathbf {d}}}_j={\mathbf {d}}_j/\left\| {\mathbf {d}}_j\right\| _2$, it is clearly to have $\left\| \widetilde{{\mathbf {d}}}_j\right\| _2=1$. Thus, the above formula is equivalent to

$$\begin{aligned} \epsilon \left( j\right)= & {} \left\| {\mathbf {R}}^{(i-1)}\right\| _F^2-\frac{\left\| {\mathbf {d}}_j^T{\mathbf {R}}^{(i-1)}\right\| _2^2}{\left\| {\mathbf {d}}_j\right\| _2^2} \\= & {} \left\| {\mathbf {R}}^{(i-1)}\right\| _F^2-\left\| \frac{{\mathbf {d}}_j^T{\mathbf {R}}^{(i-1)}}{\left\| {\mathbf {d}}_j\right\| _2}\right\| _2^2 \\= & {} \left\| {\mathbf {R}}^{(i-1)}\right\| _F^2-\left\| \widetilde{{\mathbf {d}}}_j^T{\mathbf {R}}^{(i-1)}\right\| _2^2 \\= & {} \left\| {\mathbf {R}}^{(i-1)}\right\| _F^2-\left\| \frac{\widetilde{{\mathbf {d}}}_j^T{\mathbf {R}}^{(i-1)}}{\left\| \widetilde{{\mathbf {d}}}_j\right\| _2}\right\| _2^2. \end{aligned}$$

Thus, it can be seen that using the normalized columns leads to the same choice for the index $j_0$ in step 4.

Then, let $\widetilde{{\mathbf {D}}}={\mathbf {D}}{\mathbf {W}}$ denotes the normalized version of ${\mathbf {D}}$, where ${\mathbf {W}}$ is a diagonal matrix containing $\frac{1}{\left\| {\mathbf {d}}_j\right\| _2}$ on the main-diagonal. In step 5, the Least Squares is used to minimize $\left\| {\mathbf {D}}{\mathbf {Z}}-{\mathbf {X}}\right\| _F^2$, subject to the support of each column $support\left( {\mathbf {z}}_k\right) \subseteq \mathcal {S}^{(i)}$. Let ${\mathbf {D}}_{\mathcal {S}}$ denote the sub-matrix of ${\mathbf {D}}$ that contains the columns in $\mathcal {S}$. The solution of the above minimization is found by

$$\begin{aligned} {\mathbf {Z}}_{\mathcal {S}}^{(i)}=\left( {\mathbf {D}}_{\mathcal {S}}^T {\mathbf {D}}_{\mathcal {S}}\right) ^{-1}{\mathbf {D}}_{\mathcal {S}}^T{\mathbf {X}}, \end{aligned}$$

and the residual is

$$\begin{aligned} {\mathbf {R}}^{(i)}={\mathbf {X}}-{\mathbf {D}}_{\mathcal {S}}{\mathbf {Z}}_{\mathcal {S}}^{(i)} =\left[ {\mathbf {I}}-{\mathbf {D}}_{\mathcal {S}}\left( {\mathbf {D}}_{\mathcal {S}}^T {\mathbf {D}}_{\mathcal {S}}\right) ^{-1}\right] {\mathbf {X}}. \end{aligned}$$

Let $\widetilde{{\mathbf {D}}}_{\mathcal {S}}={\mathbf {D}}_{\mathcal {S}}{\mathbf {W}}_{\mathcal {S}}$ denote the sub-matrix of the normalized matrix $\widetilde{{\mathbf {D}}}$, where ${\mathbf {W}}_{\mathcal {S}}$ denotes the sub-matrix of ${\mathbf {W}}$, whose columns are contained in $\mathcal {S}$. The residual in the i-th iteration obtained by Algorithm 1 that uses this normalized matrix is

$$\begin{aligned} \widetilde{{\mathbf {R}}}^{(i)}= & {} \left[ {\mathbf {I}}-\widetilde{{\mathbf {D}}}_{\mathcal {S}}\left( \widetilde{{\mathbf {D}}}_{\mathcal {S}}^T\widetilde{{\mathbf {D}}}_{\mathcal {S}}\right) ^{-1}\widetilde{{\mathbf {D}}}_{\mathcal {S}}^T\right] {\mathbf {X}}\\= & {} \left[ {\mathbf {I}}-{\mathbf {D}}_{\mathcal {S}}{\mathbf {W}}_{\mathcal {S}}\left( {\mathbf {W}}_{\mathcal {S}}{\mathbf {D}}_{\mathcal {S}}^T{\mathbf {D}}_{\mathcal {S}}{\mathbf {W}}_{\mathcal {S}}\right) ^{-1}{\mathbf {W}}_{\mathcal {S}}{\mathbf {D}}_{\mathcal {S}}^T\right] {\mathbf {X}}\\= & {} \left[ {\mathbf {I}}-{\mathbf {D}}_{\mathcal {S}}{\mathbf {W}}_{\mathcal {S}}{\mathbf {W}}_{\mathcal {S}}^{-1}\left( {\mathbf {D}}_{\mathcal {S}}^T{\mathbf {D}}_{\mathcal {S}}\right) ^{-1}{\mathbf {W}}_{\mathcal {S}}^{-1}{\mathbf {W}}_{\mathcal {S}}{\mathbf {D}}_{\mathcal {S}}^T\right] {\mathbf {X}}\\= & {} \left[ {\mathbf {I}}-{\mathbf {D}}_{\mathcal {S}}\left( {\mathbf {D}}_{\mathcal {S}}^T{\mathbf {D}}_{\mathcal {S}}\right) ^{-1}{\mathbf {D}}_{\mathcal {S}}^T\right] {\mathbf {X}}\\= & {} {\mathbf {R}}^{(i)}. \end{aligned}$$

It can be seen from the above formula that the residual is the same for either the original or the normalized matrix ${\mathbf {D}}$. Thus, as the residual drives step 3 in the next iteration, Algorithm 1 is indifferent to the normalization. $\square$

Theorem 3

(performance in the worst case) For a system of linear equations ${\mathbf {D}}{\mathbf {Z}}={\mathbf {X}}$, if a solution ${\mathbf {Z}}$ exists obeying

$$\begin{aligned} \left\| {\mathbf {Z}}\right\| _0, rank\left( {\mathbf {Z}}\right) <\frac{1}{2} \left( 1+\frac{\left| z_{min}\right| }{\mu _{{\mathbf {D}}}\left| z_{max}\right| }\right) , \end{aligned}$$

where $\mu _{{\mathbf {D}}}=\max _{1\le i,j\le m, i\ne j}\frac{\left| {\mathbf {d}}_i^T{\mathbf {d}}_j\right| }{\left\| {\mathbf {d}}_i\right\| _2\left\| {\mathbf {d}}_j\right\| _2}$, and $\left| z_{min}\right|$ and $\left| z_{max}\right|$ are respectively the minimal and maximal non-zero absolute values of ${\mathbf {Z}}$, Algorithm 2 is guaranteed to find it exactly.

Proof

Success of Algorithm 2 is guaranteed by the following requirement

$$\begin{aligned} \min _{i\in {\mathbf {S}}}\left\| {\mathbf {d}}_i^T{\mathbf {X}}\right\| _1 > \max _{j\notin {\mathbf {S}}}\left\| {\mathbf {d}}_j^T{\mathbf {X}}\right\| _1, \end{aligned}$$

(22)

where ${\mathbf {S}}$ denotes the set that contains the indices of the $\left| {\mathbf {S}}\right|$ non-zero rows of ${\mathbf {Z}}$. Without loss of generality, each column of ${\mathbf {D}}$ is assumed to have the unit $\ell _2$-length, i.e., $\left\| {\mathbf {d}}_j\right\| _2=1$. Plugging ${\mathbf {x}}_k=\sum _{t\in {\mathbf {S}}}z_{t,k}{\mathbf {d}}_t$ for the k-th column of ${\mathbf {X}}$, the term in the left-hand-side in Eq. (22) is

$$\begin{aligned}&\min _{i\in {\mathbf {S}}}\left\| {\mathbf {d}}_i^T{\mathbf {X}}\right\| _1 \\&\quad min_{i\in {\mathbf {S}}}\sum _k\left| {\mathbf {d}}_i^T{\mathbf {x}}_k\right| \\&\quad =\min _{i\in {\mathbf {S}}}\sum _k\left| \sum _{t\in {\mathbf {S}}}z_{t,k}{\mathbf {d}}_i^T{\mathbf {d}}_t\right| \\&\quad =\min _{i\in {\mathbf {S}}}\sum _k\left| z_{i,k}+\sum _{t\in {\mathbf {S}},t\ne i}z_{t,k}{\mathbf {d}}_i^T{\mathbf {d}}_t\right| \\&\quad \ge \min _{i\in {\mathbf {S}}}\sum _k\left\{ \left| z_{i,k}\right| -\left| \sum _{t\in {\mathbf {S}},t\ne i}z_{t,k}{\mathbf {d}}_i^T{\mathbf {d}}_t\right| \right\} \\&\quad \ge \min _{i\in {\mathbf {S}}}\sum _k\left| z_{i,k}\right| -\max _{\i \in {\mathbf {S}}}\sum _k\left| \sum _{t\in {\mathbf {S}},t\ne i}z_{t,k}{\mathbf {d}}_i^T{\mathbf {d}}_t\right| \\&\quad \ge n\left| z_{min}\right| -n\left( \left| {\mathbf {S}}\right| -1\right) \mu _{{\mathbf {D}}}\left| z_{max}\right| , \end{aligned}$$

where n denotes the number of the columns of ${\mathbf {X}}$, $\left| z_{min}\right|$ and $\left| z_{max}\right|$ are the minimum and maximum non-zero absolute values of ${\mathbf {Z}}$, and $\mu _{{\mathbf {D}}}=\max _{1\le i,j\le m, i\ne j}\frac{\left| {\mathbf {d}}_i^T{\mathbf {d}}_j\right| }{\left\| {\mathbf {d}}_i\right\| _2\left\| {\mathbf {d}}_j\right\| _2}$, for ${\mathbf {D}}\in \mathbb {R}^{d\times m}$.

Then, the term in the right-hand-side in Eq. (22) is

$$\begin{aligned}&\max _{j\notin {\mathbf {S}}}\left\| {\mathbf {d}}_j^T{\mathbf {X}}\right\| _1 \\= & {} \max _{j\notin {\mathbf {S}}}\sum _k\left| {\mathbf {d}}_j^T{\mathbf {x}}_k\right| \\= & {} \max _{j\notin {\mathbf {S}}}\sum _k\left| \sum _{t\in {\mathbf {S}}}z_{t,k}{\mathbf {d}}_j^T{\mathbf {d}}_t\right| \\\le & {} n\sum _k\left| {\mathbf {S}}\right| \mu _{{\mathbf {D}}}\left| z_{max}\right| . \end{aligned}$$

Thus, requiring

$$\begin{aligned} n\left| z_{min}\right| -n\left( \left| {\mathbf {S}}\right| -1\right) \mu _{{\mathbf {D}}}\left| z_{max}\right| >n\left| {\mathbf {S}}\right| \mu _{{\mathbf {D}}}\left| z_{max}\right| \end{aligned}$$

necessarily leads to the satisfaction that guarantees the success of Algorithm 2, i.e.,

$$\begin{aligned} rank\left( {\mathbf {Z}}\right) \le \left| {\mathbf {S}}\right| <\frac{1}{2} \left( 1+\frac{\left| z_{min}\right| }{\mu _{{\mathbf {D}}}\left| z_{max}\right| }\right) . \end{aligned}$$

The condition on the sparsity of ${\mathbf {Z}}$, can be found by following the similar steps, i.e.,

$$\begin{aligned} \left\| {\mathbf {Z}}\right\| _0\le \left| {\mathbf {S}}\right| <\frac{1}{2} \left( 1+\frac{\left| z_{min}\right| }{\mu _{{\mathbf {D}}}\left| z_{max}\right| }\right) . \end{aligned}$$

Hence, the conclusion is obtained as shown in Theorem 3. $\square$

Theorem 4

(performance in average) For a system of linear equations ${\mathbf {D}}{\mathbf {Z}}={\mathbf {X}}$ (${\mathbf {D}}\in \mathbb {R}^{d\times m}$), if a solution ${\mathbf {Z}}$ exists obeying

$$\begin{aligned} \left\| {\mathbf {Z}}\right\| _0, rank\left( {\mathbf {Z}}\right) <\frac{1}{128} \left( \frac{z_{min}}{z_{max}}\right) ^2\cdot \frac{1}{\mu _{{\mathbf {D}}}^2\log d}, \end{aligned}$$

where $\mu _{{\mathbf {D}}}=\max _{1\le i,j\le m, i\ne j}\frac{\left| {\mathbf {d}}_i^T{\mathbf {d}}_j\right| }{\left\| {\mathbf {d}}_i\right\| _2\left\| {\mathbf {d}}_j\right\| _2}$, and $\left| z_{min}\right|$ and $\left| z_{max}\right|$ are respectively the minimal and maximal non-zero absolute values of ${\mathbf {Z}}$, Algorithm 2 is likely to find it exactly with probability of failure vanishing to zero for $d\rightarrow \infty$.

Proof

It is assumed that ${\mathbf {X}}$ is constructed by ${\mathbf {x}}_k={\mathbf {D}}{\mathbf {z}}_k=\sum _{t\in {\mathbf {S}}}z_{t,k}{\mathbf {d}}_t$, for the k-th column of ${\mathbf {X}}$, where the non-zeros of ${\mathbf {z}}_k$ are chosen as arbitrary positive values from $\left[ \left| z_{min}\right| ,\left| z_{max}\right| \right]$ multiplied by i.i.d. Rademacher random variables ${\epsilon }_t$ assuming values $\pm 1$ with equal probabilities. ${\mathbf {S}}$ contains the indices of the non-zero rows of ${\mathbf {Z}}$, such that $rank\left( {\mathbf {Z}}\right) \le \left| {\mathbf {S}}\right|$.

First, the probability of failure of Algorithm 2 is found by

$$\begin{aligned}&P\left( Failure\right) \nonumber \\= & {} P\left( \min _{i\in {\mathbf {S}}}\left\| {\mathbf {d}}_j^T{\mathbf {X}}\right\| _1\le \max _{j\notin {\mathbf {S}}}\left\| {\mathbf {d}}_j^T{\mathbf {X}}\right\| _1\right) \nonumber \\\le & {} P\left( \min _{i\in {\mathbf {S}}}\left\| {\mathbf {d}}_i^T{\mathbf {X}}\right\| _1\le T\right) +P\left( \max _{j\notin {\mathbf {S}}}\left\| {\mathbf {d}}_j^T{\mathbf {X}}\right\| _1\ge T\right) , \end{aligned}$$

(23)

where T is an arbitrary threshold value. Without loss of generality, each column of ${\mathbf {D}}$ is assumed to have the unit $\ell _2$-length, i.e., $\left\| {\mathbf {d}}_j\right\| _2=1$. Plugging ${\mathbf {x}}_k={\mathbf {D}}{\mathbf {z}}_k=\sum _{t\in {\mathbf {S}}}z_{t,k}{\mathbf {d}}_t$, we get

$$\begin{aligned}&\min _{i\in {\mathbf {S}}}\left\| {\mathbf {d}}_j^T{\mathbf {X}}\right\| _1 =\min _{i\in {\mathbf {S}}}\sum _k\left| {\mathbf {d}}_j^T{\mathbf {x}}_k\right| \\&=\min _{i\in {\mathbf {S}}}\sum _k\left| \sum _{t\in {\mathbf {S}}}z_{t,k}{\mathbf {d}}_i^T{\mathbf {d}}_t\right| \\&=\min _{i\in {\mathbf {S}}}\sum _k\left| z_{i,k}+\sum _{t\in {\mathbf {S}},t\ne i}z_{t,k}{\mathbf {d}}_i^T{\mathbf {d}}_t\right| \\&\ge \min _{i\in {\mathbf {S}}}\sum _k\left\{ \left| z_{i,k}\right| -\left| \sum _{t\in {\mathbf {S}},t\ne i}z_{t,k}{\mathbf {d}}_i^T{\mathbf {d}}_t\right| \right\} \\&\ge \sum _k\left\{ \left| z_{min}\right| -\max _{i\in {\mathbf {S}}}\left| \sum _{t\in {\mathbf {S}},t\ne i}z_{t,k}{\mathbf {d}}_i^T{\mathbf {d}}_t\right| \right\} \\&\ge \sum _k\left\{ \left| z_{min}\right| -\max _{i\in {\mathbf {S}}}\left| \sum _{t\in {\mathbf {S}},t\ne i}{\epsilon }_t\left| z_{t,k}\right| {\mathbf {d}}_i^T{\mathbf {d}}_t\right| \right\} \\&=n\left| z_{min}\right| -\sum _k\max _{i\in {\mathbf {S}}}\left| \sum _{t\in {\mathbf {S}},t\ne i}{\epsilon }_t\left| z_{t,k}\right| {\mathbf {d}}_i^T{\mathbf {d}}_t\right| , \end{aligned}$$

where n denotes the number of the columns of ${\mathbf {X}}$, $\left| z_{min}\right|$ is the minimum non-zero absolute value of ${\mathbf {Z}}$. Thus, the probability is found by

$$\begin{aligned}&P\left( \min _{i\in {\mathbf {S}}}\left\| {\mathbf {d}}_i^T{\mathbf {X}}\right\| _1\le T\right) \\&\quad \le P\left( n\left| z_{min}\right| -\sum _k\max _{i\in {\mathbf {S}}}\left| \sum _{t\in {\mathbf {S}},t\ne i}{\epsilon }_t\left| z_{t,k}\right| {\mathbf {d}}_i^T{\mathbf {d}}_t\right| \le T\right) \\&\quad = P\left( \sum _k\max _{i\in {\mathbf {S}}}\left| \sum _{t\in {\mathbf {S}},t\ne i}{\epsilon }_t\left| z_{t,k}\right| {\mathbf {d}}_i^T{\mathbf {d}}_t\right| \ge n\left| z_{min}\right| -T\right) \\&\quad \le \sum _k P\left( \max _{i\in {\mathbf {S}}}\left| \sum _{t\in {\mathbf {S}},t\ne i}{\epsilon }_t\left| z_{t,k}\right| {\mathbf {d}}_i^T{\mathbf {d}}_t\right| \ge \left| z_{min}\right| -\frac{T}{n}\right) \\&\quad \le \sum _k\sum _{i\in {\mathbf {S}}}P\left( \left| \sum _{t\in {\mathbf {S}},t\ne i}{\epsilon }_t\left| z_{t,k}\right| {\mathbf {d}}_i^T{\mathbf {d}}_t\right| \ge \left| z_{min}\right| -\frac{T}{n}\right) \\&\quad \le \sum _k2\sum _{i\in {\mathbf {S}}}\exp \left\{ -\frac{\left( \left| z_{min}\right| -\frac{T}{n}\right) ^2}{32\sum _{t\in {\mathbf {S}},t\ne i}\left| z_{t,k}\right| ^2\left( {\mathbf {d}}_i^T{\mathbf {d}}_t\right) ^2}\right\} \\&\quad \le 2n\left| {\mathbf {S}}\right| \exp \left\{ -\frac{\left( \left| z_{min}\right| -\frac{T}{n}\right) ^2}{32\left| z_{max}\right| ^2\mu _{\left( \left| {\mathbf {S}}\right| -1\right) }}\right\} \\&\quad \le 2n\left| {\mathbf {S}}\right| \exp \left\{ -\frac{\left( \left| z_{min}\right| -\frac{T}{n}\right) ^2}{32\left| z_{max}\right| ^2\mu _{\left| {\mathbf {S}}\right| }}\right\} , \end{aligned}$$

where $\mu _k=\max _{\left| {\mathbf {S}}\right| =k}\max _{i\notin {\mathbf {S}}}\sum _{t\in {\mathbf {S}}}\left( {\mathbf {d}}_i^T{\mathbf {d}}_t\right) ^2$, and $\left| z_{max}\right|$ is the maximum non-zero absolute value of ${\mathbf {Z}}$. The above steps relies on the following rationales:

$$\begin{aligned}&P\left( max\left( a,b\right) \ge T\right) \le P\left( a\ge T\right) + P\left( b\ge T\right) , \\&P\left( \sum _{k=1}^{N} a_k \ge T\right) \le \sum _{k=1}^N P\left( a_k\ge \frac{1}{N}T\right) , \\&P\left( \left| \sum _t{\epsilon }_t v_t\right| \ge T\right) \le 2\exp \left\{ -\frac{T^2}{32\left\| {\mathbf {v}}\right\| _2^2}\right\} . \end{aligned}$$

Then, the second probability term in the Eq. (23) is similarly found by

$$\begin{aligned}&P\left( \max _{j\notin {\mathbf {S}}}\left\| {\mathbf {d}}_j^T{\mathbf {X}}\right\| _1\ge T\right) \\&\quad =P\left( \max _{j\notin {\mathbf {S}}}\sum _k\left| \sum _{t\in {\mathbf {S}}}{\epsilon }_t\left| z_{t,k}\right| {\mathbf {d}}_j^T{\mathbf {d}}_t\right| \ge T\right) \\&\quad \le \sum _{j\notin {\mathbf {S}}}P\left( \sum _k\left| \sum _{t\in {\mathbf {S}}}{\epsilon }_t\left| z_{t,k}\right| {\mathbf {d}}_j^T{\mathbf {d}}_t\right| \ge T\right) \\&\quad \le \sum _{j\notin {\mathbf {S}}}\sum _k P\left( \left| \sum _{t\in {\mathbf {S}}}{\epsilon }_t\left| z_{t,k}\right| {\mathbf {d}}_j^T{\mathbf {d}}_t\right| \ge \frac{T}{n}\right) \\&\quad \le 2\sum _{j\notin {\mathbf {S}}}\sum _k\exp \left\{ -\frac{\left( \frac{T}{n}\right) ^2}{32\sum _{t\in {\mathbf {S}}}\left| z_{t,k}\right| ^2\left( {\mathbf {d}}_j^T{\mathbf {d}}_t\right) ^2}\right\} \\&\quad \le 2n\left( m-\left| {\mathbf {S}}\right| \right) \exp \left\{ -\frac{\left( \frac{T}{n}\right) ^2}{32\left| z_{max}\right| ^2\mu _{\left| {\mathbf {S}}\right| }}\right\} , \end{aligned}$$

where m denotes the number of the columns of ${\mathbf {D}}$. Thus, returning to Eq. (23), the probability of a failure is bounded by

$$\begin{aligned} P\left( Failure\right)\le & {} 2n\left| {\mathbf {S}}\right| \exp \left\{ -\frac{\left( \left| z_{min}\right| -\frac{T}{n}\right) ^2}{32\left| z_{max}\right| ^2\mu _{\left| {\mathbf {S}}\right| }}\right\} \\&+2n\left( m-\left| {\mathbf {S}}\right| \right) \exp \left\{ -\frac{\left( \frac{T}{n}\right) ^2}{32\left| z_{max}\right| ^2\mu _{\left| {\mathbf {S}}\right| }}\right\} . \end{aligned}$$

The above bound is a function of T and it should be minimized with respect to T to obtain the most tight upper-bound. However, the optimal value of T is hard to be evaluated in closed form. Thus, here we simply assign $T=\frac{1}{2}n\left| z_{min}\right|$, leading to

$$\begin{aligned} P\left( Failure\right) \le 2nm\exp \left\{ -\left( \frac{z_{min}}{z_{max}}\right) ^2 \frac{1}{128\mu _{\left| {\mathbf {S}}\right| }}\right\} . \end{aligned}$$

Furthermore, it is obvious that $\mu _{\left| {\mathbf {S}}\right| }\le \left| {\mathbf {S}}\right| \mu _{\mathbf {D}}^2$. Thus, for ${\mathbf {D}}\in \mathbb {R}^{d\times m}$, denoting $\rho =\frac{z_{min}}{z_{max}}$ and $m=\lambda d$ ($\lambda >1$), the probability of failure of Algorithm 2 in recovering a solution of $rank\left( {\mathbf {Z}}\right) \le \left| {\mathbf {S}}\right|$ is found by

$$\begin{aligned} P\left( Failure\right) \le 2n\lambda d\exp \left\{ -\frac{\rho ^2}{128\left| {\mathbf {S}}\right| \mu _{\mathbf {D}}^2}\right\} . \end{aligned}$$

If further assuming that $\left| {\mathbf {S}}\right| \propto \frac{c}{\mu _{\mathbf {D}}^2\log d}$, where c is an arbitrary constant. this probability becomes

$$\begin{aligned} P\left( Failure\right) \le 2n\lambda d\exp \left\{ -\frac{\rho ^2}{128c}\log d\right\} =2n\lambda \cdot d^{1-\frac{\rho ^2}{128c}}. \end{aligned}$$

When choosing $c<\frac{1}{128}\rho ^2$, this probability becomes arbitrarily small as $n\rightarrow \infty$. Thus, we obtain

$$\begin{aligned} rank\left( {\mathbf {Z}}\right) \le \left| {\mathbf {S}}\right| <\frac{1}{128} \left( \frac{z_{min}}{z_{max}}\right) ^2\cdot \frac{1}{\mu _{\mathbf {D}}^2\log d}. \end{aligned}$$

The condition on the sparsity of ${\mathbf {Z}}$, can be found by following the similar steps, i.e.,

$$\begin{aligned} \left\| {\mathbf {Z}}\right\| _0\le \left| {\mathbf {S}}\right| <\frac{1}{128} \left( \frac{z_{min}}{z_{max}}\right) ^2\cdot \frac{1}{\mu _{\mathbf {D}}^2\log d}. \end{aligned}$$

Hence, the conclusion is obtained as shown in Theorem 4. $\square$

Appendix 2: Convex Relaxation Algorithm

The convex counterpart of the proposed greedy algorithm is addressed in this section, which relaxes the simultaneously low-rank and sparse problem to a convex problem. The minimization on trace-norm and $\ell _1$-norm, both of which are convex functions, are used to approximate rank- and $\ell _0$-minimizations:

$$\begin{aligned}&\min _{{\mathbf {Z}},{\mathbf {E}}}~\left\| {\mathbf {Z}}\right\| _*+\lambda \left\| {\mathbf {Z}}\right\| _1+\mu \left\| {\mathbf {E}}\right\| _{2,1}, \\&subject~to~~{\mathbf {X}}={\mathbf {D}}{\mathbf {Z}}+{\mathbf {E}}, \end{aligned}$$

where $\left\| \cdot \right\| _*$ denotes trace-norm of a matrix, which is computed from the sum of the singular values of this matrix. To the best of our knowledge, there is no closed-form about such a convex problem. Thus, the iterative algorithm also needs to be developed and another two slack variables are required to be introduced:

$$\begin{aligned}&\min _{{\mathbf {Z}}_1,{\mathbf {Z}}_2,{\mathbf {Z}}_3,{\mathbf {E}}}~\left\| {\mathbf {Z}}_1\right\| _*+\lambda \left\| {\mathbf {Z}}_2\right\| _1+\mu \left\| {\mathbf {E}}\right\| _{2,1} \\&subject~to~~ {\left\{ \begin{array}{ll} {\mathbf {Z}}_1={\mathbf {Z}}_2 \\ {\mathbf {Z}}_2={\mathbf {Z}}_3 \\ {\mathbf {X}}={\mathbf {D}}{\mathbf {Z}}_3+{\mathbf {E}}, \end{array}\right. } \end{aligned}$$

where ${\mathbf {Z}}$ is renamed to ${\mathbf {Z}}_1$, and ${\mathbf {Z}}_2$ and ${\mathbf {Z}}_3$ denote the newly introduced slack variables. The four subproblems over the four variables are independently convex. Thus, the iterative algorithm alternately minimizes the subproblems over the four variables, and the approximate solution can be sure to be found. Inexact augmented lagrange multiplier (Lin et al. 2010) method can be used in solve this problem:

$$\begin{aligned}&\min _{{\mathbf {Z}}_1,{\mathbf {Z}}_2,{\mathbf {Z}}_3,\mathbf {E},\mathbf {Y}_1,\mathbf {Y}_2,\mathbf {Y}_3} {\left\| {\mathbf {Z}}_1\right\| _*}+\lambda \left\| {\mathbf {Z}}_2\right\| _1+\mu \left\| \mathbf {E}\right\| _{2,1} \\&\quad +\langle \mathbf {Y}_1,{\mathbf {X}}-\mathbf {D}{\mathbf {Z}}_3-\mathbf {E} \rangle +\langle \mathbf {Y}_2,{\mathbf {Z}}_1-{\mathbf {Z}}_2 \rangle \\&\quad +\langle \mathbf {Y}_3,{\mathbf {Z}}_2-{\mathbf {Z}}_3 \rangle +\frac{\tau }{2}\left( \left\| {\mathbf {X}}-\mathbf {D}{\mathbf {Z}}_3-\mathbf {E}\right\| _F^2 \right. \\&\quad \left. +\left\| {\mathbf {Z}}_1-{\mathbf {Z}}_2\right\| _F^2 + \left\| {\mathbf {Z}}_2-{\mathbf {Z}}_3\right\| _F^2\right) , \end{aligned}$$

where $\mathbf {Y}_1$, $\mathbf {Y}_2$, and $\mathbf {Y}_3$ are the Lagrange multipliers, $\tau >0$ is a penalty parameter, and the expression $\langle \mathbf {A},\mathbf {B} \rangle =trace\left( \mathbf {A}^T\mathbf {B} \right)$ is the inner-product of matrices $\mathbf {A}$ and $\mathbf {B}$.

Before deriving the iterative algorithm, the tool function is first defined as follows.

$$\begin{aligned} \varphi \left( \sigma ,x\right) =sign\left( x\right) \max \left( 0,\left( \left| x\right| -\sigma \right) \right) , \end{aligned}$$

where $\sigma >0$ denotes the shrinkage threshold and if x is vector or matrix, it operates on each element independently.

(1) The subproblem over ${\mathbf {Z}}_1$ can be re-arranged as

$$\begin{aligned} \min _{{\mathbf {Z}}_1}~\frac{1}{\tau }\left\| {\mathbf {Z}}_1 \right\| _* +\frac{1}{2}\left\| {\mathbf {Z}}_1-\left( {\mathbf {Z}}_3+\mathbf {Y}_2/\tau \right) \right\| _F^2 \end{aligned}$$

and solved by using singular values thresholding algorithm (Cai et al. 2010):

$$\begin{aligned} {\mathbf {Z}}_1^*=\mathbf {U}\varphi \left( \lambda ,\mathbf {S}\right) \mathbf {V}^T, \end{aligned}$$

where $\left[ \mathbf {U},\mathbf {S},\mathbf {V}^T\right] =svd\left( {\mathbf {Z}}_3+\mathbf {Y}_2/\tau \right)$.

(2) The subproblem over ${\mathbf {Z}}_2$ can be re-arranged as

$$\begin{aligned} \min _{{\mathbf {Z}}_2}~\frac{\lambda }{\tau }\left\| {\mathbf {Z}}_2 \right\| _{1} +\frac{1}{2}\left\| {\mathbf {Z}}_2-\left( {\mathbf {Z}}_3+\mathbf {Y}_3/\tau \right) \right\| _F^2 \end{aligned}$$

and solved by using iterative shrinkage-thresholding algorithm (Beck and Teboulle 2009):

$$\begin{aligned} {\mathbf {Z}}_2^*=\varphi \left( \lambda ,{\mathbf {Z}}_3+\mathbf {Y}_3/\tau \right) . \end{aligned}$$

(3) The subproblem over ${\mathbf {Z}}_3$ is simply a quadric form and solved by least squares:

$$\begin{aligned} {\mathbf {Z}}_3^*= & {} \left( {\mathbf {D}}^T{\mathbf {D}}+2\mathbf {I}\right) ^{-1} \\&\cdot \left( {\mathbf {D}}^T{\mathbf {X}}-{\mathbf {D}}^T{\mathbf {E}}+{\mathbf {Z}}_1+{\mathbf {Z}}_3 +\left( {\mathbf {D}}^T\mathbf {Y}_1-\mathbf {Y}_2-\mathbf {Y}_3\right) /\tau \right) . \end{aligned}$$

(4) The subproblem over ${\mathbf {E}}$ can be re-arranged as

$$\begin{aligned} \min _{\mathbf {E}}~ \frac{\mu }{\tau }\left\| \mathbf {E} \right\| _{2,1} +\frac{1}{2}\left\| \mathbf {E}-\left( {\mathbf {X}}-\mathbf {D}{\mathbf {Z}}_3+\mathbf {Y}_1/\tau \right) \right\| _F^2 \end{aligned}$$

and solved by using accelerated proximal gradient method (Bach et al. 2011):

$$\begin{aligned} \mathbf {e}_i={\left\{ \begin{array}{ll} \left( 1-\frac{\mu }{\tau \left\| \mathbf {w}_i\right\| _2}\right) \mathbf {w}_i, &{} if~\frac{\mu }{\tau }<\left\| \mathbf {w}_i\right\| _2 \\ \mathbf {0}, &{} otherwise, \end{array}\right. } \end{aligned}$$

where $\mathbf {e}_i$ is the i-th column vector of matrix $\mathbf {E}$, and $\mathbf {w}_i$ is the i-th column vector of the matrix $\mathbf {W}={\mathbf {X}}-\mathbf {D}{\mathbf {Z}}_3+\mathbf {Y}_1/\tau$.

Algorithm 5 depicts the formal description of the convex relaxation method.