An Efficient Algorithm for Computing the Approximate t-URV and its Applications

Abstract

This paper is devoted to the definition and computation of the tensor complete orthgonal decomposition of a third-order tensor called t-URV decompositions. We first give the definition for the t-URV decomposition of a third-order tensor and derive a deterministic algorithm for computing the t-URV. We then present a randomized algorithm to approximate t-URV, named compressed randomized t-URV (cort-URV). Note that t-URV and cort-URV are extensions of URV and compressed randomized URV from the matrix case to the tensor case, respectively. We also establish the deterministic and average-case error bounds for this algorithm. Finally, we illustrate the effectiveness of the proposed algorithm via several numerical examples, and we apply cort-URV to compress the data tensors from some image and video databases.

Proofs

1.1 The Proof of Theorem 3.1

Let ${\mathbf {F}}_{I_3}\in \mathbb {C}^{I_3\times I_3}$ be the normalized discrete Fourier transform matrix. Then $({\mathbf {F}}_{I_3}\otimes {\mathbf {I}}_{I_1})\cdot \mathrm{bcirc}({\mathcal {A}})\cdot ({\mathbf {F}}_{I_3}^{\mathrm{H}}\otimes {\mathbf {I}}_{I_2})=\mathrm{bdiag}({\mathbf {D}}_1,\dots ,{\mathbf {D}}_{I_3})$, where ${\mathbf {D}}_k\in \mathbb {C}^{I_1\times I_2}$. Hence for each k, we compute the URV decomposition of ${\mathbf {D}}_k$ as ${\mathbf {D}}_k={\mathbf {U}}_k\cdot {\mathbf {R}}_k\cdot {\mathbf {V}}_k^\top$. Then

$$\begin{aligned} \begin{aligned} \mathrm{bdiag}({\mathbf {D}}_1,\dots ,{\mathbf {D}}_{I_3})=&\mathrm{bdiag}({\mathbf {U}}_1,\dots ,{\mathbf {U}}_{I_3})\cdot \mathrm{bdiag}({\mathbf {R}}_1,\dots ,{\mathbf {R}}_{I_3})\\&\cdot \mathrm{bdiag}({\mathbf {V}}_1^\top ,\dots ,{\mathbf {V}}_{I_3}^\top ). \end{aligned} \end{aligned}$$

(A.1)

When we apply ${\mathbf {F}}_{I_3}^{\mathrm{H}}\otimes {\mathbf {I}}_{I_1}$ to the left-hand side and ${\mathbf {F}}_{I_3}\otimes {\mathbf {I}}_{I_2}$ to the right-hand side of (A.1), then

$$\begin{aligned} \begin{aligned}&\left( {\mathbf {F}}_{I_3}^{\mathrm{H}}\otimes {\mathbf {I}}_{I_1}\right) \cdot \mathrm{bdiag}({\mathbf {U}}_1,\dots ,{\mathbf {U}}_{I_3})\cdot \left( {\mathbf {F}}_{I_3}\otimes {\mathbf {I}}_{I_1}\right) ,\\&\left( {\mathbf {F}}_{I_3}^{\mathrm{H}}\otimes {\mathbf {I}}_{I_1}\right) \cdot \mathrm{bdiag}({\mathbf {R}}_1,\dots ,{\mathbf {R}}_{I_3})\cdot \left( {\mathbf {F}}_{I_3}\otimes {\mathbf {I}}_{I_1}\right) ,\\&\left( {\mathbf {F}}_{I_3}^{\mathrm{H}}\otimes {\mathbf {I}}_{I_1}\right) \cdot \mathrm{bdiag}({\mathbf {V}}_1,\dots ,{\mathbf {V}}_{I_3})\cdot \left( {\mathbf {F}}_{I_3}\otimes {\mathbf {I}}_{I_1}\right) , \end{aligned} \end{aligned}$$

(A.2)

are block circulant matrices. We define $\mathrm{unfold}({\mathcal {U}})$, $\mathrm{unfold}({\mathcal {R}})$ and $\mathrm{unfold}({\mathcal {V}})$ as the first block columns of each of block circulant matrices in (A.2), respectively. By folding these results, we give a decomposition of the form ${\mathcal {U}}*{\mathcal {R}}*{\mathcal {V}}^{\top }$.

Now we prove that ${\mathcal {U}}\in \mathbb {R}^{I_1\times I_1\times I_3}$ and ${\mathcal {V}}\in \mathbb {R}^{I_2\times I_2\times I_3}$ are orthogonal. Note that

$$\begin{aligned} {\mathcal {U}}*{\mathcal {U}}^\top =\mathrm{fold}(\mathrm{bcirc}({\mathcal {U}})\cdot \mathrm{unfold}({\mathcal {U}}^\top )). \end{aligned}$$

By the definition of t-product, ${\mathcal {U}}$ is orthogonal. We can also prove that ${\mathcal {V}}$ is orthogonal.

1.2 The Proof of Theorem 4.1

From Corollary 2.1, the left-hand side of (4.2) can be rewritten as

$$\begin{aligned} \begin{aligned} \Vert {\mathcal {A}}-{\mathcal {Q}}_1*{\mathcal {M}}*{\mathcal {Q}}_2^\top \Vert _F^2=&\frac{1}{I_3}\sum _{k=1}^{I_3}\Vert \widehat{{\mathcal {A}}}_{(:,:,k)}- {\mathbf {Q}}_{1k}{\mathbf {M}}_k{\mathbf {Q}}_{2k}^{\mathrm{H}}\Vert _F^2\\ =&\frac{1}{I_3}\sum _{k=1}^{I_3}\Vert \widehat{{\mathcal {A}}}_{(:,:,k)} -{\mathbf {Q}}_{1k}{\mathbf {Q}}_{1k}^{\mathrm{H}}\widehat{{\mathcal {A}}}_{(:,:,k)}{\mathbf {Q}}_{2k}{\mathbf {Q}}_{2k}^{\mathrm{H}}\\&+{\mathbf {Q}}_{1k}{\mathbf {Q}}_{1k}^{\mathrm{H}}\widehat{{\mathcal {A}}}_{(:,:,k)}{\mathbf {Q}}_{2k}{\mathbf {Q}}_{2k}^{\mathrm{H}} -{\mathbf {Q}}_{1k}{\mathbf {M}}_k{\mathbf {Q}}_{2k}^{\mathrm{H}}\Vert _F^2\\ =&\frac{1}{I_3}\sum _{k=1}^{I_3}\Vert \widehat{{\mathcal {A}}}_{(:,:,k)} -{\mathbf {Q}}_{1k}{\mathbf {Q}}_{1k}^{\mathrm{H}}\widehat{{\mathcal {A}}}_{(:,:,k)}{\mathbf {Q}}_{2k}{\mathbf {Q}}_{2k}^{\mathrm{H}}\Vert _F^2\\&+\frac{1}{I_3}\sum _{k=1}^{I_3}\Vert {\mathbf {Q}}_{1k}{\mathbf {Q}}_{1k}^{\mathrm{H}}\widehat{{\mathcal {A}}}_{(:,:,k)}{\mathbf {Q}}_{2k}{\mathbf {Q}}_{2k}^{\mathrm{H}} -{\mathbf {Q}}_{1k}{\mathbf {M}}_k{\mathbf {Q}}_{2k}^{\mathrm{H}}\Vert _F^2\\ =&\Vert {\mathcal {A}}-{\mathcal {Q}}_1*{\mathcal {Q}}_1^\top *{\mathcal {A}} *{\mathcal {Q}}_2*{\mathcal {Q}}_2^\top \Vert _F^2+\Vert {\mathcal {Q}}_1^\top *{\mathcal {A}} *{\mathcal {Q}}_2-{\mathcal {M}}\Vert _F^2. \end{aligned} \end{aligned}$$

By Theorem 2.2, the result in (4.2) immediatedly follows.

We have

$$\begin{aligned} \begin{aligned} \Vert {\mathcal {A}}-{\mathcal {A}}_K\Vert _F&\le \Vert {\mathcal {A}}-{\mathcal {Q}}_1*{\mathcal {M}}_K*{\mathcal {Q}}_2^\top \Vert _F\le \Vert {\mathcal {A}}-{\mathcal {Q}}_1*{\mathcal {Q}}_1^\top *{\mathcal {A}}_K*{\mathcal {Q}}_2*{\mathcal {Q}}_2^\top \Vert _F, \end{aligned} \end{aligned}$$

where the first inequality holds for Theorem 2.2 and the second inequality holds due to (4.2).

Now we prove (4.3). From Corollary 2.1, we have

$$\begin{aligned} \begin{aligned}&\Vert {\mathcal {A}}-{\mathcal {Q}}_1*{\mathcal {Q}}_1^\top *{\mathcal {A}}_K*{\mathcal {Q}}_2*{\mathcal {Q}}_2^\top \Vert _F^2\\&\quad =\frac{1}{I_3}\sum _{k=1}^{I_3}\Vert \widehat{{\mathcal {A}}}_{(:,:,k)}-{\mathbf {Q}}_{1k}{\mathbf {Q}}_{1k}^{\mathrm{H}}\widehat{{\mathcal {A}}}_{(:,:,k),K}{\mathbf {Q}}_{2k}{\mathbf {Q}}_{2k}^{\mathrm{H}}\Vert _F^2\\&\quad =\frac{1}{I_3}\sum _{k=1}^{I_3}\Vert \widehat{{\mathcal {A}}}_{(:,:,k)} -{\mathbf {Q}}_{1k}{\mathbf {Q}}_{1k}^{\mathrm{H}}\widehat{{\mathcal {A}}}_{(:,:,k),K} +{\mathbf {Q}}_{1k}{\mathbf {Q}}_{1k}^{\mathrm{H}}\widehat{{\mathcal {A}}}_{(:,:,k),K}\\&\qquad -{\mathbf {Q}}_{1k}{\mathbf {Q}}_{1k}^{\mathrm{H}}\widehat{{\mathcal {A}}}_{(:,:,k),K} {\mathbf {Q}}_{2k}{\mathbf {Q}}_{2k}^{\mathrm{H}}\Vert _F^2\\&\quad =\frac{1}{I_3}\sum _{k=1}^{I_3}\Vert \widehat{{\mathcal {A}}}_{(:,:,k)}-{\mathbf {Q}}_{1k}{\mathbf {Q}}_{1k}^{\mathrm{H}}\widehat{{\mathcal {A}}}_{(:,:,k),K}\Vert _F^2+\frac{1}{I_3}\sum _{k=1}^{I_3}\Vert {\mathbf {Q}}_{1k}{\mathbf {Q}}_{1k}^{\mathrm{H}}\widehat{{\mathcal {A}}}_{(:,:,k),K}\\&\qquad -{\mathbf {Q}}_{1k}{\mathbf {Q}}_{1k}^{\mathrm{H}}\widehat{{\mathcal {A}}}_{(:,:,k),K}{\mathbf {Q}}_{2k}{\mathbf {Q}}_{2k}^{\mathrm{H}}\Vert _F^2\\&\quad \le \frac{1}{I_3}\sum _{k=1}^{I_3}\Vert \widehat{{\mathcal {A}}}_{(:,:,k)}-{\mathbf {Q}}_{1k}{\mathbf {Q}}_{1k}^{\mathrm{H}}\widehat{{\mathcal {A}}}_{(:,:,k),K}\Vert _F^2+\frac{1}{I_3}\sum _{k=1}^{I_3}\Vert \widehat{{\mathcal {A}}}_{(:,:,k),K}-\widehat{{\mathcal {A}}}_{(:,:,k),K}{\mathbf {Q}}_{2k}{\mathbf {Q}}_{2k}^{\mathrm{H}}\Vert _F^2\\&\quad =\Vert {\mathcal {A}}-{\mathcal {Q}}_1*{\mathcal {Q}}_1^\top *{\mathcal {A}}_K\Vert _F^2+\Vert {\mathcal {A}}_K-{\mathcal {A}}_K*{\mathcal {Q}}_2*{\mathcal {Q}}_2^\top \Vert _F^2, \end{aligned} \end{aligned}$$

which implies that

$$\begin{aligned} \Vert {\mathcal {A}}-{\mathcal {Q}}_1*{\mathcal {Q}}_1^\top *{\mathcal {A}}_K*{\mathcal {Q}}_2*{\mathcal {Q}}_2^\top \Vert _F\le & {} \Vert {\mathcal {A}}-{\mathcal {Q}}_1*{\mathcal {Q}}_1^\top *{\mathcal {A}}_K\Vert _F+\Vert {\mathcal {A}}_K\nonumber \\&-{\mathcal {A}}_K*{\mathcal {Q}}_2*{\mathcal {Q}}_2^\top \Vert _F. \end{aligned}$$

(A.3)

Here $\widehat{{\mathcal {A}}}_K={ fft}({\mathcal {A}}_K,[],3)$ and $\widehat{{\mathcal {A}}}_{(:,:,k),K}$ is the best rank-K approximation of $\widehat{{\mathcal {A}}}_{(:,:,k)}$. Writing ${\mathcal {A}}={\mathcal {A}}_K+{\mathcal {A}}-{\mathcal {A}}_K$, using the fact that for given three nonegative numbers $a_i\in \mathbb {R}$ with $i=1,2,3$, $a_1^2\le a_2^2+a_3^2$ implies that $a_1\le a_2+a_3$, and applying the triangle inequality gives (4.3).

1.3 The Proof of Theorem 4.3

Note that $\widehat{{\mathcal {Q}}}_1\in \mathbb {C}^{I_1\times L\times I_3}$ and $\widehat{{\mathcal {Q}}}_2\in \mathbb {C}^{I_2\times L\times I_3}$ are, respectively, defined as $\widehat{{\mathcal {Q}}}_1(:,:,k)={\mathbf {Q}}_{1k}$ and $\widehat{{\mathcal {Q}}}_2(:,:,k)={\mathbf {Q}}_{2k}$ with all k, where ${\mathbf {Q}}_{1k}$ and ${\mathbf {Q}}_{2k}$ are obtained from Step 9 of Algorithm 4.1.

Let ${\mathcal {Q}}_1={ ifft}(\widehat{{\mathcal {Q}}}_1,[],3)$ and ${\mathcal {Q}}_2={ ifft}(\widehat{{\mathcal {Q}}}_2,[],3)$. From Theorem 4.2, we have

$$\begin{aligned} \Vert {\mathcal {A}}-{\mathcal {A}}_{\mathrm{CoR}}\Vert _F\le & {} \Vert {\mathcal {A}}-{\mathcal {A}}_K\Vert _F+\Vert {\mathcal {A}}_K-{\mathcal {Q}}_1*{\mathcal {Q}}_1^\top *{\mathcal {A}}_K\Vert _F+\Vert {\mathcal {A}}_K\nonumber \\&-{\mathcal {A}}_K*{\mathcal {Q}}_2*{\mathcal {Q}}_2^\top \Vert _F. \end{aligned}$$

(A.4)

By Corollary 2.1, we have

$$\begin{aligned} {\left\{ \begin{array}{ll} &{}\Vert {\mathcal {A}}_K-{\mathcal {Q}}_1*{\mathcal {Q}}_1^\top *{\mathcal {A}}_K\Vert _F^2= \frac{1}{I_3}\sum _{k=1}^{I_3}\Vert \widehat{{\mathcal {A}}}_{K,(:,:,k)}-\widehat{{\mathcal {Q}}}_1(:,:,k)\widehat{{\mathcal {Q}}}_{1,(:,:,k)}^{\mathrm{H}}\widehat{{\mathcal {A}}}_{K,(:,:,k)}\Vert _F^2;\\ &{}\Vert {\mathcal {A}}_K-{\mathcal {A}}_K*{\mathcal {Q}}_2*{\mathcal {Q}}_2^\top \Vert _F^2= \frac{1}{I_3}\sum _{k=1}^{I_3}\Vert \widehat{{\mathcal {A}}}_{K,(:,:,k)}-\widehat{{\mathcal {A}}}_{K,(:,:,k)}\widehat{{\mathcal {Q}}}_{2,(:,:,k)}\widehat{{\mathcal {Q}}}_{2,(:,:,k)}^{\mathrm{H}}\Vert _F^2, \end{array}\right. } \end{aligned}$$

where $\widehat{{\mathcal {A}}}_{K,(:,:,k)}$, $\widehat{{\mathcal {Q}}}_{1,(:,:,k)}$ and $\widehat{{\mathcal {Q}}}_{2,(:,:,k)}$ are the k-th frontal slices of $\widehat{{\mathcal {A}}}_{K}$, $\widehat{{\mathcal {Q}}}_{1}$ and $\widehat{{\mathcal {Q}}}_{2}$, respectively.

For all k, let

$$\begin{aligned} {\left\{ \begin{array}{ll} {\mathbf {W}}_k&{}=(\varvec{\Sigma }_{3k}^\top \cdot \varvec{\Sigma }_{3k})^{q+1}\cdot \varvec{\Omega }_{2k}\cdot \varvec{\Omega }_{1k}^\dag \cdot \begin{pmatrix} \varvec{\Sigma }_{1k}^{-2q-2}\\ {\mathbf {0}}_{(L-P)\times K} \end{pmatrix},\\ {\mathbf {D}}_k&{}=\varvec{\Sigma }_{3k}\cdot (\varvec{\Sigma }_{3k}^\top \cdot \varvec{\Sigma }_{3k})^{q}\cdot \varvec{\Omega }_{2k}\cdot \varvec{\Omega }_{1k}^\dag \cdot \begin{pmatrix} \varvec{\Sigma }_{1k}^{-2q-1}\\ {\mathbf {0}}_{(L-P)\times K} \end{pmatrix}. \end{array}\right. } \end{aligned}$$

From the proof of Theorem 5 in [35], for all k, we have

$$\begin{aligned} \begin{aligned}&\Vert \widehat{{\mathcal {A}}}_{K,(:,:,k)}-\widehat{{\mathcal {Q}}}_1(:,:,k)\widehat{{\mathcal {Q}}}_{1,(:,:,k)}^{\mathrm{H}}\widehat{{\mathcal {A}}}_{K,(:,:,k)}\Vert _F\le \frac{\sqrt{K}\Vert {\mathbf {D}}_k\cdot \varvec{\Sigma }_{1k}\Vert _2{\hat{\sigma }}_{1}^{(k)}}{\sqrt{\Vert {\mathbf {D}}_k\cdot \varvec{\Sigma }_{1k}\Vert _2^2+({\hat{\sigma }}_{1}^{(k)})^2}},\\&\Vert \widehat{{\mathcal {A}}}_K(:,:,k)-\widehat{{\mathcal {A}}}_K(:,:,k)\cdot \widehat{{\mathcal {Q}}}_2(:,:,k)\cdot \widehat{{\mathcal {Q}}}_2(:,:,k)^{\mathrm{H}}\Vert _F\le \frac{\sqrt{K}\Vert {\mathbf {W}}_k\cdot \varvec{\Sigma }_{1k}\Vert _2{\hat{\sigma }}_{1}^{(k)}}{\sqrt{\Vert {\mathbf {W}}_k\cdot \varvec{\Sigma }_{1k}\Vert _2^2+({\hat{\sigma }}_{1}^{(k)})^2}}. \end{aligned} \end{aligned}$$

(A.5)

According to the definitions of ${\mathbf {W}}_k$ and ${\mathbf {D}}_k$, we have

$$\begin{aligned}&\frac{\Vert {\mathbf {W}}_k\cdot \varvec{\Sigma }_{1k}\Vert _2}{ \Vert \varvec{\Omega }_{2k}\Vert _2\Vert \varvec{\Omega }_{1k}^\dag \Vert _2}\le \frac{({\hat{\sigma }}_{L-P+1}^{(k)})^2}{{\hat{\sigma }}_{K}^{(k)}}\left( \frac{{\hat{\sigma }}_{L-P+1}^{(k)}}{{\hat{\sigma }}_{K}^{(k)}}\right) ^{2q},\nonumber \\&\quad \times \frac{\Vert {\mathbf {D}}_k\cdot \varvec{\Sigma }_{1k}\Vert _2}{\Vert \varvec{\Omega }_{2k}\Vert _2\Vert \varvec{\Omega }_{1k}^\dag \Vert _2}\le {\hat{\sigma }}_{L-P+1}^{(k)}\left( \frac{{\hat{\sigma }}_{L-P+1}^{(k)}}{{\hat{\sigma }}_{K}^{(k)}}\right) ^{2q}. \end{aligned}$$

(A.6)

Let $f(x)=\frac{x}{\sqrt{1+x^2}}$ with $x>0$. Then the derivative function of f(x) is $f'(x)=\frac{1}{(1+x^2)^{3/2}},$ which is always larger than 0 for all $x>0$. Substituting (A.6) into (A.5) and combining the property of f(x) imply this theorem.

Remark A.1

In the proof, we also use the fact: for given $(I_3+1)$ nonegative numbers $a_k\in \mathbb {R}$ with $k=1,2,\dots ,I_3$, when $a_1^2\le \frac{1}{I_3}( a_2^2+a_3^2+\dots +a_{I_3+1}^2)$, then $a_1\le \frac{1}{\sqrt{I_3}}( a_2+a_3+\dots +a_{I_3+1})$.

1.4 The Proof of Theorem 4.4

For all k, the matrices $\varvec{\Omega }_{1k}$ and $\varvec{\Omega }_{2k}$ are given in (4.4). Since the matrix ${\mathbf {G}}$ is a standard Gaussian matrix, then the entries of $\varvec{\Omega }_{1k}$ and $\varvec{\Omega }_{2k}$ are i.i.d. standard Gaussian variables. Hence, we first take expections over $\varvec{\Omega }_{2k}$ and next over $\varvec{\Omega }_{1k}$. By invoking Theorem 4.3, and Propositions 2.1 and 2.2, we have

$$\begin{aligned} \begin{aligned}&\mathbb {E}\Vert {\mathcal {A}}-{\mathcal {A}}_\mathrm{CoR}\Vert _F\\&\quad \le \Vert {\mathcal {A}}-{\mathcal {A}}_K\Vert _F\\&\qquad +\frac{1}{I_3}\sum _{k=1}^{I_3}\mathbb {E}_{\varvec{\Omega }_{1k}}\mathbb {E}_{\varvec{\Omega }_{2k}}\left( \sqrt{\frac{\alpha _k^2\Vert \varvec{\Omega }_{2k}\Vert _2^2\Vert \varvec{\Omega }_{1k}^\dag \Vert _2^2}{1+\beta _k^2\Vert \varvec{\Omega }_{2k}\Vert _2^2\Vert \varvec{\Omega }_{1k}^\dag \Vert _2^2}}+\sqrt{\frac{\eta _k^2\Vert \varvec{\Omega }_{2k}\Vert _2^2\Vert \varvec{\Omega }_{1k}^\dag \Vert _2^2}{1+\tau _k^2\Vert \varvec{\Omega }_{2k}\Vert _2^2\Vert \varvec{\Omega }_{1k}^\dag \Vert _2^2}}\right) \\&\quad \le \Vert {\mathcal {A}}-{\mathcal {A}}_K\Vert _F+\frac{1}{I_3}\sum _{k=1}^{I_3}\mathbb {E}_{\varvec{\Omega }_{1k}}\left( \sqrt{\frac{\alpha _k^2\nu _1^2\Vert \varvec{\Omega }_{1k}^\dag \Vert _2^2}{1+\beta _k^2\nu _1^2\Vert \varvec{\Omega }_{1k}^\dag \Vert _2^2}}+\sqrt{\frac{\eta _k^2\nu _1^2\Vert \varvec{\Omega }_{1k}^\dag \Vert _2^2}{1+\tau _k^2\nu _1^2\Vert \varvec{\Omega }_{1k}^\dag \Vert _2^2}}\right) \\&\quad \le \Vert {\mathcal {A}}-{\mathcal {A}}_K\Vert _F+\frac{1}{I_3}\sum _{k=1}^{I_3}\left( \sqrt{\frac{\alpha _k^2\nu _1^2\nu _2^2}{1+\beta _k^2\nu _1^2\nu _2^2}}+\sqrt{\frac{\eta _k^2\nu _1^2\nu _2^2}{1+\tau _k^2\nu _1^2\nu _2^2}}\right) \\&\quad \le \Vert {\mathcal {A}}-{\mathcal {A}}_K\Vert _F+\frac{\nu }{I_3}\sum _{k=1}^{I_3}(\alpha _k+\eta _k), \end{aligned} \end{aligned}$$

which implies the result.