An Efficient Randomized Algorithm for Computing the Approximate Tucker Decomposition

Abstract

By combining the thin QR decomposition and the subsampled randomized Fourier transform (SRFT), we obtain an efficient randomized algorithm for computing the approximate Tucker decomposition with a given target multilinear rank. We also combine this randomized algorithm with the power iteration technique to improve the efficiency of the algorithm. By using the results about the singular values of the product of orthonormal matrices with the Kronecker product of SRFT matrices, we obtain the error bounds of these two algorithms. Finally, the efficiency of these algorithms is illustrated by several numerical examples.

Appendices

Proof of Lemma 4.5

Note that

$$\begin{aligned} \Vert {\mathbf {E}}\Vert _2\le \Vert {\mathbf {E}}\Vert _F=\sqrt{\sum _{k,k'=1}^K|e_{kk'}|^2}. \end{aligned}$$

In order to obtain the bound of $\Vert {\mathbf {E}}\Vert _2$, we need to obtain the upper bound of ${\mathfrak {E}}|e_{kk'}|^2$ for all $k,k'=1,2,\dots ,K$.

Let $i=i_1+(i_2-1)I_1$, $i'=i_1'+(i_2'-1)I_1$, $j=j_1+(j_2-1)I_1$, $j'=j_1'+(j_2'-1)I_1$ and $l=l_1+(l_2-1)L_1$. By (4.1), we have the expectation

$$\begin{aligned} \begin{aligned} {\mathfrak {E}}|e_{kk'}|^2&={\mathfrak {E}}\left( \sum _{i_1=1}^{I_1} \sum _{i_2=1}^{I_2}d_{i_1i_1}^{(1)}d_{i_2i_2}^{(2)}u_{ik}\sum _{i_1'\ne i_1} \sum _{i_2'\ne i_2}{\bar{d}}_{i_1'i_1'}^{(1)}{\bar{d}}_{i_2'i_2'}^{(2)} {\bar{u}}_{i'k'}{\bar{h}}^{(1)}_{i_1i_1'}{\bar{h}}^{(2)}_{i_2i_2'}\right. \\&\left. \quad \quad \cdot \sum _{j_1=1}^{I_1}\sum _{j_2=1}^{I_2} {\bar{d}}_{j_1j_1}^{(1)}{\bar{d}}_{j_2j_2}^{(2)}{\bar{u}}_{jk} \sum _{j_1'\ne j_1}\sum _{j_2'\ne j_2}d_{j_1'j_1'}^{(1)}d_{j_2'j_2'}^{(2)} u_{j'k'}h^{(1)}_{j_1j_1'}h^{(2)}_{j_2j_2'}\right) \\&={\mathfrak {E}}\left( \sum _{i_1,j_1=1}^{I_1}\sum _{i_2,j_2=1}^{I_2} \left( d_{i_1i_1}^{(1)}d_{i_2i_2}^{(2)}u_{ik}\sum _{i_1'\ne i_1}\sum _{i_2' \ne i_2}{\bar{d}}_{i_1'i_1'}^{(1)}{\bar{d}}_{i_2'i_2'}^{(2)} {\bar{u}}_{i'k'}{\bar{h}}^{(1)}_{i_1i_1'}{\bar{h}}^{(2)}_{i_2i_2'}\right) \right. \\&\left. \quad \quad \cdot \left( {\bar{d}}_{j_1j_1}^{(1)} {\bar{d}}_{j_2j_2}^{(2)}{\bar{u}}_{jk}\sum _{j_1'\ne j_1}\sum _{j_2'\ne j_2}d_{j_1'j_1'}^{(1)}d_{j_2'j_2'}^{(2)}u_{j'k'}h^{(1)}_{j_1j_1'} h^{(2)}_{j_2j_2'}\right) \right) . \end{aligned} \end{aligned}$$

By using the fact that $|d_{i_1i_1}^{(1)}|=|d_{i_2i_2}^{(2)}|=1$, we obtain that

$$\begin{aligned}&{\mathfrak {E}}\left( \sum _{i_1,j_1=1}^{I_1}\sum _{i_2,j_2=1}^{I_2} \left( d_{i_1i_1}^{(1)}d_{i_2i_2}^{(2)}u_{ik}\sum _{i_1'\ne i_1} \sum _{i_2'\ne i_2}{\bar{d}}_{i_1'i_1'}^{(1)}{\bar{d}}_{i_2'i_2'}^{(2)} {\bar{u}}_{i'k'}{\bar{h}}^{(1)}_{i_1i_1'}{\bar{h}}^{(2)}_{i_2i_2'}\right) \right. \nonumber \\&\left. \quad \quad \cdot \left( {\bar{d}}_{j_1j_1}^{(1)} {\bar{d}}_{j_2j_2}^{(2)}{\bar{u}}_{jk}\sum _{j_1'\ne j_1}\sum _{j_2' \ne j_2}d_{j_1'j_1'}^{(1)}d_{j_2'j_2'}^{(2)}u_{j'k'}h^{(1)}_{j_1j_1'} h^{(2)}_{j_2j_2'}\right) \right) \nonumber \\&\quad ={\mathfrak {E}}\left( \sum _{i_1=1}^{I_1}\sum _{i_2=1}^{I_2}|u_{ik}|^2 \left| \sum _{i_1'\ne i_1}\sum _{i_2'\ne i_2}d_{i_1'i_1'}^{(1)} d_{i_2'i_2'}^{(2)}u_{i'k'}h^{(1)}_{i_1i_1'}h^{(2)}_{i_2i_2'}\right| ^2 \right) \nonumber \\&\quad \quad +\,{\mathfrak {E}}\left( \sum _{i_1=1}^{I_1}\sum _{i_2\ne j_2} d_{i_2i_2}^{(2)}{\bar{d}}_{j_2j_2}^{(2)}u_{ik}{\bar{u}}_{jk}\sum _{i_1' \ne i_1}\sum _{i_2'\ne i_2}{\bar{d}}_{i_1'i_1'}^{(1)} {\bar{d}}_{i_2'i_2'}^{(2)}{\bar{u}}_{i'k'}{\bar{h}}^{(1)}_{i_1i_1'} {\bar{h}}^{(2)}_{i_2i_2'}\right. \nonumber \\&\left. \quad \quad \sum _{j_1'\ne i_1}\sum _{j_2'\ne j_2}d_{j_1'j_1'}^{(1)} d_{j_2'j_2'}^{(2)}u_{j'k'}h^{(1)}_{i_1j_1'}h^{(2)}_{j_2j_2'}\right) \nonumber \\&\quad \quad +\,{\mathfrak {E}}\left( \sum _{i_1\ne j_1}\sum _{i_2=1}^{I_2} d_{i_1i_1}^{(1)}{\bar{d}}_{j_1j_1}^{(1)}u_{ik}{\bar{u}}_{jk}\sum _{i_1' \ne i_1}\sum _{i_2'\ne i_2}{\bar{d}}_{i_1'i_1'}^{(1)} {\bar{d}}_{i_2'i_2'}^{(2)}{\bar{u}}_{i'k'}{\bar{h}}^{(1)}_{i_1i_1'} {\bar{h}}^{(2)}_{i_2i_2'}\right. \nonumber \\&\left. \quad \quad \sum _{j_1'\ne j_1}\sum _{j_2'\ne i_2}d_{j_1'j_1'}^{(1)} d_{j_2'j_2'}^{(2)}u_{j'k'}h^{(1)}_{j_1j_1'}h^{(2)}_{i_2j_2'}\right) \nonumber \\&\quad \quad +\,{\mathfrak {E}}\left( \sum _{i_1\ne j_1}\sum _{i_2\ne j_2} d_{i_1i_1}^{(1)}d_{i_2i_2}^{(2)}{\bar{d}}_{j_1j_1}^{(1)} {\bar{d}}_{j_2j_2}^{(2)}u_{ik}{\bar{u}}_{jk}\sum _{i_1'\ne i_1} \sum _{i_2'\ne i_2}{\bar{d}}_{i_1'i_1'}^{(1)}{\bar{d}}_{i_2'i_2'}^{(2)} {\bar{u}}_{i'k'}{\bar{h}}^{(1)}_{i_1i_1'}{\bar{h}}^{(2)}_{i_2i_2'}\right. \nonumber \\&\left. \quad \quad \sum _{j_1'\ne j_1}\sum _{j_2'\ne j_2}d_{j_1'j_1'}^{(1)}d_{j_2'j_2'}^{(2)}u_{j'k'} h^{(1)}_{j_1j_1'}h^{(2)}_{j_2j_2'}\right) . \end{aligned}$$

(A.1)

To bound the first term in the right-hand side of (A.1), we have

$$\begin{aligned} \begin{aligned}&{\mathfrak {E}}\left( \sum _{i_1=1}^{I_1}\sum _{i_2=1}^{I_2}|u_{ik}|^2 \left| \sum _{i_1'\ne i_1}\sum _{i_2'\ne i_2} d_{i_1'i_1'}^{(1)}d_{i_2'i_2'}^{(2)}u_{i'k'}h^{(1)}_{i_1i_1'} h^{(2)}_{i_2i_2'}\right| ^2\right) =\sum _{i_1=1}^{I_1}\sum _{i_2=1}^{I_2} |u_{ik}|^2\\&\qquad {\mathfrak {E}}\left| \sum _{i_1'\ne i_1}\sum _{i_2'\ne i_2}d_{i_1'i_1'}^{(1)}d_{i_2'i_2'}^{(2)}u_{i'k'}h^{(1)}_{i_1i_1'} h^{(2)}_{i_2i_2'}\right| ^2. \end{aligned} \end{aligned}$$

Note that

$$\begin{aligned} \begin{aligned}&{\mathfrak {E}}\left| \sum _{i_1'\ne i_1}\sum _{i_2'\ne i_2} d_{i_1'i_1'}^{(1)}d_{i_2'i_2'}^{(2)}u_{i'k'}h^{(1)}_{i_1i_1'} h^{(2)}_{i_2i_2'}\right| ^2\\&\quad ={\mathfrak {E}}\left( \sum _{i_1'\ne i_1}\sum _{i_2' \ne i_2}{\bar{d}}_{i_1'i_1'}^{(1)}{\bar{d}}_{i_2'i_2'}^{(2)} {\bar{u}}_{i'k'}{\bar{h}}^{(1)}_{i_1i_1'}{\bar{h}}^{(2)}_{i_2i_2'} \sum _{j_1'\ne i_1}\sum _{j_2'\ne i_2}d_{j_1'j_1'}^{(1)}d_{j_2'j_2'}^{(2)} u_{j'k'}h^{(1)}_{i_1j_1'}h^{(2)}_{i_2j_2'}\right) \\&\quad =\sum _{i_1',j_1'\ne i_1}\sum _{i_2',j_2'\ne i_2}{\bar{u}}_{i'k'}u_{j'k'}{\mathfrak {E}} \left( {\bar{d}}_{i_1'i_1'}^{(1)}{\bar{d}}_{i_2'i_2'}^{(2)}d_{j_1'j_1'}^{(1)}d_{j_2'j_2'}^{(2)}{\bar{h}}^{(1)}_{i_1i_1'}{\bar{h}}^{(2)}_{i_2i_2'} h^{(1)}_{i_1j_1'}h^{(2)}_{i_2j_2'}\right) . \end{aligned} \end{aligned}$$

By the fact that $|d_{i_1i_1}^{(1)}|=|d_{i_2i_2}^{(2)}|=1$, we obtain that

$$\begin{aligned} \begin{aligned}&\sum _{i_1',j_1'\ne i_1}\sum _{i_2',j_2'\ne i_2} {\bar{u}}_{i'k'}u_{j'k'}{\mathfrak {E}}\left( {\bar{d}}_{i_1'i_1'}^{(1)} {\bar{d}}_{i_2'i_2'}^{(2)}d_{j_1'j_1'}^{(1)}d_{j_2'j_2'}^{(2)} {\bar{h}}^{(1)}_{i_1i_1'}{\bar{h}}^{(2)}_{i_2i_2'}h^{(1)}_{i_1j_1'} h^{(2)}_{i_2j_2'}\right) \\&\quad =\sum _{i_1'\ne i_1}\sum _{i_2'\ne i_2}|u_{i'k'}|^2{\mathfrak {E}} \left( |h^{(1)}_{i_1i_1'}|^2|h^{(2)}_{i_2i_2'}|^2\right) \\&\qquad +\,\sum _{i_1'\ne i_1}\sum _{i_2',j_2'\ne i_2, i_2'\ne j_2'} {\bar{u}}_{i'k'}u_{j'k'}{\mathfrak {E}}\left( {\bar{d}}_{i_2'i_2'}^{(2)} d_{j_2'j_2'}^{(2)}|h^{(1)}_{i_1i_1'}|^2{\bar{h}}^{(2)}_{i_2i_2'} h^{(2)}_{i_2j_2'}\right) \\&\qquad +\,\sum _{i_1',j_1'\ne i_1, i_1'\ne j_1'}\sum _{i_2'\ne i_2} {\bar{u}}_{i'k'}u_{j'k'}{\mathfrak {E}}\left( {\bar{d}}_{i_1'i_1'}^{(1)} d_{j_1'j_1'}^{(1)}|h^{(2)}_{i_2i_2'}|^2{\bar{h}}^{(1)}_{i_1i_1'} h^{(1)}_{i_1j_1'}\right) \\&\qquad +\,\sum _{i_1',j_1'\ne i_1, i_1'\ne j_1'}\sum _{i_2',j_2'\ne i_2, i_2'\ne j_2'}{\bar{u}}_{i'k'}u_{j'k'}{\mathfrak {E}}\left( {\bar{d}}_{i_1'i_1'}^{(1)} {\bar{d}}_{i_2'i_2'}^{(2)}d_{j_1'j_1'}^{(1)}d_{j_2'j_2'}^{(2)} {\bar{h}}^{(1)}_{i_1i_1'}{\bar{h}}^{(2)}_{i_2i_2'}h^{(1)}_{i_1j_1'} h^{(2)}_{i_2j_2'}\right) . \end{aligned} \end{aligned}$$

The independence of random variables involves that

$$\begin{aligned} \begin{aligned}&\sum _{i_1',j_1'\ne i_1}\sum _{i_2',j_2'\ne i_2}{\bar{u}}_{i'k'} u_{j'k'}{\mathfrak {E}}\left( {\bar{d}}_{i_1'i_1'}^{(1)} {\bar{d}}_{i_2'i_2'}^{(2)}d_{j_1'j_1'}^{(1)}d_{j_2'j_2'}^{(2)} {\bar{h}}^{(1)}_{i_1i_1'}{\bar{h}}^{(2)}_{i_2i_2'}h^{(1)}_{i_1j_1'} h^{(2)}_{i_2j_2'}\right) \\&\quad =\sum _{i_1'\ne i_1}\sum _{i_2'\ne i_2}|u_{i'k'}|^2{\mathfrak {E}} \left( |h^{(1)}_{i_1i_1'}|^2\right) {\mathfrak {E}} \left( |h^{(2)}_{i_2i_2'}|^2\right) \\&\qquad +\,\sum _{i_1'\ne i_1}\sum _{i_2',j_2'\ne i_2, i_2'\ne j_2'} {\bar{u}}_{i'k'}u_{j'k'}{\mathfrak {E}}\left( {\bar{d}}_{i_2'i_2'}^{(2)} \right) {\mathfrak {E}}\left( d_{j_2'j_2'}^{(2)}\right) {\mathfrak {E}}|h^{(1)}_{i_1i_1'}|^2{\mathfrak {E}} \left( {\bar{h}}^{(2)}_{i_2i_2'}h^{(2)}_{i_2j_2'}\right) \\&\qquad +\,\sum _{i_1',j_1'\ne i_1, i_1'\ne j_1'}\sum _{i_2'\ne i_2} {\bar{u}}_{i'k'}u_{j'k'}{\mathfrak {E}}\left( {\bar{d}}_{i_1'i_1'}^{(1)} \right) {\mathfrak {E}}\left( d_{j_1'j_1'}^{(1)}\right) {\mathfrak {E}} |h^{(2)}_{i_2i_2'}|^2{\mathfrak {E}}\left( {\bar{h}}^{(1)}_{i_1i_1'} h^{(1)}_{i_1j_1'}\right) \\&\qquad +\,\sum _{i_1',j_1'\ne i_1, i_1'\ne j_1'}\sum _{i_2',j_2'\ne i_2, i_2'\ne j_2'}{\bar{u}}_{i'k'}u_{j'k'}{\mathfrak {E}} \left( {\bar{d}}_{i_1'i_1'}^{(1)}\right) {\mathfrak {E}} \left( {\bar{d}}_{i_2'i_2'}^{(2)}\right) {\mathfrak {E}} \left( d_{j_1'j_1'}^{(1)}\right) {\mathfrak {E}}\left( d_{j_2'j_2'}^{(2)} \right) \\&\qquad {\mathfrak {E}}\left( {\bar{h}}^{(1)}_{i_1i_1'} {\bar{h}}^{(2)}_{i_2i_2'}h^{(1)}_{i_1j_1'}h^{(2)}_{i_2j_2'}\right) . \end{aligned} \end{aligned}$$

From Lemma 4.1, the fact that ${\mathfrak {E}}(d_{i_1i_1}^{(1)})={\mathfrak {E}}(d_{i_2i_2}^{(2)})=0$ and the fact that the columns of ${\mathbf {U}}$ are unitary, we have

$$\begin{aligned} \begin{aligned}&\sum _{i_1',j_1'\ne i_1}\sum _{i_2',j_2'\ne i_2}{\bar{u}}_{i'k'}u_{j'k'} {\mathfrak {E}}\left( {\bar{d}}_{i_1'i_1'}^{(1)}{\bar{d}}_{i_2'i_2'}^{(2)} d_{j_1'j_1'}^{(1)}d_{j_2'j_2'}^{(2)}{\bar{h}}^{(1)}_{i_1i_1'} {\bar{h}}^{(2)}_{i_2i_2'}h^{(1)}_{i_1j_1'}h^{(2)}_{i_2j_2'}\right) \\&\quad =\sum _{i_1'\ne i_1}\sum _{i_2'\ne i_2}|u_{i'k'}|^2{\mathfrak {E}} \left( |h^{(1)}_{i_1i_1'}|^2\right) {\mathfrak {E}} \left( |h^{(2)}_{i_2i_2'}|^2\right) =L_1L_2\sum _{i_1'\ne i_1}\sum _{i_2' \ne i_2}|u_{i'k'}|^2\\&\quad \le L_1L_2\sum _{i_1'=1}^{I_1}\sum _{i_2'=1}^{I_2}|u_{i'k'}|^2=L_1L_2, \end{aligned} \end{aligned}$$

which implies that

$$\begin{aligned} {\mathfrak {E}}\left( \sum _{i_1=1}^{I_1}\sum _{i_2=1}^{I_2}|u_{ik}|^2 \left| \sum _{i_1'\ne i_1}\sum _{i_2'\ne i_2} d_{i_1'i_1'}^{(1)}d_{i_2'i_2'}^{(2)}u_{i'k'}h^{(1)}_{i_1i_1'} h^{(2)}_{i_2i_2'}\right| ^2\right) \le L_1L_2. \end{aligned}$$

We now bound the second term in the right-hand side of (A.1). Note that we have

$$\begin{aligned} \begin{aligned}&{\mathfrak {E}}\left( \sum _{i_1=1}^{I_1}\sum _{i_2\ne j_2} d_{i_2i_2}^{(2)}{\bar{d}}_{j_2j_2}^{(2)}u_{ik}{\bar{u}}_{jk} \sum _{i_1'\ne i_1}\sum _{i_2'\ne i_2}{\bar{d}}_{i_1'i_1'}^{(1)} {\bar{d}}_{i_2'i_2'}^{(2)}{\bar{u}}_{i'k'}{\bar{h}}^{(1)}_{i_1i_1'} {\bar{h}}^{(2)}_{i_2i_2'}\sum _{j_1'\ne i_1}\sum _{j_2'\ne j_2} d_{j_1'j_1'}^{(1)}d_{j_2'j_2'}^{(2)}u_{j'k'}h^{(1)}_{i_1j_1'} h^{(2)}_{j_2j_2'}\right) \\&\quad ={\mathfrak {E}}\left( \sum _{i_1\ne i_1'}\sum _{i_2\ne j_2} \left| d_{i_1'i_1'}^{(1)}\right| ^2\left( d_{i_2i_2}^{(2)}\right) ^2 \left( {\bar{d}}_{j_2j_2}^{(2)}\right) ^2u_{ik}u_{j'k'}{\bar{u}}_{jk} {\bar{u}}_{i'k'}{\bar{h}}^{(1)}_{i_1i_1'}{\bar{h}}^{(2)}_{i_2j_2} h^{(1)}_{i_1i_1'}h^{(2)}_{j_2i_2}\right) \\&\qquad +\,{\mathfrak {E}}\left( \sum _{i_1\ne i_1'}\sum _{i_2\ne j_2}\sum _{j_2' \ne i_2,j_2}\left| d_{i_1'i_1'}^{(1)}\right| ^2d_{i_2i_2}^{(2)} d_{j_2'j_2'}^{(2)}\left( {\bar{d}}_{j_2j_2}^{(2)}\right) ^2u_{ik}u_{j'k'} {\bar{u}}_{jk}{\bar{u}}_{i'k'}{\bar{h}}^{(1)}_{i_1i_1'} {\bar{h}}^{(2)}_{i_2j_2}h^{(1)}_{i_1i_1'}h^{(2)}_{j_2j_2'}\right) \\&\qquad +\,{\mathfrak {E}}\left( \sum _{i_1\ne i_1'}\sum _{i_2\ne j_2}\sum _{i_2' \ne i_2,j_2}\left| d_{i_1'i_1'}^{(1)}\right| ^2\left( d_{i_2i_2}^{(2)} \right) ^2{\bar{d}}_{i_2'i_2'}^{(2)}{\bar{d}}_{j_2j_2}^{(2)}u_{ik}u_{j'k'} {\bar{u}}_{jk}{\bar{u}}_{i'k'}{\bar{h}}^{(1)}_{i_1i_1'} {\bar{h}}^{(2)}_{i_2i_2'}h^{(1)}_{i_1i_1'}h^{(2)}_{j_2i_2}\right) \\&\qquad +\,{\mathfrak {E}}\left( \sum _{\begin{array}{c} i_1',j_1'\ne i_1\\ i_1'\ne j_1' \end{array}}\sum _{i_2\ne j_2}d_{i_2i_2}^{(2)}{\bar{d}}_{j_2j_2}^{(2)}u_{ik}{\bar{u}}_{jk} \sum _{i_2'\ne i_2,j_2}{\bar{d}}_{i_1'i_1'}^{(1)}{\bar{d}}_{i_2'i_2'}^{(2)} {\bar{u}}_{i'k'}{\bar{h}}^{(1)}_{i_1i_1'}{\bar{h}}^{(2)}_{i_2i_2'} \sum _{j_2'\ne i_2,j_2}d_{j_1'j_1'}^{(1)}d_{j_2'j_2'}^{(2)}u_{j'k'}h^{(1)}_{i_1j_1'} h^{(2)}_{j_2j_2'}\right) . \end{aligned} \end{aligned}$$

By the fact that ${\mathfrak {E}}(d_{i_1i_1}^{(1)})={\mathfrak {E}}(d_{i_2i_2}^{(2)})=0$, $|d_{i_1i_1}^{(1)}|=|d_{i_2i_2}^{(2)}|=1$ and ${\mathfrak {E}}((d_{i_1i_1}^{(1)})^2)={\mathfrak {E}}((d_{i_2i_2}^{(2)})^2) ={\mathfrak {E}}(({\bar{d}}_{i_1i_1}^{(1)})^2)={\mathfrak {E}} (({\bar{d}}_{i_2i_2}^{(2)})^2)=0$, we have

$$\begin{aligned} \begin{aligned}&{\mathfrak {E}}\left( \sum _{i_1=1}^{I_1}\sum _{i_2\ne j_2}d_{i_2i_2}^{(2)} {\bar{d}}_{j_2j_2}^{(2)}u_{ik}{\bar{u}}_{jk}\sum _{i_1'\ne i_1} \sum _{i_2'\ne i_2}{\bar{d}}_{i_1'i_1'}^{(1)}{\bar{d}}_{i_2'i_2'}^{(2)} {\bar{u}}_{i'k'}{\bar{h}}^{(1)}_{i_1i_1'}{\bar{h}}^{(2)}_{i_2i_2'}\right. \\&\left. \quad \quad \sum _{j_1'\ne i_1}\sum _{j_2'\ne j_2}d_{j_1'j_1'}^{(1)}d_{j_2'j_2'}^{(2)}u_{j'k'} h^{(1)}_{i_1j_1'}h^{(2)}_{j_2j_2'}\right) =0. \end{aligned} \end{aligned}$$

Similarly, we can prove that

$$\begin{aligned} \begin{aligned}&{\mathfrak {E}}\left( \sum _{i_1\ne j_1}\sum _{i_2=1}^{I_2}d_{i_1i_1}^{(1)} {\bar{d}}_{j_1j_1}^{(1)}u_{ik}{\bar{u}}_{jk}\sum _{i_1'\ne i_1}\sum _{i_2' \ne i_2}{\bar{d}}_{i_1'i_1'}^{(1)}{\bar{d}}_{i_2'i_2'}^{(2)} {\bar{u}}_{i'k'}{\bar{h}}^{(1)}_{i_1i_1'}{\bar{h}}^{(2)}_{i_2i_2'}\right. \\&\left. \quad \quad \sum _{j_1'\ne j_1}\sum _{j_2'\ne i_2}d_{j_1'j_1'}^{(1)} d_{j_2'j_2'}^{(2)}u_{j'k'}h^{(1)}_{j_1j_1'}h^{(2)}_{i_2j_2'}\right) =0,\\&{\mathfrak {E}}\left( \sum _{i_1\ne j_1}\sum _{i_2\ne j_2}d_{i_1i_1}^{(1)} d_{i_2i_2}^{(2)}{\bar{d}}_{j_1j_1}^{(1)}{\bar{d}}_{j_2j_2}^{(2)}u_{ik} {\bar{u}}_{jk}\sum _{i_1'\ne i_1}\sum _{i_2'\ne i_2} {\bar{d}}_{i_1'i_1'}^{(1)}{\bar{d}}_{i_2'i_2'}^{(2)}{\bar{u}}_{i'k'} {\bar{h}}^{(1)}_{i_1i_1'}{\bar{h}}^{(2)}_{i_2i_2'}\right. \\&\left. \quad \quad \sum _{j_1'\ne j_1}\sum _{j_2'\ne j_2}d_{j_1'j_1'}^{(1)}d_{j_2'j_2'}^{(2)}u_{j'k'}h^{(1)}_{j_1j_1'} h^{(2)}_{j_2j_2'}\right) =0, \end{aligned} \end{aligned}$$

which implies that

$$\begin{aligned} {\mathfrak {E}}|e_{kk'}|^2\le L_1L_2. \end{aligned}$$

Now we have

$$\begin{aligned} {\mathfrak {E}}\Vert {\mathbf {E}}\Vert _F^2={\mathfrak {E}} \sum _{k,k'=1}^K|e_{kk'}|^2\le K^2L_1L_2, \end{aligned}$$

which implies that ${\mathfrak {E}}\Vert {\mathbf {E}}\Vert _2\le {\mathfrak {E}}\Vert {\mathbf {E}}\Vert _F\le \sqrt{K^2L_1L_2}$. By the Markov inequality, we have that

$$\begin{aligned} \Vert {\mathbf {E}}\Vert _2\le \sqrt{\beta K^2L_1L_2}, \end{aligned}$$

holds with probability at least $1-1/\beta$. Hence, the proof of Lemma 4.5 is complete.

Some Remarks for the Existing Algorithms

For Tucker-SVD, Tucker-pSVD, tucker_als, mlsvd, mlsvd_rsi, Adap-Tucker and ran-Tucker, we have the following statements:

• For Tucker-pSVD, the number of subspace iterations to be performed is 1. For Tucker-SVD, Tucker-pSVD, Adap-Tucker and ran-Tucker, the positive integer K is set to 10.

• For tucker_als, the maximum number of iterations is set to 50, the order to loop through dimensions is $\{1,2,3\}$, the entries of initial values are i.i.d. standard Gaussian variables and the tolerance on difference in fit is set to 0.0001.

• For mlsvd, the order to loop through dimensions is $\{1,2,3\}$ and a faster but possibly less accurate eigenvalue decomposition is used to compute the factor matrices.

• For mlsvd_rsi, the oversampling parameter is 10, the number of subspace iterations to be performed is 2 and we remove the parts of the factor matrices and core tensor corresponding due to the oversampling.