Tensor Completion via A Generalized Transformed Tensor T-Product Decomposition Without t-SVD

Abstract

Matrix and tensor nuclear norms have been successfully used to promote the low-rankness of tensors in low-rank tensor completion. However, singular value decomposition (SVD), which is computationally expensive for large-scale matrices, frequently appears in solving those nuclear norm minimization models. Based on the tensor-tensor product (T-product), in this paper, we first establish the equivalence between the so-called transformed tubal nuclear norm for a third-order tensor and the minimum of the sum of two factor tensors’ squared Frobenius norms under a general invertible linear transform. Gainfully, we introduce a mode-unfolding (often named as “spatio-temporal” in the internet traffic data recovery literature) regularized tensor completion model that is able to efficiently exploit the hidden structures of tensors. Then, we propose an implementable alternating minimization algorithm to solve the underlying optimization model. It is remarkable that our approach does not require any SVDs and all subproblems of our algorithm enjoy closed-form solutions. A series of numerical experiments on traffic data recovery, color images and videos inpainting demonstrate that our SVD-free approach takes less computing time to achieve satisfactory accuracy than some state-of-the-art tensor nuclear norm minimization approaches.

A Appendix: Convergence Properties of Algorithm 1

In this appendix, we will show the convergence properties of Algorithm 1. We begin our analysis with introducing the following notations. Throughout this appendix, we use the notation ${\mathbb {E}}:={\mathbb {E}}_1\times {\mathbb {E}}_2\times \cdots \times {\mathbb {E}}_5$ with ${\mathbb {E}}_1={\mathbb {K}}_p^{m\times r}$ , ${\mathbb {E}}_2={\mathbb {K}}_p^{r\times n}$, and ${\mathbb {E}}_i={\mathbb {K}}_p^{m\times n}$ for $i=3,4,5$, and let ${\mathcal {W}}:=({\mathcal {X}},{\mathcal {Y}},{\mathcal {Z}},{\mathcal {U}},{\mathcal {V}})\in {\mathbb {E}}$ and $\varPsi ({\mathcal {W}})=\varPsi ({\mathcal {X}},{\mathcal {Y}},{\mathcal {Z}},{\mathcal {U}},{\mathcal {V}})$. In particular, we denote

where ‘$+{\mathcal {X}}^{q+1}$’ means using ${\mathcal {X}}^{q+1}$ in place of ${\mathcal {X}}^{q}$ in $\varPsi \left( {\mathcal {W}}^{q}\right)$ and ‘$-{\mathcal {X}}^{q}$’ means taking ${\mathcal {X}}^{q}$ instead of ${\mathcal {X}}^{q+1}$ in $\varPsi \left( {\mathcal {W}}^{q+1}\right)$.

Definition A.1

A point ${\mathcal {W}}^{\star }=({\mathcal {X}}^{\star },{\mathcal {Y}}^{\star },{\mathcal {Z}}^{\star },{\mathcal {U}}^{\star },{\mathcal {V}}^{\star })\in {\mathbb {E}}$ is a coordinate-wise minimum point of $\varPsi$, if ${\mathscr {P}}_\varOmega ({\mathcal {Z}}^{\star })={\mathscr {P}}_\varOmega ({\mathcal {G}})$ and

$$\begin{aligned} \varPsi ({\mathcal {W}}^{\star })\le \varPsi \left( {\mathcal {W}}^{\star }+{\mathcal {T}}_i({\mathcal {H}})\right) ~~~~i=1,2,\ldots ,5, \;\; {\mathcal {H}}\in {\mathbb {E}}_i, \end{aligned}$$

where ${\mathcal {T}}_i({\mathcal {H}})=({\varvec{0}},\ldots ,\underbrace{{\mathcal {H}}}_{\mathrm{the}~i\mathrm{th~tensor}},\ldots ,{\varvec{0}})$ for $i=1,2,\ldots ,5$, and in addition, ${\mathscr {P}}_\varOmega ({\mathcal {Z}}^{\star }+{\mathcal {H}})={\mathscr {P}}_\varOmega ({\mathcal {G}})$ when $i=3$.

Consequently, problem (4.4) has at least one minimizer, and for any $\bar{{\mathcal {W}}}\in {\mathbb {E}}$, all subproblems in (4.6)–(4.10) possess unique minimizer, which implies that Algorithm 1 is well-defined.

Proposition A.1

Let $\left\{ {\mathcal {W}}^q\right\}$ be the sequence generated by Algorithm 1. Then $\left\{ {\mathcal {W}}^q\right\}$ is bounded, and any limit point of $\left\{ {\mathcal {W}}^q\right\}$ is a coordinate-wise minimum of (4.4).

Proof

From the special structure of the objective function $\varPsi$, we see that $\varPsi$ has bounded level set, i.e., $\mathrm{Lev}(\varPsi , \alpha )=\{{\mathcal {W}}\in {\mathbb {E}}~|~\varPsi ({\mathcal {W}})\le \alpha \}$ is bounded for any $\alpha \in {\mathbb {R}}_+$. Moreover, by Algorithm 1, the sequence of function values $\{\varPsi \left( {\mathcal {W}}^q\right) \}$ is nonincreasing, which in particular implies that $\left\{ {\mathcal {W}}^q\right\} \subseteq \mathrm{Lev}\left( \varPsi ,\varPsi ({\mathcal {W}}^0)\right)$. Hence, we know that the sequence $\left\{ {\mathcal {W}}^q\right\}$ is bounded.

On the other hand, the strong convexity of every subproblem implies that each one has one unique minimizer. By [2, Theorem 14.3], it follows that any limit point of $\left\{ {\mathcal {W}}^q\right\}$ is a coordinate-wise minimum point of (4.4). $\square$

Theorem A.1

Every coordinate-wise minimum point of (4.4) is its a stationary point.

Proof

Let ${\mathcal {W}}^{\star }$ be a coordinate-wise minimum point of (4.4). Then it follows that

$$\begin{aligned} {\mathcal {X}}^{\star }\in \mathrm{argmin}_{{\mathcal {X}}\in {\mathbb {K}}_p^{m\times n}}{\bar{\varPsi }}_1({\mathcal {X}}), \end{aligned}$$

where ${\bar{\varPsi }}_1({\mathcal {X}}):=\varPsi ({\mathcal {X}},{\mathcal {Y}}^{\star },{\mathcal {Z}}^{\star },{\mathcal {U}}^{\star },{\mathcal {V}}^{\star })$, which implies $\nabla {\bar{\varPsi }}_1({\mathcal {X}}^{\star })=0$. Since $\nabla _{{\mathcal {X}}}{\varPsi }({\mathcal {W}}^{\star })=\nabla {\bar{\varPsi }}_1({\mathcal {X}}^{\star })$, we obtain $\nabla _{{\mathcal {X}}}{\varPsi }({\mathcal {W}}^{\star })=0$. Similarly, we can prove

$$\begin{aligned} \left( \nabla _{{\mathcal {Y}}}{\varPsi }({\mathcal {W}}^{\star }),\nabla _{{\mathcal {U}}}{\varPsi }({\mathcal {W}}^{\star }),\nabla _{{\mathcal {V}}}{\varPsi }({\mathcal {W}}^{\star })\right) =0. \end{aligned}$$

Note that

$$\begin{aligned} {\mathcal {Z}}^{\star }=\arg \min _{{\mathcal {Z}}\in {\mathbb {K}}_p^{m\times n}}{\bar{\varPsi }}_3({\mathcal {Z}}):=\varPsi ({\mathcal {X}}^{\star },{\mathcal {Y}}^{\star },{\mathcal {Z}},{\mathcal {U}}^{\star },{\mathcal {V}}^{\star })+\delta _{{\mathbb {S}}}({\mathcal {Z}}) \end{aligned}$$

with $\delta _{{\mathbb {S}}}(\cdot )$ being an indicator function associated to ${\mathbb {S}}=\{{\mathcal {Z}}\in {\mathbb {K}}_p^{m\times n}~|~{\mathscr {P}}_\varOmega ({\mathcal {Z}})={\mathscr {P}}_\varOmega ({\mathcal {G}})\}$, which implies $-\nabla _{{\mathcal {Z}}}\varPsi ({\mathcal {W}}^{\star })\in \partial \delta _{{\mathbb {S}}}({\mathcal {Z}}^{\star })$. From the definition of the subdifferential $\partial \delta _{{\mathbb {S}}}(\cdot )$, we have $\langle -\nabla _{{\mathcal {Z}}}\varPsi ({\mathcal {W}}^{\star }), {\mathcal {Z}}-{\mathcal {Z}}^{\star }\rangle \le 0$ for any ${\mathcal {Z}}\in S$. Consequently, it follows that $(\nabla _{{\mathcal {Z}}}\varPsi ({\mathcal {W}}^{\star }))_{ijk}=0$ for any $(i,j,k)\not \in \varOmega$, and ${\mathcal {Z}}_{ijk}={\mathcal {G}}_{ijk}$ for any $(i,j,k)\in \varOmega$. Summarizing the above arguments, we know that ${\mathcal {W}}^{\star }$ is a stationary point of (4.4). $\square$

We now recall the well-known descent lemma for smooth functions, e.g., see [5].

Lemma A.1

(Descent lemma) Let $\varphi : {\mathbb {R}}^d\rightarrow {\mathbb {R}}$ be a continuously differentiable function with gradient $\nabla \varphi$ assumed to be $\xi$-Lipschitz continuous. Then,

$$\begin{aligned} \varphi ({\varvec{u}})\le \varphi ({\varvec{v}})+\langle {\varvec{u}}-{\varvec{v}},\nabla \varphi ({\varvec{v}})\rangle +\frac{\xi }{2}\Vert {\varvec{u}}-{\varvec{v}}\Vert ^2,~~~~\forall ~{\varvec{u}},{\varvec{v}}\in {\mathbb {R}}^d. \end{aligned}$$

(A.3)

Proposition A.2

Let $\left\{ {\mathcal {W}}^q\right\}$ be a sequence generated by Algorithm 1. Then, we have

where ${{{\bar{\tau }}}}_1^q={\min }_{1\le k\le p}\left\{ \tau _1^{(q,k)}\sigma ^2_{\min }(L)\right\}$ and ${{{\bar{\tau }}}}_2^q={\min }_{1\le k\le p}\left\{ \tau _2^{(q,k)}\sigma ^2_{\min }(L)\right\}$ with

Proof

We only prove (A.4), and inequality (A.5) can be proved similarly. It follows from the definition of $\varPsi ({\mathcal {W}})$ in (4.5) and notation $(\varPhi _L({\mathcal {X}}^q))^{(k)}=(\widehat{{\mathcal {X}}}^q)^{(k)}$ that

$$\begin{aligned} \varPsi \left( {\mathcal {W}}^{q}\right) -\varPsi ({\mathcal {W}}^{q},+{\mathcal {X}}^{q+1}) = \sum _{k=1}^p\left( h^q_k\left( (\widehat{{\mathcal {X}}}^{q})^{(k)}\right) -h^q_k\left( (\widehat{{\mathcal {X}}}^{q+1})^{(k)}\right) \right) , \end{aligned}$$

(A.6)

where $h^q_k(\cdot )$ is given by (4.13). It is trivial to see that the gradient $\nabla h^q_k(\cdot )$ of $h^q_k(\cdot )$ is $\xi _{1}^{(q,k)}$-Lipschitz continuous, where $\xi _{1}^{(q,k)}:=\rho +\sigma ^2_{\max }((\widehat{{\mathcal {Y}}}^{q})^{(k)})$. Then, applying Lemma A.1 to $h^q_k(\cdot )$ immediately yields

$$\begin{aligned} h^q_k\left( (\widehat{{\mathcal {X}}}^{q})^{(k)}\right) -h^q_k\left( (\widehat{{\mathcal {X}}}^{q+1})^{(k)}\right) \ge&-\left\langle \nabla h^q_k\left( (\widehat{{\mathcal {X}}}^{q})^{(k)}\right) , (\widehat{{\mathcal {X}}}^{q+1})^{(k)}-(\widehat{{\mathcal {X}}}^{q})^{(k)}\right\rangle \nonumber \\&\; -\frac{\xi _{1}^{(q,k)}}{2}\left\| (\widehat{{\mathcal {X}}}^{q+1})^{(k)}-(\widehat{{\mathcal {X}}}^{q})^{(k)}\right\| _F^2. \end{aligned}$$

(A.7)

By invoking the updating scheme of $(\widehat{{\mathcal {X}}}^{q+1})^{(k)}$, we have

$$\begin{aligned} -\nabla h^q_k\left( (\widehat{{\mathcal {X}}}^{q})^{(k)}\right) =\left( (\widehat{{\mathcal {X}}}^{q+1})^{(k)}-(\widehat{{\mathcal {X}}}^{q})^{(k)}\right) \left( (\widehat{{\mathcal {Y}}}^{q})^{(k)}\left( (\widehat{{\mathcal {Y}}}^{q})^{(k)}\right) ^* +\rho I\right) . \end{aligned}$$

Consequently, it holds that

$$\begin{aligned}&-\left\langle \nabla h^q_k\left( (\widehat{{\mathcal {X}}}^{q})^{(k)}\right) , (\widehat{{\mathcal {X}}}^{q+1})^{(k)}-(\widehat{{\mathcal {X}}}^{q})^{(k)}\right\rangle \nonumber \\&\quad =\left\langle \left( (\widehat{{\mathcal {X}}}^{q+1})^{(k)}-(\widehat{{\mathcal {X}}}^{q})^{(k)}\right) \left( (\widehat{{\mathcal {Y}}}^{q})^{(k)}\left( (\widehat{{\mathcal {Y}}}^{q})^{(k)}\right) ^* +\rho I\right) , \; (\widehat{{\mathcal {X}}}^{q+1})^{(k)}-(\widehat{{\mathcal {X}}}^{q})^{(k)} \right\rangle \nonumber \\&\quad \ge \left( \rho +\sigma ^2_{\min }\left( (\widehat{{\mathcal {Y}}}^q)^{(k)}\right) \right) \left\| (\widehat{{\mathcal {X}}}^{q+1})^{(k)}-(\widehat{{\mathcal {X}}}^{q})^{(k)}\right\| _F^2, \end{aligned}$$

which, together with (A.7), implies that

$$\begin{aligned} h^q_k\left( (\widehat{{\mathcal {X}}}^{q})^{(k)}\right) -h^q_k\left( (\widehat{{\mathcal {X}}}^{q+1})^{(k)}\right)&\ge \tau _1^{(q,k)}\left\| (\widehat{{\mathcal {X}}}^{q+1})^{(k)}-(\widehat{{\mathcal {X}}}^{q})^{(k)}\right\| _F^2 \nonumber \\&\ge \tau _1^{(q,k)}\sigma _{\min }^2(L)\left\| ({{\mathcal {X}}}^{q+1})^{(k)}-({{\mathcal {X}}}^{q})^{(k)}\right\| _F^2. \end{aligned}$$

(A.8)

We then conclude from (A.6) and (A.8) that

$$\begin{aligned} \varPsi \left( {\mathcal {W}}^{q}\right) -\varPsi ({\mathcal {W}}^{q},+{\mathcal {X}}^{q+1})&\ge \sum _{k=1}^p\tau _1^{(q,k)}\left\| (\widehat{{\mathcal {X}}}^{q+1})^{(k)}-(\widehat{{\mathcal {X}}}^{q})^{(k)}\right\| _F^2 \nonumber \\&\ge {{{\bar{\tau }}}}_1^q\sum _{k=1}^p\left\| ({{\mathcal {X}}}^{q+1})^{(k)}-({{\mathcal {X}}}^{q})^{(k)}\right\| _F^2 \nonumber \\&={{{\bar{\tau }}}}_1^q \left\| {{\mathcal {X}}}^{q+1}-{{\mathcal {X}}}^{q}\right\| _F^2. \end{aligned}$$

We obtain the desired results and complete the proof. $\square$

Proposition A.3

Let $\left\{ {\mathcal {W}}^q\right\}$ be a sequence generated by Algorithm 1. Then, we have

$$\begin{aligned} \varPsi \left( {\mathcal {W}}^{q+1},-{\mathcal {Z}}^{q},-{\mathcal {U}}^{q},-{\mathcal {V}}^{q}\right) -\varPsi \left( {\mathcal {W}}^{q+1},-{\mathcal {U}}^{q},-{\mathcal {V}}^{q}\right) \ge {{\bar{\tau }}}_{3}\left\| {\mathcal {Z}}^{q+1}-{\mathcal {Z}}^q\right\| _F^2, \end{aligned}$$

(A.9)

where ${{\bar{\tau }}}_{3}=(1+\beta _1+\beta _2)/2$.

Proof

Let us divide the tensor ${\mathcal {Z}}$ into two parts, denoted as ${\mathcal {Z}}_{\varOmega }$ and ${\mathcal {Z}}_{\varOmega ^c}$ respectively, where $\varOmega ^c$ is the complement of $\varOmega$, and the elements in ${\mathcal {Z}}_{\varOmega }$ are composed of elements in ${\mathcal {G}}_{\varOmega }$, i.e., ${\mathcal {Z}}_{\varOmega }={\mathcal {G}}_{\varOmega }$. Accordingly, denote

$$\begin{aligned} f^q({\mathcal {Z}}_{\varOmega ^c}) =\frac{1}{2}\left\| ({\mathcal {X}}^{q+1}\circledast _{L}{\mathcal {Y}}^{q+1})_{\varOmega ^c}-{\mathcal {Z}}_{\varOmega ^c}\right\| _F^2+\frac{\beta _1}{2}\left\| {\mathcal {Z}}_{\varOmega ^c}-{\mathcal {U}}^{q}_{\varOmega ^c}\right\| _F^2+\frac{\beta _2}{2}\left\| {\mathcal {Z}}_{\varOmega ^c}-{\mathcal {V}}^{q}_{\varOmega ^c}\right\| _F^2. \end{aligned}$$

It is easy to see that

$$\begin{aligned} \varPsi \left( {\mathcal {W}}^{q+1},-{\mathcal {Z}}^{q},-{\mathcal {U}}^{q},-{\mathcal {V}}^{q}\right) -\varPsi \left( {\mathcal {W}}^{q+1},-{\mathcal {U}}^{q},-{\mathcal {V}}^{q}\right) =f^q({\mathcal {Z}}^q_{\varOmega ^c})-f^q({\mathcal {Z}}^{q+1}_{\varOmega ^c}). \end{aligned}$$

Moreover, it can be easily seen that $\nabla f^q(\cdot )$ is Lipschitz continuous with constant $(1+\beta _1+\beta _2)$. By $-\nabla f^q({\mathcal {Z}}^q_{\varOmega ^c})=(1+\beta _1+\beta _2)\left( {\mathcal {Z}}^{q+1}_{\varOmega ^c}-{\mathcal {Z}}^{q}_{\varOmega ^c}\right)$, an application of Lemma A.1 leads to

$$\begin{aligned} f^q({\mathcal {Z}}^q_{\varOmega ^c})-f^q({\mathcal {Z}}^{q+1}_{\varOmega ^c})\ge {{\bar{\tau }}}_{3}\left\| {\mathcal {Z}}_{\varOmega ^c}^{q+1}-{\mathcal {Z}}_{\varOmega ^c}^q\right\| _F^2, \end{aligned}$$

which implies that (A.9) holds, since $\left\| {\mathcal {Z}}_{\varOmega ^c}^{q+1}-{\mathcal {Z}}_{\varOmega ^c}^q\right\| _F^2=\left\| {\mathcal {Z}}^{q+1}-{\mathcal {Z}}^q\right\| _F^2$. The proof is completed. $\square$

Proposition A.4

Let $\left\{ {\mathcal {W}}^q\right\}$ be a sequence generated by Algorithm 1. Then, we have

where

Proof

We only prove (A.10). The inequality (A.11) can be proved in a similar way. First, it is easy to see that

$$\begin{aligned}&\varPsi ({\mathcal {W}}^{q+1},-{\mathcal {U}}^{q},-{\mathcal {V}}^{q})-\varPsi ({\mathcal {W}}^{q+1},-{\mathcal {V}}^{q}) \nonumber \\&\quad =\frac{1}{2}\left\{ \mu _1\left\| H{\mathcal {U}}^q_{[1]}\right\| _F^2+\beta _1\left\| {\mathcal {Z}}^{q+1}-{\mathcal {U}}^{q}\right\| _F^2\right\} -\frac{1}{2}\left\{ \mu _1\left\| H{\mathcal {U}}^{q+1}_{[1]}\right\| _F^2+\beta _1\left\| {\mathcal {Z}}^{q+1}-{\mathcal {U}}^{q+1}\right\| _F^2\right\} \nonumber \\&\quad =\frac{1}{2}\sum _{k=1}^p\left( \mu _1\left\| H({\mathcal {U}}^q)^{(k)}\right\| _F^2+\beta _1\left\| ({\mathcal {Z}}^{q+1})^{(k)}-({\mathcal {U}}^{q})^{(k)}\right\| _F^2\right) \nonumber \\&\quad \quad -\frac{1}{2}\sum _{k=1}^p\left( \mu _1\left\| H({\mathcal {U}}^{q+1})^{(k)}\right\| _F^2+\beta _1\left\| ({\mathcal {Z}}^{q+1})^{(k)}-({\mathcal {U}}^{q+1})^{(k)}\right\| _F^2\right) , \end{aligned}$$

(A.12)

where the last equality comes from (4.2) and $\Vert {\mathcal {A}}\Vert ^2_F=\sum _{k=1}^p\Vert {\mathcal {A}}^{(k)}\Vert _F^2$ for any ${\mathcal {A}}\in {\mathbb {K}}_p^{m\times n}$. Let

$$\begin{aligned} g_k^q(U)=\frac{1}{2}\left( \mu _1\left\| HU\right\| _F^2+\beta _1\left\| ({\mathcal {Z}}^{q+1})^{(k)}-U\right\| _F^2\right) ,~~~U\in {\mathbb {R}}^{m\times n}. \end{aligned}$$

By the iterative scheme of $({\mathcal {U}}^{q+1})^{(k)}$, it is obvious that the gradient of $g_k^q(\cdot )$ at $({\mathcal {U}}^q)^{(k)}$ reads as

$$\begin{aligned} \nabla g_k^q\left( ({\mathcal {U}}^q)^{(k)}\right) =-\left( \beta _1 I+\mu _1H^* H\right) \left( ({\mathcal {U}}^{q+1})^{(k)}-({\mathcal {U}}^{q})^{(k)}\right) . \end{aligned}$$

Consequently, it holds that

$$\begin{aligned} \left\langle \nabla g_k^q\left( ({\mathcal {U}}^q)^{(k)}\right) ,({\mathcal {U}}^{q})^{(k)}-({\mathcal {U}}^{q+1})^{(k)}\right\rangle \ge \left( \beta _1 +\mu _1\sigma ^2_{\min }(H)\right) \left\| ({\mathcal {U}}^{q+1})^{(k)}-({\mathcal {U}}^{q})^{(k)}\right\| _F^2.\nonumber \\ \end{aligned}$$

(A.13)

By using the Lipschitz continuity of $\nabla g_k^q(\cdot )$, we have

$$\begin{aligned} \left\| \nabla g_k^q\left( ({\mathcal {U}}^q)^{(k)}\right) -\nabla g_k^q\left( ({\mathcal {U}}^{q+1})^{(k)}\right) \right\| _F\le \left( \beta _1+\mu _1\sigma _{\max }^2(H)\right) \left\| ({\mathcal {U}}^q)^{(k)}-({\mathcal {U}}^{q+1})^{(k)}\right\| _F, \end{aligned}$$

Applying Lemma A.1 to $g_k^q(\cdot )$ and combining with (A.13) immediately yields

$$\begin{aligned}&g^q_k\left( ({\mathcal {U}}^{q})^{(k)}\right) -g^q_k\left( ({\mathcal {U}}^{q+1})^{(k)}\right) \\&\quad \ge \left\langle \nabla g_k^q\left( ({\mathcal {U}}^q)^{(k)}\right) ,({\mathcal {U}}^{q})^{(k)}-({\mathcal {U}}^{q+1})^{(k)}\right\rangle -\frac{\beta _1+\mu _1\sigma _{\max }^2(H)}{2}\left\| ({\mathcal {U}}^{q+1})^{(k)}-({\mathcal {U}}^{q})^{(k)}\right\| _F^2 \\&\quad \ge \bar{\tau }_4\left\| ({\mathcal {U}}^{q+1})^{(k)}-({\mathcal {U}}^{q})^{(k)}\right\| _F^2, \end{aligned}$$

which, together with (A.12), immediately implies that

$$\begin{aligned} \varPsi ({\mathcal {W}}^{q+1},-{\mathcal {U}}^{q},-{\mathcal {V}}^{q})-\varPsi ({\mathcal {W}}^{q+1},-{\mathcal {V}}^{q})&=\sum _{k=1}^pg^q_k\left( ({\mathcal {U}}^{q})^{(k)}\right) -g^q_k\left( ({\mathcal {U}}^{q+1})^{(k)}\right) \\&\ge {{\bar{\tau }}}_4\sum _{k=1}^p\left\| ({\mathcal {U}}^{q+1})^{(k)}-({\mathcal {U}}^{q})^{(k)}\right\| _F^2 \\&={{\bar{\tau }}}_{4}\left\| {\mathcal {U}}^{q+1}-{\mathcal {U}}^{q}\right\| _F^2. \end{aligned}$$

We obtain the desired results and complete the proof. $\square$

Theorem A.2

(Sufficient decrease property) Let $\left\{ {\mathcal {W}}^q\right\}$ be a sequence generated by Algorithm 1. Then, the sequence $\{\varPsi \left( {\mathcal {W}}^q\right) \}$ is nonincreasing. In particular, if there exists a real number ${{\bar{\kappa }}}>0$ such that

$$\begin{aligned} {\min }\left\{ {\min }\left\{ {{\bar{\tau }}}_{1}^q~|~q=1,2,\ldots \right\} , {\min }\left\{ {{\bar{\tau }}}_{2}^q~|~q=1,2,\ldots \right\} ,{{\bar{\tau }}}_3,{{\bar{\tau }}}_4,{{\bar{\tau }}}_5\right\} \ge {{\bar{\kappa }}}, \end{aligned}$$

then

$$\begin{aligned} \varPsi \left( {\mathcal {W}}^{q+1}\right) +\bar{\kappa }\left\| {\mathcal {W}}^{q+1}-{\mathcal {W}}^q\right\| _F^2\le \varPsi \left( {\mathcal {W}}^{q+1}\right) . \end{aligned}$$

(A.14)

Proof

By Propositions A.2, A.4 and A.3, we have

$$\begin{aligned} \varPsi \left( {\mathcal {W}}^{q}\right) -\varPsi \left( {\mathcal {W}}^{q+1}\right)&\ge {{{\bar{\tau }}}}_1^q\left\| {\mathcal {X}}^{q+1}-{\mathcal {X}}^q\right\| _F^2+{{{\bar{\tau }}}}_2^q\left\| {\mathcal {Y}}^{q+1}-{\mathcal {Y}}^q\right\| _F^2+{{\bar{\tau }}}_{3}\left\| {\mathcal {Z}}^{q+1}-{\mathcal {Z}}^q\right\| _F^2 \nonumber \\&\quad +{{\bar{\tau }}}_4\left\| {\mathcal {U}}^{q+1}-{\mathcal {U}}^q\right\| _F^2+{{\bar{\tau }}}_5\left\| {\mathcal {V}}^{q+1}-{\mathcal {V}}^q\right\| _F^2 \nonumber \\&\ge 0, \end{aligned}$$

(A.15)

which implies that the sequence $\left\{ \varPsi \left( {\mathcal {W}}^q\right) \right\}$ is nonincreasing. As a consequence, combining the given condition on ${{\bar{\kappa }}}$ and (A.15) immediately leads to

$$\begin{aligned} \varPsi \left( {\mathcal {W}}^{q}\right) -\varPsi \left( {\mathcal {W}}^{q+1}\right)&\ge {{\bar{\kappa }}}\Big \{\left\| {\mathcal {X}}^{q+1}-{\mathcal {X}}^q\right\| _F^2+\left\| {\mathcal {Y}}^{q+1}-{\mathcal {Y}}^q\right\| _F^2+\left\| {\mathcal {Z}}^{q+1}-{\mathcal {Z}}^q\right\| _F^2\\&\quad +\left\| {\mathcal {U}}^{q+1}-{\mathcal {U}}^q\right\| _F^2+\left\| {\mathcal {V}}^{q+1}-{\mathcal {V}}^q\right\| _F^2\Big \}\\&={{\bar{\kappa }}}\left\| {\mathcal {W}}^{q+1}-{\mathcal {W}}^q\right\| _F^2, \end{aligned}$$

which means the desired result follows.$\square$

Proposition A.5

Let $\left\{ {\mathcal {W}}^q\right\}$ be a sequence generated by Algorithm 1. Then there exists $c>0$ such that for every q, we have

$$\begin{aligned} \left\| \nabla _{{\mathcal {X}}}\varPsi \left( {\mathcal {W}}^{q+1}\right) \right\| _F\le {\hat{c}}c\big \Vert {\mathcal {Z}}^{q+1}-{\mathcal {Z}}^q\big \Vert _F+{\hat{c}}c(2c^2+1)\big \Vert {\mathcal {Y}}^{q+1}-{\mathcal {Y}}^{q}\big \Vert _F \end{aligned}$$

(6.16)

and

$$\begin{aligned} \left\| \nabla _{{\mathcal {Y}}}\varPsi \left( {\mathcal {W}}^{q+1}\right) \right\| _F\le {\hat{c}}c\left\| {\mathcal {Z}}^{q+1}-{\mathcal {Z}}^q\right\| _F, \end{aligned}$$

(6.17)

where $c={\min }\left\{ \Vert {\mathcal {W}}^q\Vert _F~|~q=1,2,\ldots \right\}$ and ${\hat{c}}=\sigma ^2_{\max }(L)\max _{1\le k\le p}\{\vartheta _k\}$ is a constant.

Proof

We first prove (6.16). With notation $\widehat{{\mathcal {A}}}:=\varPhi _L({\mathcal {A}})$ and (4.11), we let

$$\begin{aligned} \varGamma _k({\mathcal {X}},{\mathcal {Y}},{\mathcal {Z}})=\frac{1}{2}\left\| \widehat{{\mathcal {X}}}^{(k)}\widehat{{\mathcal {Y}}}^{(k)}-\widehat{{\mathcal {Z}}}^{(k)}\right\| _F^2 +\frac{\rho }{2}\left( \left\| \widehat{{\mathcal {X}}}^{(k)}\right\| _F^2+\left\| \widehat{{\mathcal {Y}}}^{(k)}\right\| _F^2\right) . \end{aligned}$$

Then we have

$$\begin{aligned} \nabla _{{\mathcal {X}}}\varPsi ({\mathcal {W}})=\sum _{k=1}^p\nabla _{{\mathcal {X}}}\varGamma _k({\mathcal {X}},{\mathcal {Y}},{\mathcal {Z}}) \end{aligned}$$

(6.18)

By a direct computation, we obtain

$$\begin{aligned} \nabla _{{\mathcal {X}}}\varGamma _k({\mathcal {X}},{\mathcal {Y}},{\mathcal {Z}}) =\left( \widehat{{\mathcal {Y}}}^{(k)}\left( \widehat{{\mathcal {Y}}}^{(k)}\right) ^* +\rho I\right) \bullet {\mathscr {M}}_{L_k}(\mathbf{x}) -\left( \widehat{{\mathcal {Z}}}^{(k)}\left( \widehat{{\mathcal {Y}}}^{(k)}\right) ^* \right) \cdot \mathrm{tube}\left( L_{k\cdot }^* \right) , \end{aligned}$$

(6.19)

where throughout $L_{k\cdot }$ represents the k-th row of matrix L and

$$\begin{aligned} {\mathscr {M}}_{L_k}(\mathbf{x})= \left[ \begin{array}{ccc} \mathrm{tube}\left( L_{k\cdot }^* L_{k\cdot }\mathrm{vec}(\mathbf{x}_{11})\right) \;&{}\cdots &{}\;\mathrm{tube}\left( L_{k\cdot }^* L_{k\cdot }\mathrm{vec}(\mathbf{x}_{1r})\right) \\ \mathrm{tube}\left( L_{k\cdot }^* L_{k\cdot }\mathrm{vec}(\mathbf{x}_{21})\right) \;&{}\cdots &{}\;\mathrm{tube}\left( L_{k\cdot }^* L_{k\cdot }\mathrm{vec}(\mathbf{x}_{2r})\right) \\ \vdots &{}\vdots &{}\vdots \\ \mathrm{tube}\left( L_{k\cdot }^* L_{k\cdot }\mathrm{vec}(\mathbf{x}_{m1})\right) \;&{}\cdots &{}\;\mathrm{tube}\left( L_{k\cdot }^* L_{k\cdot }\mathrm{vec}(\mathbf{x}_{mr})\right) \\ \end{array} \right] . \end{aligned}$$

Consequently, it follows from (6.19) that

$$\begin{aligned}&\left\| \nabla _{{\mathcal {X}}}\varGamma _k({\mathcal {X}}^{q+1},{\mathcal {Y}}^{q+1},{\mathcal {Z}}^{q+1})-\nabla _{{\mathcal {X}}}\varGamma _k({\mathcal {X}}^{q+1},{\mathcal {Y}}^q,{\mathcal {Z}}^q)\right\| _F \nonumber \\&\quad \le \left\| \nabla _{{\mathcal {X}}}\varGamma _k({\mathcal {X}}^{q+1},{\mathcal {Y}}^{q+1},{\mathcal {Z}}^{q+1})-\nabla _{{\mathcal {X}}}\varGamma _k({\mathcal {X}}^{q+1},{\mathcal {Y}}^{q+1},{\mathcal {Z}}^q)\right\| _F \nonumber \\&\quad ~~+\left\| \nabla _{{\mathcal {X}}}\varGamma _k({\mathcal {X}}^{q+1},{\mathcal {Y}}^{q+1},{\mathcal {Z}}^{q})-\nabla _{{\mathcal {X}}}\varGamma _k({\mathcal {X}}^{q+1},{\mathcal {Y}}^q,{\mathcal {Z}}^q)\right\| _F \nonumber \\&\quad \le \left\| (\widehat{{\mathcal {Y}}}^{q+1})^{(k)}\right\| _F \left\| (\widehat{{\mathcal {Z}}}^{q+1})^{(k)}-(\widehat{{\mathcal {Z}}}^{q})^{(k)}\right\| _F \left\| \mathrm{tube}\left( L_{k\cdot }^* \right) \right\| \nonumber \\&\quad ~~+\left\| (\widehat{{\mathcal {Y}}}^{q+1})^{(k)}\left( (\widehat{{\mathcal {Y}}}^{q+1})^{(k)}\right) ^* -(\widehat{{\mathcal {Y}}}^q)^{(k)}\left( (\widehat{{\mathcal {Y}}}^q)^{(k)}\right) ^* \right\| _F \left\| {\mathscr {M}}_{L_k}(\mathbf{x}^{q+1})\right\| _F \nonumber \\&\quad ~~+\left\| \mathrm{tube}\left( L_{k\cdot }^* \right) \right\| \left\| (\widehat{{\mathcal {Z}}}^q)^{(k)}\right\| _F \left\| (\widehat{{\mathcal {Y}}}^{q+1})^{(k)}-(\widehat{{\mathcal {Y}}}^q)^{(k)}\right\| _F. \end{aligned}$$

(6.20)

By the invertible transform L, we have

$$\begin{aligned} \sum _{k=1}^p\left\| (\widehat{{\mathcal {Y}}}^{q+1})^{(k)}\right\| _F \le \sigma _{\max }(L)\left\| {\mathcal {Y}}^{q+1}\right\| _F~\mathrm{and}~\sum _{k=1}^p\left\| (\widehat{{\mathcal {Z}}}^{q})^{(k)}\right\| _F \le \sigma _{\max }(L)\left\| {\mathcal {Z}}^{q}\right\| _F.\nonumber \\ \end{aligned}$$

(6.21)

Especially, when L is a unitary matrix, $\sigma _{\max }(L)=1$ and the equality holds. Let $\vartheta _k:=\big \Vert \mathrm{tube}(L_{k\cdot }^* )\big \Vert$ for $k=1,2,\ldots , p$. Consequently, for $i=1,2,\ldots ,m$ and $l=1,2,\ldots ,r$, we have

$$\begin{aligned} \left\| \mathrm{tube}\left( L_{k\cdot }^* L_{k\cdot }\mathrm{vec}(\mathbf{x}^{q+1}_{il})\right) \right\| ^2=\big \Vert L_{k\cdot }^* L_{k\cdot }\mathrm{vec}(\mathbf{x}^{q+1}_{il})\big \Vert ^2 \le \vartheta _k^2\big \Vert \mathbf{x}^{q+1}_{il}\big \Vert ^2, \end{aligned}$$

which implies

$$\begin{aligned} \left\| {\mathscr {M}}_{L_k}(\mathbf{x}^{q+1})\right\| _F^2 =\sum _{i,l}\left\| \mathrm{tube}\left( L_{k\cdot }^* L_{k\cdot }\mathrm{vec}(\mathbf{x}^{q+1}_{il})\right) \right\| ^2 \le \sum _{i,l}\vartheta _k^2\left\| \mathbf{x}^{q+1}_{il}\right\| ^2 =\vartheta _k^2\left\| {\mathcal {X}}^{q+1}\right\| _F^2. \end{aligned}$$

(6.22)

Recall the fact that

$$\begin{aligned} {\mathcal {X}}^{q+1}= \arg \min _{{\mathcal {X}}\in {\mathbb {K}}_p^{m\times n}}\varPsi ({\mathcal {X}},{\mathcal {Y}}^q,{\mathcal {Z}}^q,{\mathcal {U}}^q,{\mathcal {V}}^q), \end{aligned}$$

which implies $\nabla _{{\mathcal {X}}}\varPsi ({\mathcal {X}}^{q+1}, {\mathcal {Y}}^q,{\mathcal {Z}}^q,{\mathcal {U}}^q,{\mathcal {V}}^q)=0$. Consequently, by (6.18), (6.20), (6.21) and (6.22), it holds that

$$\begin{aligned}&\left\| \nabla _{{\mathcal {X}}}\varPsi \left( {\mathcal {W}}^{q+1}\right) \right\| _F \nonumber \\&\quad =\left\| \nabla _{{\mathcal {X}}}\varPsi \left( {\mathcal {W}}^{q+1}\right) -\nabla _{{\mathcal {X}}}\varPsi ({\mathcal {X}}^{q+1}, {\mathcal {Y}}^q,{\mathcal {Z}}^q,{\mathcal {U}}^q,{\mathcal {V}}^q)\right\| _F \nonumber \\&\quad \le \sum _{k=1}^p\left\| \nabla _{{\mathcal {X}}}\varGamma _k({\mathcal {X}}^{q+1},{\mathcal {Y}}^{q+1},{\mathcal {Z}}^{q+1})-\nabla _{{\mathcal {X}}}\varGamma _k({\mathcal {X}}^{q+1},{\mathcal {Y}}^q,{\mathcal {Z}}^q)\right\| _F \nonumber \\&\quad \le \sum _{k=1}^p \vartheta _k\left\| (\widehat{{\mathcal {Y}}}^{q+1})^{(k)}\right\| _F \left\| (\widehat{{\mathcal {Z}}}^{q+1})^{(k)}-(\widehat{{\mathcal {Z}}}^{q})^{(k)}\right\| _F \nonumber \\&\qquad +\left\| {\mathcal {X}}^{q+1}\right\| _F\sum _{k=1}^p\vartheta _k\left\| (\widehat{{\mathcal {Y}}}^{q+1})^{(k)}\left( (\widehat{{\mathcal {Y}}}^{q+1})^{(k)}\right) ^* -(\widehat{{\mathcal {Y}}}^q)^{(k)}\left( (\widehat{{\mathcal {Y}}}^q)^{(k)}\right) ^* \right\| _F \nonumber \\&\qquad +\sum _{k=1}^p\vartheta _k\left\| (\widehat{{\mathcal {Z}}}^q)^{(k)}\right\| _F\left\| (\widehat{{\mathcal {Y}}}^{q+1})^{(k)}-(\widehat{{\mathcal {Y}}}^q)^{(k)}\right\| _F \nonumber \\&\quad \le \sigma _{\max }(L)\left\| {\mathcal {Y}}^{q+1}\right\| _F \sum _{k=1}^p\vartheta _k\left\| (\widehat{{\mathcal {Z}}}^{q+1})^{(k)}-(\widehat{{\mathcal {Z}}}^{q})^{(k)}\right\| _F \nonumber \\&\qquad +\sigma _{\max }(L)\left\| {\mathcal {X}}^{q+1}\right\| _F\left( \big \Vert {\mathcal {Y}}^{q+1}\big \Vert _F+\big \Vert {\mathcal {Y}}^{q}\big \Vert _F\right) \sum _{k=1}^p\vartheta _k\left\| (\widehat{{\mathcal {Y}}}^{q+1})^{(k)}-(\widehat{{\mathcal {Y}}}^q)^{(k)}\right\| _F \nonumber \\&\qquad +\sigma _{\max }(L)\left\| {\mathcal {Z}}^{q}\right\| _F\sum _{k=1}^p \vartheta _k\left\| (\widehat{{\mathcal {Y}}}^{q+1})^{(k)}-(\widehat{{\mathcal {Y}}}^q)^{(k)}\right\| _F \nonumber \\&\quad \le \sigma _{\max }^2(L)\max _{1\le k\le p}\{\vartheta _k\} \left\{ \big \Vert {\mathcal {X}}^{q+1}\big \Vert _F\left( \big \Vert {\mathcal {Y}}^{q+1}\big \Vert _F+\big \Vert {\mathcal {Y}}^{q}\big \Vert _F\right) \left\| {\mathcal {Y}}^{q+1}-{\mathcal {Y}}^{q}\right\| _F\right. \nonumber \\&\qquad +\left. \left\| {\mathcal {Y}}^{q+1}\right\| _F \left\| {\mathcal {Z}}^{q+1}-{\mathcal {Z}}^q\right\| _F+\left\| {\mathcal {Z}}^{q}\right\| _F\left\| {\mathcal {Y}}^{q+1}-{\mathcal {Y}}^{q}\right\| _F\right\} \nonumber \\&\quad \le {\hat{c}}c\left\| {\mathcal {Z}}^{q+1}-{\mathcal {Z}}^q\big \Vert _F+{\hat{c}}c(2c^2+1)\big \Vert {\mathcal {Y}}^{q+1}-{\mathcal {Y}}^{q}\right\| _F. \end{aligned}$$

(6.23)

We obtain the desired result (6.16).

Now we prove (6.17). Similarly, we have

$$\begin{aligned} \nabla _{{\mathcal {Y}}}\varGamma _k({\mathcal {X}},{\mathcal {Y}},{\mathcal {Z}}) =\left( \left( \widehat{{\mathcal {X}}}^{(k)}\right) ^* \widehat{{\mathcal {X}}}^{(k)}+\rho I\right) \bullet {\mathscr {N}}_{L_k}(\mathbf{y}) -\left( \left( \widehat{{\mathcal {X}}}^{(k)}\right) ^* \widehat{{\mathcal {Z}}}^{(k)}\right) \cdot \mathrm{tube}\left( L_{k\cdot }^* \right) \nonumber \\ \end{aligned}$$

(6.24)

with

$$\begin{aligned} {\mathscr {N}}_{L_k}(\mathbf{y})=\left[ \begin{array}{ccc} \mathrm{tube}\left( L_{k\cdot }^* L_{k\cdot }\mathrm{vec}(\mathbf{y}_{11})\right) &{}\cdots &{}\mathrm{tube}\left( L_{k\cdot }^* L_{k\cdot }\mathrm{vec}(\mathbf{y}_{1n})\right) \\ \mathrm{tube}\left( L_{k\cdot }^* L_{k\cdot }\mathrm{vec}(\mathbf{y}_{21})\right) &{}\cdots &{}\mathrm{tube}\left( L_{k\cdot }^* L_{k\cdot }\mathrm{vec}(\mathbf{y}_{2n})\right) \\ \vdots &{}\vdots &{}\vdots \\ \mathrm{tube}\left( L_{k\cdot }^* L_{k\cdot }\mathrm{vec}(\mathbf{y}_{r1})\right) &{}\cdots &{}\mathrm{tube}\left( L_{k\cdot }^* L_{k\cdot }\mathrm{vec}(\mathbf{y}_{rn})\right) \\ \end{array} \right] , \end{aligned}$$

which implies

$$\begin{aligned}&\left\| \nabla _{{\mathcal {Y}}}\varGamma _k({\mathcal {X}}^{q+1},{\mathcal {Y}}^{q+1},{\mathcal {Z}}^{q+1})-\nabla _{{\mathcal {Y}}}\varGamma _k({\mathcal {X}}^{q+1},{\mathcal {Y}}^{q+1},{\mathcal {Z}}^q)\right\| _F \nonumber \\&\quad \le \left\| \mathrm{tube}\left( L_{k\cdot }^* \right) \right\| \left\| (\widehat{{\mathcal {X}}}^{q+1})^{(k)}\right\| _F \left\| (\widehat{{\mathcal {Z}}}^{q+1})^{(k)}-(\widehat{{\mathcal {Z}}}^q)^{(k)}\right\| _F \nonumber \\&\quad =\vartheta _k\left\| (\widehat{{\mathcal {X}}}^{q+1})^{(k)}\right\| _F \left\| (\widehat{{\mathcal {Z}}}^{q+1})^{(k)}-(\widehat{{\mathcal {Z}}}^q)^{(k)}\right\| _F. \end{aligned}$$

(6.25)

Similar to the optimal point ${\mathcal {X}}^{q+1}$, we also have $\nabla _{{\mathcal {Y}}}\varPsi ({\mathcal {X}}^{q+1}, {\mathcal {Y}}^{q+1},{\mathcal {Z}}^q,{\mathcal {U}}^q,{\mathcal {V}}^q)=0$. Consequently, by (6.25), we have

$$\begin{aligned} \left\| \nabla _{{\mathcal {Y}}}\varPsi \left( {\mathcal {W}}^{q+1}\right) \right\| _F&\le \sum _{k=1}^p\left\| \nabla _{{\mathcal {Y}}}\varGamma _k({\mathcal {X}}^{q+1},{\mathcal {Y}}^{q+1},{\mathcal {Z}}^{q+1})-\nabla _{{\mathcal {Y}}}\varGamma _k({\mathcal {X}}^{q+1},{\mathcal {Y}}^{q+1},{\mathcal {Z}}^{q})\right\| \\&\le \sum _{k=1}^p\vartheta _k\left\| (\widehat{{\mathcal {X}}}^{q+1})^{(k)}\right\| _F \left\| (\widehat{{\mathcal {Z}}}^{q+1})^{(k)}-(\widehat{{\mathcal {Z}}}^q)^{(k)}\right\| _F \\&\le {\hat{c}}c\left\| {\mathcal {Z}}^{q+1}-{\mathcal {Z}}^{q}\right\| _F, \end{aligned}$$

which means (6.17) holds. $\square$

Proposition A.6

Let $\left\{ {\mathcal {W}}^q\right\}$ be a sequence generated by Algorithm 1. Then for every q, we have

$$\begin{aligned} \big \Vert \nabla _{{\mathcal {Z}}}\varPsi \left( {\mathcal {W}}^{q+1}\right) \big \Vert _F\le \beta _1\big \Vert {\mathcal {U}}^{q+1}-{\mathcal {U}}^{q}\big \Vert _F+ \beta _2\big \Vert {\mathcal {V}}^{q+1}-{\mathcal {V}}^{q}\big \Vert _F. \end{aligned}$$

(6.26)

Proof

By the definition of $\varPsi ({\mathcal {W}})$ and $f^q$ given in Proposition A.3, we have

$$\begin{aligned} \nabla _{{\mathcal {Z}}_{\varOmega ^c}}\varPsi ({\mathcal {W}})=({\mathcal {Z}}-{\mathcal {X}}\circledast _{L}{\mathcal {Y}})_{\varOmega ^c}+\beta _1\left( {\mathcal {Z}}-{\mathcal {U}}\right) _{\varOmega ^c}+\beta _2\left( {\mathcal {Z}}-{\mathcal {U}}\right) _{\varOmega ^c}. \end{aligned}$$

Since $\nabla _{{\mathcal {Z}}_{\varOmega ^c}}\varPsi ({\mathcal {X}}^{q+1},{\mathcal {Y}}^{q+1},{\mathcal {Z}}^{q+1},{\mathcal {U}}^{q},{\mathcal {V}}^{q})=0$, it holds that

$$\begin{aligned}&\big \Vert \nabla _{{\mathcal {Z}}_{\varOmega ^c}}\varPsi \left( {\mathcal {W}}^{q+1}\right) \big \Vert _F \nonumber \\&\quad =\left\| \nabla _{{\mathcal {Z}}_{\varOmega ^c}}\varPsi \left( {\mathcal {W}}^{q+1}\right) -\nabla _{{\mathcal {Z}}_{\varOmega ^c}}\varPsi ({\mathcal {X}}^{q+1},{\mathcal {Y}}^{q+1},{\mathcal {Z}}^{q+1},{\mathcal {U}}^{q},{\mathcal {V}}^{q})\right\| _F \nonumber \\&\quad =\beta _1\big \Vert ({\mathcal {U}}^{q+1})_{\varOmega ^c}-({\mathcal {U}}^{q})_{\varOmega ^c}\big \Vert _F+\beta _2\big \Vert ({\mathcal {V}}^{q+1})_{\varOmega ^c}-({\mathcal {V}}^{q})_{\varOmega ^c}\big \Vert _F \nonumber \\&\quad \le \beta _1\big \Vert {\mathcal {U}}^{q+1}-{\mathcal {U}}^{q}\big \Vert _F+ \beta _2\big \Vert {\mathcal {V}}^{q+1}-{\mathcal {V}}^{q}\big \Vert _F. \end{aligned}$$

We obtain the desired result and complete the proof. $\square$

Proposition A.7

Let $\left\{ {\mathcal {W}}^q\right\}$ be a sequence generated by Algorithm 1. Then for every q, we have

$$\begin{aligned} \big \Vert \nabla _{{\mathcal {U}}}\varPsi \left( {\mathcal {W}}^{q+1}\right) \big \Vert _F=\big \Vert \nabla _{{\mathcal {V}}}\varPsi \left( {\mathcal {W}}^{q+1}\right) \big \Vert _F=0. \end{aligned}$$

(6.27)

Proof

From the structure of $\varPsi$, we know that

$$\begin{aligned} \nabla _{{\mathcal {U}}}\varPsi ({\mathcal {W}})=\beta _1({\mathcal {U}}-{\mathcal {Z}})+\mu _1\mathrm{fold}\left( H^* H{\mathcal {U}}^{(1)},\ldots ,H^* H{\mathcal {U}}^{(p)}\right) , \end{aligned}$$

which implies, together with the fact $\nabla _{{\mathcal {U}}}\varPsi ({\mathcal {X}}^{q+1},{\mathcal {Y}}^{q+1},{\mathcal {Z}}^{q+1},{\mathcal {U}}^{q+1},{\mathcal {V}}^{q})=0$, that $\nabla _{{\mathcal {U}}}\varPsi \left( {\mathcal {W}}^{q+1}\right) =0$, i.e., $\big \Vert \nabla _{{\mathcal {U}}}\varPsi \left( {\mathcal {W}}^{q+1}\right) \big \Vert _F=0$. Clearly, $\big \Vert \nabla _{{\mathcal {V}}}\varPsi \left( {\mathcal {W}}^{q+1}\right) \big \Vert _F=0$ can be proved similarly. $\square$

Theorem A.3

(Relative error property) Let $\left\{ {\mathcal {W}}^q\right\}$ be a sequence generated by Algorithm 1. Then there exists $\varrho >0$ such that

$$\begin{aligned} \left\| \nabla \varPsi \left( {\mathcal {W}}^{q+1}\right) \right\| _F\le \varrho \left\| {\mathcal {W}}^{q+1}-{\mathcal {W}}^{q}\right\| _F, \end{aligned}$$

(6.28)

holds for every q.

Proof

It is obvious that

$$\begin{aligned} \left\| \nabla \varPsi \left( {\mathcal {W}}^{q+1}\right) \right\| _F^2&=\left\| \nabla _{{\mathcal {X}}}\varPsi \left( {\mathcal {W}}^{q+1}\right) \right\| _F^2+\left\| \nabla _{{\mathcal {Y}}}\varPsi \left( {\mathcal {W}}^{q+1}\right) \right\| _F^2\\&\quad +\left\| \nabla _{{\mathcal {Z}}}\varPsi \left( {\mathcal {W}}^{q+1}\right) \right\| _F^2+\left\| \nabla _{{\mathcal {U}}}\varPsi \left( {\mathcal {W}}^{q+1}\right) \right\| _F^2+\left\| \nabla _{{\mathcal {V}}}\varPsi \left( {\mathcal {W}}^{q+1}\right) \right\| _F^2. \end{aligned}$$

It follows from Propositions A.5, A.6, and A.7. $\square$

Theorem A.4

(Convergence to a stationary point) Let $\left\{ {\mathcal {W}}^q\right\}$ be a sequence generated by Algorithm 1. Then the sequence $\left\{ {\mathcal {W}}^q\right\}$ has a finite length, i.e.,

$$\begin{aligned} \sum _{k=1}^{+\infty }\big \Vert {\mathcal {W}}^{q+1}-{\mathcal {W}}^q\big \Vert _F<+\infty , \end{aligned}$$

and hence $\left\{ {\mathcal {W}}^q\right\}$ is a Cauchy sequence which converges to a stationary point of (4.4).

Proof

It is obvious that $\varPsi$ satisfies the Kurdyka-Łojasiewicz inequality, since $\varPsi (\cdot )$ is a semi-algebraic function. Moreover, by Theorems A.2 and A.3, we know that all conditions in [1, Theorem 2.9] are satisfied, and hence the desired result follows. $\square$

Theorem A.5

(Local convergence to global minima) For each $\delta _0>0$, there exist $\delta \in (0,\delta _0)$ and $\eta >0$ such that for the starting point ${\mathcal {W}}^0$ satisfying $\big \Vert {\mathcal {W}}^0-{\mathcal {W}}^{\star }\big \Vert _F<\delta$ and ${\min }\;\varPsi ({\mathcal {W}})<\varPsi ({\mathcal {W}}^0)<{\min }\;\varPsi ({\mathcal {W}})+\eta$, the sequence $\left\{ {\mathcal {W}}^q\right\}$ generated by Algorithm 1 satisfies

• (i). ${\mathcal {W}}^q\in {\mathbb {N}}({\mathcal {W}}^{\star },\delta _0)$ for every q;

• (ii). $\left\{ {\mathcal {W}}^q\right\}$ converges to some ${{\mathcal {W}}^{\infty }}$ and $\sum _{q=1}^{+\infty }\big \Vert {\mathcal {W}}^{q+1}-{\mathcal {W}}^q\big \Vert _F<+\infty$;

• (iii). $\varPsi ({{\mathcal {W}}^{\infty }})=\varPsi ({\mathcal {W}}^{\star })$.

Proof

It follows from [1, Theorem 2.12]. $\square$