Characterization Of sampling patterns for low-tt-rank tensor retrieval

Abstract

In this paper, we analyze the fundamental conditions for low-rank tensor completion given the separation or tensor-train (TT) rank, i.e., ranks of TT unfoldings. We exploit the algebraic structure of the TT decomposition to obtain the deterministic necessary and sufficient conditions on the locations of the samples to ensure finite completability. Specifically, we propose an algebraic geometric analysis on the TT manifold that can incorporate the whole rank vector simultaneously in contrast to the existing approach based on the Grassmannian manifold that can only incorporate one rank component. Our proposed technique characterizes the algebraic independence of a set of polynomials defined based on the sampling pattern and the TT decomposition, which is instrumental to obtaining the deterministic condition on the sampling pattern for finite completability. In addition, based on the proposed analysis, assuming that the entries of the tensor are sampled independently with probability p, we derive a lower bound on the sampling probability p, or equivalently, the number of sampled entries that ensures finite completability with high probability. Moreover, we also provide the deterministic and probabilistic conditions for unique completability.

Appendix: A Canonical Decomposition and The Degree of Freedom

We are interested in providing a structure on the decomposition $\mathbb {U}$ such that there is one decomposition among all possible decompositions of the sampled tensor $\mathcal {U}$ that captures the structure. Before describing such a structure on TT decomposition, we start with a similar structure for matrix decomposition.

Lemma 15

Let X denote a generically chosen matrix from the manifold of n₁ × n₂ matrices of rank r. Then, there exists a unique decomposition X = YZ such that $\mathbf {Y} \in \mathbb {R}^{n_{1} \times r}$, $\mathbf {Z} \in \mathbb {R}^{r \times n_{2}}$ and Y(1 : r, 1 : r) = I_r, where Y(1 : r, 1 : r) represents the submatrix of Y consists of the first r columns and the first r rows and I_r denotes the r × r identity matrix.

Proof

Weshow that there exists exactly one decomposition X = YZ such that Y(1 : r, 1 : r) = I_r with probability one. Considering the first r rows of X = YZ, we conclude X(1 : r, :) = I_rZ = Z. Therefore, we need to show that there exists exactly one Y(r + 1 : n₁, :) such that X(r + 1 : n₁, :) = Y(r + 1 : n₁, :)Z or equivalently X(r + 1 : n₁, :)^⊤ = X(1 : r, :)^⊤Y(r + 1 : n₁, :)^⊤. It suffices to show that each column of Y(r + 1 : n₁, :) can be determined uniquely having x = X(1 : r, :)^⊤y where $\mathbf {x} \in \mathbb {R}^{n_{2} \times 1}$ and $\mathbf {y} \in \mathbb {R}^{r \times 1}$. As X is a generically chosen n₁ × n₂ matrix of rank r, we have $\text {rank}\left (\mathbf {X}(1:r,:)\right ) = r$ with probability one. Hence, x(1 : r) = X(1 : r, 1 : r)^⊤y results in r independent degree-1 equations in terms of the r variables (entries of y), and therefore y has exactly one solution with probability one. □

Remark 6

Note that the genericity assumption is necessary as we can find counter examples for Lemma 15 in the absence of genericity assumption, e.g., it is easily verified that the following decomposition is not possible:

Remark 2

Assume that $\mathbf {Q} \in \mathbb {R}^{r \times r}$ is an arbitrary given full rank matrix. Then, for any submatrix¹$\mathbf {P} \in \mathbb {R}^{r \times r}$ of Y, Lemma 15 also holds if we replace Y(1 : r, 1 : r) = I_r by P = Q in the statement. The proof is similar to the proof of Lemma 15 and thus it is omitted.

As mentioned earlier, similar to the matrix case, we are interested in obtaining a structure on TT decomposition of a tensor such that there exists one decomposition among all possible TT decompositions of a tensor that captures the structure. Hence, we define the following structure on the decomposition in order to characterize a condition on the sampling pattern to study the algebraic independency of the above-mentioned polynomials.

Definition 4

Consider any d − 1 submatrices $\mathbf {P}_{1},\dots ,\mathbf {P}_{d-1}$ of $\mathbf {U}^{(1)},\mathbf {U}^{(2)}_{(2)},\mathbf {U}^{(3)}_{(2)},\dots ,$$\mathbf {U}^{(d-1)}_{(2)}$, respectively such that (i) $\mathbf {P}_{i} \in \mathbb {R}^{r_{i} \times r_{i}}$, $i=1,\dots ,d-1$, (ii) the r_i columns of $\mathbf {U}^{(i)}_{(2)}$ corresponding to columns of P_i belong to r_i distinct rows of $\mathbf {U}^{(i)}_{(3)}$, $i=2,\dots ,d-1$. Then, $\mathbb {U}$ is said to have a proper structure if P_i is full rank, $i=1,\dots ,d$.²

Define the matrices $\mathbf {P}_{1}^{\text {can}},\dots ,\mathbf {P}_{d-1}^{\text {can}}$ such that for any 1 ≤ x_i ≤ r_i and any $1 \leq x_{i}^{\prime } \leq r_{i}$ we have:

$$ \mathbf{P}_{i}^{\text{can}} (x_{i},k_{i}) = \mathcal{U}^{(i)}(1,x_{i},k_{i}) \in \mathbb{R}^{r_{i} \times r_{i}}, \ \ \ \ \ \ \ \ i = 2,\dots,d-1, $$

(43)

and

$$ \mathbf{P}_{1}^{\text{can}} (x_{1},k_{1}) = \mathcal{U}^{(1)}(x_{1},k_{1}) \in \mathbb{R}^{r_{1} \times r_{1}}. $$

(44)

It is easy to verify that $\mathbf {P}_{1}^{\text {can}},\dots ,\mathbf {P}_{d-1}^{\text {can}}$ satisfy properties (i) and (ii) in Definition 4.

Definition 5

(Canonical basis) We call $\mathbb {U}$ a canonical decomposition if for $i =1,\dots ,d$ we have $\mathbf {P}_{i}^{\text {can}} = \mathbf {I}_{r_{i}}$, where $\mathbf {I}_{r_{i}}$ is the r_i × r_i identity matrix.

Lemma 16

Consider the TT decomposition in (1). Then, $\mathbf {U}^{(1)} \in \mathbb {R}^{n_{1} \times r_{1}}$, $\mathbf {U}^{(d)} \in \mathbb {R}^{r_{d-1} \times n_{d}}$, $\mathbf {U}^{(i)}_{(1)} \in \mathbb {R}^{r_{i-1} \times n_{i} r_{i}}$ and $\mathbf {U}^{(i)}_{(3)} \in \mathbb {R}^{r_{i} \times r_{i-1} n_{i}}$, $i = 2,\dots ,d-1$, are full rank matrices.

Proof

In general, besides the separation rank $(r_{1},\dots ,r_{d-1})$, we may be able to obtain a TT decomposition for other vectors $(r_{1}^{\prime },\dots ,r_{d-1}^{\prime })$ as well. However, according to [20] among all possible TT decomposition for different values of $r_{i}^{\prime }$’s, $r_{i}^{\prime } = \text {rank}(\mathbf {\widetilde U}_{(i)}) = r_{i}$, $i=1,\dots ,d-1$, is minimal, in the sense that there does not exist any decomposition with $r_{i}^{\prime }$’s such that $r_{i}^{\prime } \leq r_{i}$ for $i=1,\dots ,d-1$ and $r_{i}^{\prime } < r_{i}$ for at least one $i \in \{1,\dots ,d-1\}$. By contradiction, assume that $\mathbf {U}_{(1)}^{(i+1)}$ is not full rank. Then, $\text {rank}\left (\mathbf { \widetilde U}_{(2)}^{(i)} \mathbf {U}_{(1)}^{(i+1)} \right ) < r_{i}$.

Let X denote the matrix $\mathbf { \widetilde U}_{(2)}^{(i)} \mathbf {U}_{(1)}^{(i+1)}$. Since $\text {rank}\left (\mathbf X \right ) = r_{i}^{\prime } < r_{i}$, there exists a decomposition $\mathbf {X} = \mathbf { \widetilde U}_{(2)}^{{(i)}^{\prime }} \mathbf {U}_{(1)}^{{(i+1)}^{\prime }}$ such that $\mathbf { \widetilde U}_{(2)}^{{(i)}^{\prime }} \in \mathbb {R}^{r_{i-1}n_{i} \times r_{i}^{\prime }}$ and also $\mathbf {U}_{(1)}^{{(i+1)}^{\prime }} \in \mathbb {R}^{r_{i}^{\prime } \times n_{i+1}r_{i+1}}$. Hence, the existence of the TT decomposition with $\mathcal { U}^{(i)}$ and $\mathcal {U}^{(i+1)}$ replaced by $\mathcal { U}^{{(i)}^{\prime }}$ and $\mathcal {U}^{{(i+1)}^{\prime }}$ contradicts the above-mentioned minimum property of the separation rank. Note that for a three-way tensor, the second TT unfolding is the transpose of the third Tucker unfolding, and therefore $\text {rank}\left (\mathbf { \widetilde U}_{(2)}^{(i)} \right )= \text {rank}\left (\mathbf { U}_{(3)}^{(i)} \right )$ and the rest of the cases can be verified similarly. □

Lemma 17

Assume that $\mathbf {Q}_{i} \in \mathbf {R}^{r_{i} \times r_{i}}$ is an arbitrary given full rank matrix, 1 ≤ i ≤ d − 1. Consider a set of matrices $\mathbf {P}_{1},\dots ,\mathbf {P}_{d-1}$ that satisfy properties (i) and (ii) in Definition 4. Then, there exists exactly one decomposition $\mathbb {U}$ of the sampled tensor $\mathcal {U}$ such that P_i = Q_i, $i=1,\dots ,d-1$.

Proof

Consider an arbitrary decomposition $\mathbb {U}$ of the sampled tensor $\mathcal {U}$. Let $\mathcal {A}^{(i)} = \mathcal {U}^{(i)} \mathcal {U}^{(i+1)}$$\in \mathbb {R}^{r_{i-1} \times n_{i} \times n_{i+1} \times r_{i+1}}$, $i=1,\dots ,d-1$, where the above multiplication is the same tensor multiplication in TT decomposition (1). Note that for a three-way tensor, the second TT unfolding is the transpose of the third Tucker unfolding, and therefore their ranks are the same. According to Lemma 16, $\text {rank}\left (\mathbf {U}^{(1)} \right )=\text {rank}\left (\mathbf {U}^{(2)}_{(1)}\right )=r_{1}$, $\text {rank}\left (\widetilde {\mathbf {U}}_{(2)}^{(2)} \right )=\text {rank}\left (\mathbf {U}_{(1)}^{(3)}\right )=r_{2}$, $\dots$, and $\text {rank}\left (\widetilde {\mathbf {U}}_{(2)}^{(d-1)} \right )=\text {rank}\left (\mathbf {U}^{(d)}\right )=r_{d-1}$.

As a result, we have $\text {rank}\left (\mathbf {U}^{(1)} \mathbf {U}_{(1)}^{(2)}\right )=r_{1}$, $\text {rank}\left (\widetilde {\mathbf {U}}^{(2)}_{(2)} \mathbf {U}_{(1)}^{(3)}\right )=r_{2}, \dots ,$$\text {rank}\left (\widetilde {\mathbf {U}}_{(2)}^{(d-1)} \mathbf {U}^{(d)}\right )$ = r_d− 1. Observe that $\widetilde {\mathbf {U}}_{(2)}^{(i)} \mathbf {U}_{(1)}^{(i+1)} = \widetilde {\mathbf {A}}_{(2)}^{(i)}$, and therefore $\text {rank}\left (\widetilde {\mathbf {A}}_{(2)}^{(i)} \right ) = r_{i}$ for $i=2,\dots ,d-2$ and similarly $\text {rank}\left (\widetilde {\mathbf {A}}_{(1)}^{(1)} \right ) = r_{1}$ and $\text {rank}\left (\widetilde {\mathbf {A}}_{(2)}^{(d)} \right ) = r_{d}$. According to Lemma 15 and Remark 7, for an n₁ × n₂ matrix X of rank r there exists a unique decomposition X = X₁X₂ such that $\mathbf {X}_{1} \in \mathbb {R}^{n_{1} \times r}$ and $\mathbf {X}_{2} \in \mathbb {R}^{r \times n_{2}}$ and an arbitrary r × r submatrix of X₁ is equal to the given r × r full rank matrix.

We claim that there exist $(\mathcal {V}^{(i)},\mathcal {V}^{(i+1)})$ such that $\mathcal {V}^{(i)} \mathcal {V}^{(i+1)} = \mathcal {A}^{(i)}$ and the corresponding submatrix P_i is equal to the given full rank matrix Q_i, $i=1,\dots ,d-1$. We repeat this procedure for each $i=1,\dots ,d-1$ and update two core tensors of TT decomposition $(\mathcal {V}^{(i)},\mathcal {V}^{(i+1)})$ at iteration i and at the end, we obtain a TT decomposition that has the mentioned structure in the statement of Lemma 17. In the following we show the existence of such $(\mathcal {V}^{(i)},\mathcal {V}^{(i+1)})$ at each iteration. At step one, we find $(\mathcal {V}^{(1)},\mathcal {V}^{(2)})$ such that $\mathcal {V}^{(1)} \mathcal {V}^{(2)} = \mathcal {A}^{(1)}$ and the corresponding submatrix P₁ of $\mathcal {V}^{(1)}$ is equal to Q₁. We update the decomposition with $\mathcal {U}^{(1)}$ and $\mathcal {U}^{(2)}$ replaced by $\mathcal {V}^{(1)}$ and $\mathcal {V}^{(2)}$, and therefore we obtain a new decomposition $\mathbb {U}^{1}$ of the sampled tensor $\mathcal {U}$ such that the submatrix of $\mathcal {V}^{(1)}$ corresponding to P₁ is equal to Q₁. Then, in step 2 we consider $\mathcal {A}^{(2)}$ and similarly we update the second and third factor of the decomposition obtained in the last step. Eventually after d − 1 steps, we obtain a decomposition of the sampled tensor $\mathcal {U}$ that P_i = Q_i, $i=1,\dots ,d-1$. To show the uniqueness of such decomposition, we show that each core tensor of the TT decomposition can be determined uniquely. Remark 7 for rank component r₁ results that $\mathcal {U}^{(1)}$ and the multiplication of the rest of the core tensors of the TT decomposition can be determined uniquely. By repeating this procedure for other rank components the uniqueness of such decomposition can be verified by showing the uniqueness of the core tensors one by one. □

Lemma 17 leads to the fact that given $\mathcal {U}^{(d)}$, the dimension of all tuples $(\mathcal {U}^{(1)},{\dots } ,\mathcal {U}^{(d-1)})$ that satisfy TT decomposition is ${\sum }_{i=1}^{d-1} r_{i-1}n_{i}r_{i} -{\sum }_{i=1}^{d-1} {r_{i}^{2}}$, as ${\sum }_{i=1}^{d-1} r_{i-1}n_{i}r_{i}$ is the total number of entries of $(\mathcal {U}^{(1)} , {\dots } , \mathcal {U}^{(d-1)})$ and ${\sum }_{i=1}^{d-1} {r_{i}^{2}}$ is the total number of the entries of the pattern or structure that is equivalent to the uniqueness of TT decomposition. We make the following assumption which will be referred to, when it is needed.

Note that for Lemma 17, we need the strong low-rankness assumption r_i ≤ n_i. However, these results are also consequences of the analysis in [20]. The purpose is to present a simple and intuitive proof.