A Globally and Quadratically Convergent Algorithm for Solving Multilinear Systems with ${{\mathcal {M}}}$-tensors

Abstract

We consider multilinear systems of equations whose coefficient tensors are ${{\mathcal {M}}}$-tensors. Multilinear systems of equations have many applications in engineering and scientific computing, such as data mining and numerical partial differential equations. In this paper, we show that solving multilinear systems with ${{\mathcal {M}}}$-tensors is equivalent to solving nonlinear systems of equations where the involving functions are P-functions. Based on this result, we propose a Newton-type method to solve multilinear systems with ${{\mathcal {M}}}$-tensors. For a multilinear system with a nonsingular ${{\mathcal {M}}}$-tensor and a positive right side vector, we prove that the sequence generated by the proposed method converges to the unique solution of the multilinear system and the convergence rate is quadratic. Numerical results are reported to show that the proposed method is promising.

Appendices

Appendix

In this appendix, we present the detailed Proofs of Proposition 3.2, Theorems 3.2 and 3.3.

Proof of Proposition 3.2

Proof

On a contrary, suppose that there exists a sequence $\{(t^k,y^k) \}\subset \mathfrak {R}_{++} \times \mathfrak {R}^n_{++}$ such that ${{\tilde{t}}}\le t^k\le \bar{t}$, $\Vert y^k\Vert \rightarrow +\infty$, and the sequence $\{\psi (t^k, y^k)\}$ is bounded.

We define the index set J by $J:=\{i\in [n] : y ^k_i \ \mathrm{is \ unbounded}\}.$ Then, $J\ne \emptyset$ because $\Vert y^k\Vert \rightarrow +\infty$. For each k, let

$$\begin{aligned} \bar{y}^k_i =\left\{ \begin{array}{ll} y^k_i, &{}\quad \mathrm{if}\ i\notin J\\ 1, &{}\quad \mathrm{if }\ i\in J \end{array} \right. , \quad i = 1, 2, ..., n. \end{aligned}$$

Let $\bar{y}^k := (\bar{y}^k_1,\bar{y}^k_2,\ldots ,\bar{y}^k_n)^\top$. Then, the sequence $\{\bar{y}^k\} \subset \mathfrak {R}^n_{++}$ is bounded. By Lemma 2.2, we can see that W(y) defined in (2.2) is P-function in $\mathfrak {R}^n_{++}$. Hence, for each k, there exists an $i_k \in J$ such that

$$\begin{aligned} (y^k_{i_k} - 1) \left[ W(y^k) - W(\bar{y}^k)\right] _{i_k} > 0. \end{aligned}$$

Then, we have

$$\begin{aligned} (y^k_{i_k} - 1)\left[ G(t^k, y^k) - G(t^k, \bar{y}^k)\right] _{i_k}&= (y^k_{i_k} - 1) \left[ W(y^k) - W({\bar{y}}^k) + t^k (y^k - {\bar{y}}^k)\right] _{i_k}\\&= (y^k_{i_k} - 1) \left[ W(y^k) - W({\bar{y}}^k)\right] _{i_k} + t^k (y^k_{i_k} - 1)^2\\&> t^k (y^k_{i_k} - 1)^2. \end{aligned}$$

Since $y^k_{i_k} \rightarrow +\infty$ for $i_k\in J$, there exists an integer N such that $y^k_{i_k} > 1$ for all $k\ge N$. So,

$$\begin{aligned} \left[ G(t^k, y^k) - G(t^k, \bar{y}^k)\right] _{i_k} > t^k (y^k_{i_k} - 1). \end{aligned}$$

Since $t^k \ge {{\tilde{t}}} >0$ and $y^k_{i_k} \rightarrow +\infty$, we have $[G(t^k, y^k) - G(t^k, \bar{y}^k)]_{i_k} \rightarrow \infty \ \mathrm{as } \ k\rightarrow \infty .$ Note that $\{\Vert G(t^k, \bar{y}^k)\Vert \}$ is bounded as $\{\bar{y}^k\}$ is bounded. It then follows that $[G(t^k, y^k)]_{i_k} \rightarrow +\infty .$ Since J has only a finite number of elements, by taking a subsequence if necessary, we may assume that there exists an $i\in J$ such that

$$\begin{aligned}_{i} \rightarrow +\infty . \end{aligned}$$

Thus, by (3.1), the sequence $\{\psi (t^k, y^k) = \Vert H(t^k, y^k)\Vert ^2\}$ is unbounded. This is a contradiction which shows that this proposition holds. $\square$

Proof of Theorem 3.2

Proof

(i). It follows from Lemma 3.2 that an infinite sequence $\{(t^k, y^k)\}$ is generated such that $t^k \ge \beta (t^k, y^k) {\bar{t}}$ for all $k\ge 0$. From the design of Algorithm 1, $\psi (t^{k+1}, y^{k+1})< \psi (t^k, y^k)$ for all $k\ge 0$. Hence the sequences $\{ t^k\}$, $\{\psi (t^k, y^k)\}$ and $\{\beta (t^k, y^k)\}$ are monotonically decreasing. Since both $\psi (t^k, y^k)>0$ and $\beta (t^k, y^k) > 0$ for all $k\ge 0$, there exist ${\tilde{\psi }}\ge 0$ and ${\tilde{\beta }} \ge 0$ such that $\psi (t^k, y^k) \rightarrow {\tilde{\psi }}$ and $\beta (t^k, y^k) \rightarrow {\tilde{\beta }}$ as $k\rightarrow \infty$, respectively. Suppose that ${{\tilde{\psi }}}>0$. Then, from Lemma 3.2,

$$\begin{aligned} \lim _{k\rightarrow +\infty }t^k = {{\tilde{t}}}\ge {{\tilde{\beta }}}{\bar{t}} > 0. \end{aligned}$$

By Proposition 3.2, it can be easily seen that the sequence $\{(t^k, y^k)\}$ is bounded. From Lemma 3.3, we have ${{\tilde{\psi }}} = 0$. This contradiction shows that ${{\tilde{\psi }}} = 0$, i.e.,

$$\begin{aligned} \lim _{k\rightarrow +\infty } H(t^k, y^k) = 0\;\;\ \mathrm{and}\ \;\; \lim _{k\rightarrow +\infty }t^k = 0. \end{aligned}$$

(ii). Suppose that the infinite sequence $\{(t^k, y^k)\}$ is not bounded. Then the sequence $\{y^k\}$ is not bounded. Let $y^*\in \mathfrak {R}^n_{++}$ be the unique solution of $W(y) = 0$, i.e., the solution of $\phi _0(y) = 0$. Without loss of generality, assume that $\{\Vert y^k\Vert \}\rightarrow \infty$. Hence there exists a compact set ${\varvec{C}}\subset \mathfrak {R}^n_{++}$ with $y^*\in \mathrm{int} {\varvec{C}}$ and

$$\begin{aligned} y^k \in \mathfrak {R}^n_{++} \backslash {\varvec{C}} \end{aligned}$$

(B.1)

for all k sufficiently large. Since

$$\begin{aligned} {{\bar{\xi }}} := \min _{y\in \partial {\varvec{C}}}\phi _0(y) > 0, \end{aligned}$$

we can apply Lemma 3.4 with $\zeta := {{\bar{\xi }}}/4$ and conclude that

$$\begin{aligned} \phi _{t^k}(y^*) \le {1\over 4}{{\bar{\xi }}} \end{aligned}$$

(B.2)

and

$$\begin{aligned} \xi := \min _{y\in \partial {\varvec{C}}}\phi _{t^k}(y) \ge {3\over 4}{{\bar{\xi }}} \end{aligned}$$

(B.3)

for all k sufficiently large. From Item (i) of this theorem, we have

$$\begin{aligned} \phi _{t^k}(y^k) \le {1\over 4}{{\bar{\xi }}}, \end{aligned}$$

(B.4)

for all k sufficiently large. Now let us fix an index ${{\bar{k}}}$ such that $t^{{\bar{k}}}\ne 0$ and (B.1)–(B.4) hold. By Proposition 3.2, it is easy to see that for any $\{y^k\}$ with property $\Vert y^k\Vert \rightarrow +\infty$, we have $\lim _{k\rightarrow \infty }\phi _{t^{{\bar{k}}}}(y^k)=+\infty$. Consequently, by applying Theorem 3.1 with ${\varvec{d}} := y^k$ and ${\varvec{a}} := y^*$, we obtain the existence of a vector ${\varvec{c}}\in \mathfrak {R}^n_{++}$ such that

$$\begin{aligned} \nabla \phi _{t^{{\bar{k}}}}({\varvec{c}}) = 0 \quad \mathrm{and}\quad \phi _{t^{{\bar{k}}}}({\varvec{c}}) \ge {3\over 4}{{\bar{\xi }}} > 0. \end{aligned}$$

From (3.14) we have $G(t^{{\bar{k}}}, {\varvec{c}}) = 0$, i.e., $\phi _{t^{{\bar{k}}}}({\varvec{c}}) = 0$. This contradiction implies that Item (ii) of this theorem holds.

(iii). It follows from Items (i) and (ii) of this theorem that Item (iii) holds immediately. $\square$

Proof of Theorem 3.3

Proof

First, from Theorem 3.2, the sequence $\{(t^k, y^k)\}$ generated by Algorithm 1 converges to $(0, y^*)$. Now we show that (3.16) holds. In the following, let $z^*: = (0, y^*)$, $z^k: = (t^k, y^k)$, $\varDelta z^k := (\varDelta t^k, \varDelta y^k)$, and $r_k:=(\beta _k{\bar{t}},\mathbf{0})$ with $\beta _k := \beta (t^k, y^k)$. From Step 4 of Algorithm 1, we have

$$\begin{aligned} H(z^k) + H^{\prime }(z^k) \varDelta z^k = r_k \quad \mathrm{and}\quad \Vert r_k\Vert = \beta _k\bar{t}. \end{aligned}$$

(C.1)

Since H is smooth on $\mathfrak {R}\times \mathfrak {R}^n_{++}$ and $H^{\prime }(z^*)$ is nonsingular, there exist a closed neighbourhood ${{{\mathcal {N}}}}(z^*)\subset \mathfrak {R}\times \mathfrak {R}^n_{++}$ and two scalars $L_1$ and $L_2$ such that for all $z:=(t, y) \in {{{\mathcal {N}}}}(z^*)$,

$$\begin{aligned} \Vert H^{\prime }(z)^{-1}\Vert =\Vert H^{\prime }(t, y)^{-1}\Vert \le L_1 \end{aligned}$$

and

$$\begin{aligned} \Vert H(z)-H(z^*)-H'(z)(z-z^*)\Vert \le L_2\Vert z-z^*\Vert ^2. \end{aligned}$$

Then, for $z^k$ sufficiently close to $z^*$, we have

$$\begin{aligned} \Vert z^k+\varDelta z^k-z^*\Vert&= \Vert z^k+ H^{\prime }(z^k) ^{-1} [ -H(z^k) + r_k ] -z^*\Vert \nonumber \\&\le \Vert z^k-z^*-H^{\prime }(z^k)^{-1} H(z^k)\Vert + L_1\beta _k {\bar{t}} \nonumber \\&\le L_1 \Vert H(z^k)-H(z^*)-H^{\prime }(z^k) (z^k-z^*) \Vert +L_1\beta _k {\bar{t}}\nonumber \\&\le L_1 L_2 \Vert z^k-z^* \Vert ^2 +L_1\beta _k {\bar{t}}\nonumber \\&= O(\Vert z^k-z^*\Vert ^2) + O (\psi _k), \end{aligned}$$

(C.2)

where the last equality follows from the definition of $\beta _k$ in (3.5), and $\psi _k:=\psi (z^k)=\psi (t^k,y^k)$ throughout the proof. Then, because H is smooth at $z^*$, for all $z^k$ close to $z^*$,

$$\begin{aligned} \psi _k = \Vert H(z^k)\Vert ^{2} = O(\Vert z^k-z^*\Vert ^{2}). \end{aligned}$$

(C.3)

Therefore, from (C.2) and (C.3), for all $z^k$ sufficiently close to $z^*$,

$$\begin{aligned} \Vert z^{k}+\varDelta z^k-z^*\Vert = O(\Vert z^k-z^*\Vert ^{2}). \end{aligned}$$

(C.4)

By (C.4), for any $\epsilon \in (0, {1\over 2})$, there is a $k(\epsilon )$ such that for all $k\ge k(\epsilon )$,

$$\begin{aligned} \Vert z^{k}+\varDelta z^k-z^*\Vert \le \epsilon \Vert z^k-z^*\Vert . \end{aligned}$$

(C.5)

Using (C.1) leads to

$$\begin{aligned} \Vert \varDelta z^k\Vert&= \Vert H^{\prime }(z^k) ^{-1} [-H(z^k) + r_k ]\Vert \nonumber \\&\le L_1\Vert H(z^k)\Vert + {\bar{t}} L_1 \psi _k^{1\over 2} \nonumber \\&= (1+{\bar{t}}\ )L_1\Vert H(z^k)\Vert . \end{aligned}$$

(C.6)

It then follows from (C.5) and (C.6) that

$$\begin{aligned} \Vert z^k -z^*\Vert&= \Vert \varDelta z^k\Vert + \Vert z^{k}+\varDelta z^k-z^*\Vert \\&\le (1+{\bar{t}}\ )L_1\Vert H(z^k)\Vert + \epsilon \Vert z^k-z^*\Vert . \end{aligned}$$

Consequently, it is clear from $\epsilon \in (0,\frac{1}{2})$ that

$$\begin{aligned} \Vert z^k -z^*\Vert \le 2(1+{\bar{t}}\ )L_1\Vert H(z^k)\Vert . \end{aligned}$$

Since H is smooth at $z^*$, for all $z^k$ sufficiently close to $z^*$, we have

$$\begin{aligned} \psi (z^k+\varDelta z^k)&= \Vert H(z^k+\varDelta z^k)\Vert ^2 \\&=O(\Vert z^k+\varDelta z^k-z^*\Vert ^2) \\&= O(\Vert z^k-z^*\Vert ^{4}) \\&= O(\Vert H(z^k)-H(z^*)\Vert ^{4}) \\&=O(\psi _k^{2}). \end{aligned}$$

Therefore, for all $z^k$ sufficiently close to $z^*$ we have

$$\begin{aligned} z^{k+1}=z^k+\varDelta z^k. \end{aligned}$$

Hence, by (C.4), we immediately have that (3.16) holds. $\square$