Proximal linearization methods for Schatten p-quasi-norm minimization

Abstract

Schatten p-quasi-norm minimization has advantages over nuclear norm minimization in recovering low-rank matrices. However, Schatten p-quasi-norm minimization is much more difficult, especially for generic linear matrix equations. In this paper, we first extend the lower bound theory of $\ell _p$ minimization to Schatten p-quasi-norm minimization. We prove that the positive singular values of local minimizers are bounded from below by a constant. Motivated by this property, we propose a proximal linearization method, whose subproblems can be solved efficiently by the (linearized) alternating direction method of multipliers. The convergence analysis of the proposed method involves the nonsmooth analysis of singular value functions. We give a necessary and sufficient condition for a singular value function to be a Kurdyka–Łojasiewicz function. The subdifferentials of related singular value functions are computed. The global convergence of the proposed method is established under some assumptions. Experiments on matrix completion, Sylvester equation and image deblurring show the effectiveness of the algorithm.

Appendix

This appendix summarizes some important results on KL theory and gives some examples. The following definition is adopted from Attouch et al. [2, Definition 4.1].

Definition 7.1

(o-minimal structure on $\mathbb {R}$) Let $\mathscr {O} = \{\mathscr {O}_n\}_{n \in \mathbb {N}}$ such that each $\mathscr {O}_n$ is a collection of subsets of $\mathbb {R}^n$. The family $\mathscr {O}$ is an o-minimal structure on $\mathbb {R}$, if it satisfies the following axioms:

• (i) Each $\mathscr {O}_n$ is a boolean algebra. Namely $\emptyset \in \mathscr {O}_n$ and for each $\mathscr {A},\mathscr {B}$ in $\mathscr {O}_n$, $\mathscr {A} \cup \mathscr {B}$, $\mathscr {A} \cap \mathscr {B}$, and $\mathbb {R}^n \setminus \mathscr {A}$ belong to $\mathscr {O}_n$.

• (ii) For all $\mathscr {A}$ in $\mathscr {O}_n$, $\mathscr {A} \times \mathbb {R}$ and $\mathbb {R} \times \mathscr {A}$ belong to $\mathscr {O}_{n+1}$.

• (iii) For all $\mathscr {A}$ in $\mathscr {O}_{n+1}$, $\{(x_1,\ldots ,x_n)\in \mathbb {R}^n: (x_1,\ldots ,x_n,x_{n+1}) \in \mathscr {A}\}$ belongs to $\mathscr {O}_{n}$.

• (iv) For all $i \ne j$ in $\{1,2,\ldots ,n\}$, $\{(x_1,\ldots ,x_n) \in \mathbb {R}^n: x_i = x_j\}$ belongs to $\mathscr {O}_n$.

• (v) The set $\{(x_1,x_2) \in \mathbb {R}^2: x_1<x_2\}$ belongs to $\mathscr {O}_2$.

• (vi) The elements of $\mathscr {O}_1$ are exactly finite unions of intervals.

Let $\mathscr {O}$ be an o-minimal structure on $\mathbb {R}$. We say that a set $\mathscr {A} \subseteq \mathbb {R}^n$ is definable (on $\mathscr {O}$) if $\mathscr {A} \in \mathscr {O}_n$. A function $f: \mathbb {R}^n \rightarrow (-\infty , +\infty ]$ is definable if its graph $\{(\varvec{x},y) \in \mathbb {R}^n \times (-\infty ,+\infty ]: y \in f(\varvec{x})\}$ is definable on $\mathscr {O}$. We list some known elementary properties of definable functions below.

Property 7.2

(See [2]) Finite sums of definable functions are definable; indicator functions of definable sets are definable; compositions of definable functions or mappings are definable.

It is known that any proper lower semicontinuous function that is definable is a KL function; see [2, Theorem 4.1].

Example 7.3

A class of o-minimal structure is the log-exp structure [45, Example 2.5]. By this structure, the following functions are all definable:

• 1. semi-algebraic functions; see Definition below.

• 2. the function $\mathbb {R} \rightarrow \mathbb {R}$ given by

  $$\begin{aligned} x \mapsto {\left\{ \begin{array}{ll} x^r, &{} x >0 \\ 0, &{} x \le 0, \end{array}\right. } \end{aligned}$$

  where $r \in \mathbb {R}$.

• 3. the exponential function: $\mathbb {R} \rightarrow \mathbb {R}$ given by $x \mapsto e^x$ and the logarithm function: $(0,\infty ) \rightarrow \mathbb {R}$ given by $x \mapsto \log (x)$.

Definition 7.4

(See [5, Definition 5]) A subset $\mathscr {S}$ of $\mathbb {R}^d$ is a real semi-algebraic set if there exists a finite number of real polynomial functions $f_{ij},g_{ij}: \mathbb {R}^d \rightarrow \mathbb {R}$ such that

$$\begin{aligned} \mathscr {S}=\bigcup _{j=1}^s\bigcap _{i=1}^t \left\{ \varvec{x}\in \mathbb {R}^{d}: f_{ij}(\varvec{x})=0 \text { and } g_{ij}(\varvec{x})<0 \right\} \end{aligned}$$

A function $f: \mathbb {R}^{d} \rightarrow (-\infty ,+\infty ]$ is called semi-algebraic if its graph

$$\begin{aligned} \left\{ (\varvec{x},y)\in \mathbb {R}^{d+1}: f(\varvec{x})=y \right\} \end{aligned}$$

is a semi-algebraic subset of $\mathbb {R}^{d+1}$.

The class of semi-algebraic sets is stable under the following operations: finite unions, finite intersections, complementation and Cartesian products.

Example 7.5

[5, Example 2] There is broad class of semi-algebraic sets and functions arising in optimization.

• 1. Real polynomial functions.

• 2. Indicator functions of semi-algebraic sets.

• 3. Finite sums and product of semi-algebraic functions.

• 4. Composition of semi-algebraic functions.

• 5. In matrix theory, all the following are semi-algebraic sets: cone of positive semidefinite matrices, Stiefel manifolds and constant rank matrices.

Example 7.6

Define $f(\varvec{x}): \mathbb {R}^{n}\rightarrow \mathbb {R}$ by $f(\varvec{x})=\sum _{i=1}^{n}|x_i|^p$. We prove that f is definable. First, consider the function $g(t)=|t|$. The graph of g(t) is

$$\begin{aligned}{} & {} \{(t,y)\in \mathbb {R}^2: y = t, t>0 \}\cup \{(t,y)\in \mathbb {R}^2: y = -t, t>0 \}\\{} & {} \quad \cup \{(t,y)\in \mathbb {R}^2: y = t, t=0 \}. \end{aligned}$$

Hence, g(t) is a semi-algebraic function. From Property and Example , we know that $f(\varvec{x})$ is definable.

Example 7.7

We prove that the function $f(\varvec{x})$ defined in (47) is a semi-algebraic function. From Example , we see that it suffices to prove that the function $f_i(\varvec{x}):=b_i \underline{x}_i,i=1,\ldots ,r$ is semi-algebraic. We only prove that $f_1(\varvec{x})=b_1 \underline{x}_1$ is semi-algebraic, and the other cases are similar. Define

$$\begin{aligned} \mathscr {T}_j:=\{\varvec{x}\in \mathbb {R}^h: x_j=\underline{x}_1\}, \quad j=1,\ldots ,h. \end{aligned}$$

By the definition of $\underline{\varvec{x}}$, we have

$$\begin{aligned} \mathscr {T}_j=\bigcap _{k=1}^{h}\{\varvec{x}\in \mathbb {R}^h: |x_j|\ge |x_k|\} \end{aligned}$$

and $\{\varvec{x}\in \mathbb {R}^h: |x_j|\ge |x_k|\}$ can be written as a union of some semi-algebraic sets. Hence, $\mathscr {T}_j$ is semi-algebraic. The graph of $f_1(\varvec{x})$ is

$$\begin{aligned} \bigcup _{j=1}^h \left( \left\{ (\varvec{x},y)\in \mathbb {R}^{h+1}:y = b_1 |x_j| \right\} \cap (\mathscr {T}_j\times \mathbb {R}) \right) , \end{aligned}$$

which is a semi-algebraic set.