Minimization of the $q$-ratio sparsity with $1<q\le \infty$ for signal recovery

Highlights

• We propose a general scale invariant approach for sparse signal recovery via the minimization of the q-ratio sparsity.

• We establish a verifiable exact reconstruction condition and derive concise error bounds in terms of q-ratio constrained minimal singular values (CMSV).

• We investigate two kinds of methods for solving the proposed problem including the parametric methods and the change of variable method.

• We conduct numerical experiments to demonstrate the advantageous performance of the proposed approaches over the state-of-the-art sparse recovery methods.

Abstract

In this paper, we propose a general scale invariant approach for sparse signal recovery via the minimization of the $q$-ratio sparsity $s_q\left(z\right)={(\parallel z\parallel }_1/{\parallel z\parallel }_q){}^{\frac{q}{q-1}}$ with $q\in [0,\infty ]$. The properties of the $q$-ratio sparsity measure are studied and illustrated with examples. For the proposed $q$-ratio sparsity minimization problem with $1<q\le \infty$, we establish a verifiable exact reconstruction condition and derive its concise error bounds in terms of $q$-ratio constrained minimal singular values (CMSV). From an algorithmic point of view, we recognize that the proposed problem belongs to the nonlinear fractional programming and investigate two kinds of methods for solving it including the parametric methods and the change of variable method. Numerical experiments are conducted to demonstrate the advantageous performance of the proposed approaches over the state-of-the-art sparse recovery methods.

Introduction

Over the past decade, an extensive literature on compressive sensing (CS) has been developed, see the monographs [10], [14] for a comprehensive view. CS aims to find the sparsest solution $x\in {\mathbb{R}}^N$ from few noisy linear measurements $y=Ax+\epsilon \in {\mathbb{R}}^m$ where $A\in {\mathbb{R}}^{m\times N}$ with $m\ll N$, and ${\parallel \epsilon \parallel }_2\le \eta$, which can be formulated as solving a constrained ${\ell }_0$-minimization problem:

$$\underset{z\in {\mathbb{R}}^N}{min}{\parallel z\parallel }_0\text{subject}\text{to}{\parallel Az-y\parallel }_2\le \eta .$$

Unfortunately, it is a combinatorial problem which is known to be computationally NP-hard to solve [26]. Instead, a widely used solver is the following constrained ${\ell }_1$-minimization problem [9]:

$$\underset{z\in {\mathbb{R}}^N}{min}{\parallel z\parallel }_1\text{subject}\text{to}{\parallel Az-y\parallel }_2\le \eta ,$$

which acts as a convex relaxation of ${\ell }_0$-minimization. In order to make the CS construction process much more efficient and robust, different sparse recovery methods exist in the literature, such as CS with prior information [25], robust CS [17] and Bayesian CS [18], [45].

Besides, a variety of non-convex recovery methods have been proposed to enhance sparsity, including ${\ell }_p$ ($0<p<1$) [4], [5], [6], [13], [42], smoothed-L0 (SL0) [24], ${\ell }_1-{\ell }_2$ [22], [23], [44], transformed ${\ell }_1$ (TL1) [48], smoothly clipped absolute deviation (SCAD) [12], minimax concave penalty (MCP) [46], and ${\ell }_1/{\ell }_2$ [30], [40], to name a few. These non-convex methods result in the difficulties of theoretical analysis and computational algorithms, but do lead to better recovery performances compared to the convex ${\ell }_1$-minimization in certain contexts. Among them, ${\ell }_p$ gives superior results for incoherent measurement matrices, while ${\ell }_1-{\ell }_2$ and ${\ell }_1/{\ell }_2$ are better choices for highly coherent measurement matrices. TL1 is a robust choice no matter whether the measurement matrix is coherent or not.

Regarding the ${\ell }_1/{\ell }_2$ method, there are few literatures concerning on it due to its complex structure. In fact, ${\ell }_1/{\ell }_2$ is neither convex nor concave, and it is not even globally continuous. Some theoretical analyses have been done when it is restricted to non-negative signals [11], [43]. Very recently, a few new attempts have been made. Rahimi et al. [30] gave some local optimality results and proposed to solve it based on the Alternating Direction Method of Multipliers (ADMM) [2]. Some accelerated schemes were used for the ${\ell }_1/{\ell }_2$ minimization in [40]. Wang et al. [39] adopted the ${\ell }_1/{\ell }_2$ minimization in computed tomography (CT) reconstruction. Xu et al. [41] investigated exact recovery conditions as well as the stability of the ${\ell }_1/{\ell }_2$ method, and provided the conditions under which ${\ell }_0$ minimization is equivalent to ${\ell }_1/{\ell }_2$ minimization. Petrosyan et al. [28] showed that the ${\ell }_1/{\ell }_2$ method outperforms the ${\ell }_1$ minimization in jointly sparse vectors reconstruction problems and can be effectively solved via manifold optimization methods. Meanwhile, [1] proposed a proximal subgradient algorithm with extrapolations for solving nonconvex and nonsmooth fractional programmings which encompass the ${\ell }_1/{\ell }_2$ minization problem.

Inspired by the fact that ${\parallel z\parallel }_1^2/{\parallel z\parallel }_2^2$ is a special case of the $q$-ratio sparsity measure $s_q\left(z\right)={\left(\frac{{\parallel z\parallel }_1}{{\parallel z\parallel }_q}\right)}^{\frac{q}{q-1}}$ [21] with $q=2$, we propose in this paper a more general scale invariant approach for sparse signal recovery via minimizing the $q$-ratio sparsity measure $s_q(·)$. We aim to theoretically and numerically investigate the minimization of the $q$-ratio sparsity. The main contributions of the present paper are four folds: (1) We give a further study on the properties of $q$-ratio sparsity and illustrate them with examples. (2) We propose the minimization of $q$-ratio sparsity for sparse signal recovery, which encompasses the well-known ${\ell }_0$-minimization, ${\ell }_1/{\ell }_2$ and ${\ell }_1/{\ell }_{\infty }$ methods. (3) We give a verifiable sufficient condition for the exact sparse recovery and derive concise bounds on both ${\ell }_q$ norm and ${\ell }_1$ norm of the reconstruction error for the ${\ell }_1/{\ell }_q$ with $q\in (1,\infty ]$ in terms of the $q$-ratio constrained minimal singular values (CMSV) introduced in [51]. We establish the corresponding stable and robust recovery results involving both sparsity defect and measurement error. To the best of our knowledge, in the study of this kind of non-convex methods we are the first to establish the results for the compressible (not exactly sparse) case since all the literature mentioned above merely considered the exactly sparse case. (4) We present efficient algorithms to solve the proposed methods via nonlinear fractional programming and conduct various numerical experiments to illustrate their superior performance.

The paper is organized as follows. In Section 2, we present the definition of $q$-ratio sparsity and some further study on its properties. In Section 3, we propose the sparse signal recovery methodology via the minimization of $q$-ratio sparsity. In Section 4, we provide a verifiable sufficient condition for the exact sparse recovery and derive the reconstruction error bounds based on $q$-ratio CMSV for the proposed method in the case of $1<q\le \infty$. In Section 5, we design algorithms to solve the problem. Section 6 contains the numerical experiments. Finally, conclusions and future works are included in Section 7.

Throughout the paper, we denote vectors by lower case letters e.g., $x$, and matrices by upper case letters e.g., $A$. Vectors are columns by defaults. $x^T$ denotes the transpose of $x$, while the notation $x_i$ denotes the $i$th component of $x$. We introduce the notations $\left[N\right]$ for the set $\{1,2,\cdots ,N\}$ and $\left|S\right|$ for the cardinality of a set $S$. Furthermore, we write $S^c$ for the complement $\left[N\right]\setminus S$ of a set $S$ in $\left[N\right]$. The support of a vector $x\in {\mathbb{R}}^N$ is the index set of its nonzero entries, i.e., $\mathrm{supp}\left(x\right):=\{i\in \left[N\right]:x_i\ne 0\}$. For any vector $x\in {\mathbb{R}}^N$, we denote by ${\parallel x\parallel }_0={\sum }_{i=1}^N{1}_{\{x_i\ne 0\}}=\left|\mathrm{supp}\left(x\right)\right|$ and we say $x$ is $k$-sparse if at most $k$ of its entries are nonzero, i.e., if ${\parallel x\parallel }_0\le k$. The ${\ell }_q$-norm ${\parallel x\parallel }_q=({\sum }_{i=1}^N|x_i{{|}^q)}^{1/q}$ for any $q\in (0,\infty )$, while ${\parallel x\parallel }_{\infty }={\max }_{1\le i\le N}\left|x_i\right|$. For a vector $x\in {\mathbb{R}}^N$ and a set $S\subseteq \left[N\right]$, we denote by $x_S$ the vector which coincides with $x$ on the indices in $S$ and is extended to zero outside $S$. In addition, for any matrix $A\in {\mathbb{R}}^{m\times N}$, we denote the kernel of $A$ by $\ker \left(A\right)=\{x\in {\mathbb{R}}^N|Ax=0\}$.