Stability and robustness of the l₂/l_q-minimization for block sparse recovery

Highlights

• We give a new sufficient condition to exactly recover block sparse vectors via the l₂/l_q-minimization.

• We investigate the instance optimality of the encoder-decoder pairs $(A,{\Delta }_{l_2/l_q})$.

• We establish the stability and robustness estimate of the l₂/l_q-minimization.

Abstract

This paper focuses on block sparse recovery with the l₂/l_q-minimization for 0 < q ≤ 1. We first give the l_q stable block Null Space Property (NSP), a new sufficient condition to exactly recover block sparse signals via the l₂/l_q-minimization, and it is weaker than the block Restricted Isometry Property (RIP). Second, we propose the l_{p, q}(0 < q ≤ p) robust block NSP and generalize the instance optimality and quotient property to the block sparse case. Furthermore, we show that Gaussian random matrices and random matrices whose columns are drawn uniformly from the sphere satisfy the block quotient property with high probability. Finally, we obtain the stability estimate of the decoder ${\Delta }_{l_2/l_q}^{ϵ}$ for $y=Ax+e$ with a priori ‖e‖₂ ≤ ϵ based on the robust block NSP. In addition, for arbitrary measurement error, we also obtain the robustness estimate of the decoder ${\Delta }_{l_2/l_q}$ for $y=Ax+e$ without requiring the knowledge of noise level, which provides a practical advantage when the estimates of measurement noise levels are absent. The results demonstrate that the l₂/l_q-minimization can perform well for block sparse recovery, and remains not only stable but also robust for reconstructing noisy signals when the measurement matrices satisfy the robust block NSP and the block quotient property.

Introduction

Compressed sensing [4], [6], [14] is a scheme which shows that some signals can be reconstructed from fewer measurements compared to the classical Nyquist-Shannon sampling method. Suppose the observed data $y\in {\mathbb{R}}^n,$ one will recover the signal $x\in {\mathbb{R}}^N$ via the linear system

$$y=Ax+e,$$

where $A\in {\mathbb{R}}^{n\times N}$ (n < N) is a real matrix, called the measurement matrix, and $e\in {\mathbb{R}}^n$ is a noise vector. To extract the information x, one applies a decoder Δ to y which is, typically, a nonlinear operator mapping from ${\mathbb{R}}^n$ to ${\mathbb{R}}^N$. The vector

$$x^{*}=\Delta (Ax+e)$$

is viewed as an approximation to x. The central question of compressed sensing is: What are the good encoder-decoder pairs (A, Δ) [2]? It is natural to hope that there exists an encoder-decoder pair (A, Δ), such that the measure error ${\parallel x-\Delta (Ax+e)\parallel }_q$ is as small as possible, where,

$$\begin{array}{c}\hfill {\parallel x\parallel }_q=\{\begin{array}{cc}\hfill {\left(\sum _{i=1}^N{\left|x_i\right|}^q\right)}^{1/q},& \hfill 0<q<\infty ;\\ \hfill \underset{i=1,\dots ,N}{\max }\left|x_i\right|,& \hfill q=\infty .\end{array}\end{array}$$

To measure the performance of an encoder-decoder pair (A, Δ), one used Gelfand width [2], [13] to characterize the degree of approximation of $x-\Delta \left(Ax\right)$ in the noise-free case. Then, as for a general matrix $A\in {\mathbb{R}}^{n\times N},$ does there exist a decoder Δ, such that $x=\Delta \left(Ax\right)$? In fact, in classic compressed sensing problem, there exists an essential decoder Δ₀:

$$\begin{array}{c}\hfill {\Delta }_0\left(y\right):=\underset{x\in {\mathbb{R}}^N}{\arg \min }{\parallel x\parallel }_{0}s.t.y=Ax,\end{array}$$

which can lead to $x={\Delta }_0\left(Ax\right)$ for all s-sparse vectors $x\in {\mathbb{R}}^N$ whenever $s<\frac{n}{2}$ [16]. Here, ‖x‖₀ denotes the number of non-zero entries of the vector x, an s-sparse vector $x\in {\mathbb{R}}^N$ is defined by ‖x‖₀ ≤ s < <N.

However, the l₀-minimization (4) is a nonconvex and NP-hard optimization problem [32]. To overcome this problem, one proposed the decoder Δ_q or l_q-minimization for 0 < q ≤ 1 [5], [6], [15], [17], [23], [27], [39].

$${\Delta }_q\left(y\right):=\underset{x\in {\mathbb{R}}^N}{\arg \min }{\parallel x\parallel }_{q}s.t.y=Ax.$$

When $q=1,$ [3] proved that the solutions to (5) are equivalent to those of (4) provided that the measurement matrices satisfy the Restricted Isometry Property (RIP) [5] with some definite Restrict Isometry Constant (RIC) δ_s ∈ (0, 1), here δ_s is defined as the smallest constants satisfying

$$(1-{\delta }_s){\parallel x\parallel }_2^2\le {\parallel Ax\parallel }_2^2\le (1+{\delta }_s){\parallel x\parallel }_2^2$$

for any s-sparse vectors $x\in {\mathbb{R}}^N$. For 0 < q < 1, recently, [34] also proved that the solutions to (5) are equivalent to those of (4) as long as q is smaller than a definite constant q₀ < 1, where q₀ depends on A and y. The l_q(0 < q < 1)-minimization is a natural extension from the l₁-minimization to the l₀-minimization, because, compared to the l₁-norm ‖x‖₁, ${\parallel x\parallel }_q^q$ for 0 < q < 1 can make a closer approximation to ‖x‖₀ and remain to induce sparsity. Besides, it was shown that the l_q(0 < q < 1)-minimization often needs less restrictive the RIP requirements, but still could guarantee perfect recovery for smaller q [8], [29], [36] compared to the l₁-minimization. Numerical experiments [7] also showed that the l_q(0 < q < 1)-minimization recovers sparse signals from fewer linear measurements than does the l₁-minimization. Of course, there exist other kind of nonconvex examples which replace the l₀-minimization (4), typically, the smoothly clipped absolute deviation (SCAD) [21], the minimax concave penalty (MCP) [44] and the capped l₁-norm [45]. These examples can often induce better sparsity and reduce the bias, and they are relatively easy to be implemented compared to the l_q(0 < q < 1)-minimization from the algorithmic point of view. But they need theoretical guarantees to determine an appropriate regularization parameter based on the corresponding iterative thresholding algorithm [42]. However, the l_q(0 < q < 1)-minimization strategy offers more theoretical advantages compared to the above mentioned nonconvex relaxed models in compressed sensing. Therefore, this paper mainly focuses on the l_q(0 < q < 1)-minimization in what following.

In addition to recovering sparse vectors from error-free measurement, one requires that the decoder should be robust to noise and stable with regards to the compressibility of $x\in {\mathbb{R}}^N$ [25]. That is $y=Ax+e$ with ‖e‖₂ ≤ ϵ, the decoder reads as:

$${\Delta }_q^{ϵ}\left(y\right):=\underset{x\in {\mathbb{R}}^N}{\arg \min }{\parallel x\parallel }_{q}s.t.{\parallel y-Ax\parallel }_2\le ϵ,$$

which provides a stability estimate of the form (see [3] for $q=1$ and [23] for a general statement with 0 < q < 1.)

$$\begin{array}{c}\hfill \parallel x-{\Delta }_q^{ϵ}{(Ax+e)\parallel }_2\le \frac{C}{s^{1/q-1/2}}{\sigma }_s{\left(x\right)}_q+Dϵ,\end{array}$$

where σ_s(x)_q denotes the best s-term approximation error of $x\in {\mathbb{R}}^N$ in l_q (quasi-)norm where q > 0, i.e.,

$${\sigma }_s{\left(x\right)}_q:=\underset{{\parallel z\parallel }_0\le s}{\inf }{\parallel x-z\parallel }_q.$$

Here and in the rest of the present paper, C and D denote positive absolute constants whose values may change from instance to instance.

The inequality (8) was obtained based on the RIP. However, using the Null Space Property (NSP) [25], [26] (see [25] for $q=1,$ and [26] for 0 < q < 1), one also gets the results similar to (8). Furthermore, if we set $e=0,$ then the inequality (8) reads as

$$\begin{array}{c}\hfill \parallel x-{\Delta }_q{\left(Ax\right)\parallel }_2\le \frac{C}{s^{1/q-1/2}}{\sigma }_s{\left(x\right)}_q.\end{array}$$

This motivates the concept of the instance optimality. Let ${\mathcal{A}}_{n,N}$ denote the set of all encoder-decoder pairs (A, Δ), that is

$${\mathcal{A}}_{n,N}=\{(A,\Delta )|A\in {\mathbb{R}}^{n\times N},\Delta :{\mathbb{R}}^n\to {\mathbb{R}}^N\}.$$

Then the instance optimality is defined as follows.

Definition 1

Let 0 < q ≤ p, for $s\in \left[N\right]=\{1,2,\dots ,N\}$ and for all $x\in {\mathbb{R}}^N,$ there exist a pair $(A,\Delta )\in {\mathcal{A}}_{n,N}$ and a constant C > 0, if

$${\parallel x-\Delta \left(Ax\right)\parallel }_p\le \frac{C}{s^{1/q-1/p}}{\sigma }_s{\left(x\right)}_q,$$

then the encoder-decoder pair (A, Δ) is called the (p, q) instance optimality of order s.

From Definition 1, we can conclude that an s-signal $x\in {\mathbb{R}}^N$ can be exactly recover if there exists a pair (A, Δ) that satisfies the instance optimality. We can also see from (8) that the encoder-decoder pair (A, Δ_q) satisfies (2, q) instance optimality when the matrix A has the RIP or the NSP in terms of different q ≤ 1.

We see that the above stability result (8) requires a proper estimate of the level ϵ of measurement error, but sometimes, it does not need to estimate the bound of ‖e‖₂, that is, a priori ‖e‖₂ ≤ ϵ is not available in many practical scenarios. Therefore, one hopes that the result (8) could be improved, for this purpose, one proposed the following quotient property.

Definition 2

Given a matrix $A\in {\mathbb{R}}^{n\times N},$ if there is a constant α > 0, such that

$$\alpha B_2^n\subset A\left(B_q^N\right),$$

then the matrix A is said to satisfy the l_q quotient property with constant α, where $B_q^N$ denotes the unit ball relative to l_q norm (q ≥ 1) or quasi-norm (0 < q < 1), that is $B_q^N:=\{x\in {\mathbb{R}}^N{,\parallel x\parallel }_q\le 1\}$.

Using the quotient property, one can obtain the following robustness estimate (see [25], [43] for $q=1$ and [35] for 0 < q < 1),

$$\begin{array}{c}\hfill \parallel x-{\Delta }_q{(Ax+e)\parallel }_2\le \frac{C}{s^{1/q-1/2}}{\sigma }_s{\left(x\right)}_q+D{\parallel e\parallel }_2.\end{array}$$

Obviously, (12) indicates that the l_q-minimization can perform well for arbitrary measurement error without estimating the upper bound of ‖e‖₂.

However, the above classic compressed sensing only considers the sparsity of the reconstructed signal, but it does not take into account any further structure. In many practical applications, the reconstructed signal is not only sparse but also non-zero entries are aligned to some blocks rather than being arbitrarily spread throughout the vector. These signals are called the block sparse signals and arise in several areas of signal processing and machine learning, for example, color imaging [31], DNA microarrays [33], equalization of sparse communication channels [11], multi-response linear regression [37], image annotation [28], etc.

To define the block sparsity, it is necessary to introduce some further notations. Suppose that $x\in {\mathbb{R}}^N$ is split into m blocks, $x\left[1\right],x\left[2\right],\dots ,x\left[m\right],$ which are of length $d_1,d_2,\dots ,d_m,$ respectively, that is

$$x={[\underset{x\left[1\right]}{\underbrace{x_1,\dots ,x_{d_1}}},\underset{x\left[2\right]}{\underbrace{x_{d_1+1},\dots ,x_{d_1+d_2}}},\dots ,\underset{x\left[m\right]}{\underbrace{x_{N-d_m+1},\dots ,x_N}}]}^T,$$

and $N={\sum }_{i=1}^m{d}_i$. A vector $x\in {\mathbb{R}}^N$ is called block s-sparse over $\mathcal{I}=\{d_1,d_2,\dots ,d_m\}$ if x[i] is nonzero for at most s indices i [19]. Obviously, $d_i=1$ for each i, the block sparsity reduces to the conventional definition of a sparse vector.

Furthermore, we introduce the following notations,

$$\begin{array}{c}\hfill {\parallel x\parallel }_{2,q}=\{\begin{array}{cc}\hfill {\left(\sum _{i=1}^m{\parallel x\left[i\right]\parallel }_2^q\right)}^{1/q},& \hfill 0<q<\infty ;\\ \hfill \underset{i=1,\dots ,m}{\max }{\parallel x\left[i\right]\parallel }_2,& \hfill q=\infty ;\\ \hfill \sum _{i=1}^m{I(\parallel x\left[i\right]\parallel }_2),& \hfill q=0,\end{array}\end{array}$$

where I(x) is an indicator function that obtains the value 1 if x > 0 and 0 otherwise. So a block s-sparse vector x can be defined by ‖x‖_{2, 0} ≤ s, and ${\parallel x\parallel }_0={\sum }_{i=1}^m{\parallel x\left[i\right]\parallel }_0$. The same as ‖x‖_q, for 1 ≤ q ≤ ∞, ‖x‖_{2, q} is a norm, while for 0 < q < 1, it is a quasi-norm and satisfies the q-triangle inequality:

$${\parallel x+y\parallel }_{2,q}^q\le {\parallel x\parallel }_{2,q}^q+{\parallel y\parallel }_{2,q}^q,x,y\in {\mathbb{R}}^N.$$

Obviously, for an m-block signal $x\in {\mathbb{R}}^N$ whose structure is like (13), then ${\parallel x\parallel }_{2,2}={\parallel x\parallel }_2$ and for 0 < q ≤ p, we have

$${\parallel x\parallel }_{2,p}\le {\parallel x\parallel }_{2,q}\le m^{1/q-1/p}{\parallel x\parallel }_{2,p},$$

and

$${\parallel x\parallel }_{2,q}\le {\parallel x\parallel }_q,$$

especially, ‖x‖_{2, 1} ≤ ‖x‖₁. Let Σ_s denote the set of all block s-sparse vectors: ${\Sigma }_s=\{x\in {\mathbb{R}}^N{:\parallel x\parallel }_{2,0}\le s\}$. Similar to the definition of σ_s(x)_q, we use σ_s(x)_{2, q} to denote the best block s-term approximation error of $x\in {\mathbb{R}}^N$ in l₂/l_q (quasi-)norm where q > 0, i.e.,

$${\sigma }_s{\left(x\right)}_{2,q}:=\underset{z\in {\Sigma }_s}{\inf }{\parallel x-z\parallel }_{2,q}.$$

It is clear that ${\sigma }_s{\left(x\right)}_{2,q}=0$ for all x ∈ Σ_s.

To recover a block sparse signal, similar to the standard l₀-minimization, one seeks the sparsest block sparse vector via the following l₂/l₀-minimization [18], [19], [20],

$${\Delta }_{l_2/l_0}\left(y\right):=\underset{x\in {\mathbb{R}}^N}{\arg \min }{\parallel x\parallel }_{2,0}s.t.y=Ax.$$

But the l₂/l₀-minimization problem is also NP-hard. It is natural to use the l₂/l₁-minimization to replace the l₂/l₀-minimization. Consider [18], [19], [20], [38]

$${\Delta }_{l_2/l_1}\left(y\right):=\underset{x\in {\mathbb{R}}^N}{\arg \min }{\parallel x\parallel }_{2,1}s.t.y=Ax.$$

To characterize the performance of this method, Eldar and Mishali [19] proposed the block Restricted Isometry Property (block RIP).

Definition 3 Block RIP

Given a matrix $A\in {\mathbb{R}}^{n\times N},$ for every block s-sparse $x\in {\mathbb{R}}^N$ over $\mathcal{I}=\{d_1,d_2,\dots ,d_m\},$ there exists a positive constant $0<{\delta }_{s|\mathcal{I}}<1,$ if

$$(1-{\delta }_{s|\mathcal{I}}){\parallel x\parallel }_2^2\le {\parallel Ax\parallel }_2^2\le (1+{\delta }_{s|\mathcal{I}}){\parallel x\parallel }_2^2,$$

then the matrix A satisfies the s-order block RIP over $\mathcal{I}$.

Obviously, the block RIP is an extension of the standard RIP, but it is a less stringent requirement comparing with the standard RIP [1], [19]. Eldar et al. [19] proved that the l₂/l₁-minimization can exactly recover any block s-sparse signal when the measurement matrices A satisfy the block RIP with ${\delta }_{2s|\mathcal{I}}<0.414$. The block RIP constant can be also improved, for example, Lin and Li [30] improved to ${\delta }_{2s|\mathcal{I}}<0.4931,$ and established another sufficient condition ${\delta }_{s|\mathcal{I}}<0.307$ for exact recovery.

Based on the performance of the l_q(0 < q < 1)-minimization [7], [12], [23], it is also natural to extend the l_q-minimization to the setting of block sparse recovery, this motivates us to consider the l₂/l_q-minimization. It is defined by [31], [40], [41]

$${\Delta }_{l_2/l_q}\left(y\right):=\underset{x\in {\mathbb{R}}^N}{\arg \min }{\parallel x\parallel }_{2,q}s.t.y=Ax,$$

or

$${\Delta }_{l_2/l_q}^{ϵ}\left(y\right):=\underset{x\in {\mathbb{R}}^N}{\arg \min }{\parallel x\parallel }_{2,q}s.t.{\parallel y-Ax\parallel }_2\le ϵ$$

for the inaccurate measurement $y=Ax+e$ with ‖e‖₂ ≤ ϵ. Note that ${\Delta }_{l_2/l_q}^0={\Delta }_{l_2/l_q}$.

Like the l_q(0 < q < 1)-minimization, the l₂/l_q(0 < q < 1)-minimization also has superior properties compared to the l₂/l₁-minimization. Some numerical experiments demonstrated that fewer measurements are needed for exact recovery with the decoder ${\Delta }_{l_2/l_q}$ when 0 < q < 1 than when $q=1,$ see [31], [40], [41]. Moreover, [41] studied the exact recovery conditions and gave the stability estimate of the l₂/l_q(0 < q < 1)-minimization based on the block restricted q-isometry property. But there, it requires a proper estimate of the level ϵ of measurement error. However, such an estimate may not be a priori available in some settings, i.e., it is not necessary to estimate the upper bound of ‖e‖₂. Thus, a question arises if the decoder ${\Delta }_{l_2/l_q}$ can perform well for arbitrary measurement error when the estimates of measurement noise levels are absent. Besides, another interesting question arises if the block RIP can be weakened to exactly recover block-sparse vector via the l₂/l_q-minimization. The purpose of this paper is to investigate the above two questions. To achieve this goal, we propose the l_q stable block NSP, which weakens the block RIP. We also propose the l_{p, q} robust block NSP for 0 < q ≤ p, the block quotient property and the block instance optimality, which are very crucial to characterize the stability and robustness of the decoder ${\Delta }_{l_2/l_q}$ for arbitrary noise e without the need to estimate ‖e‖₂.

The remainder of the paper is organized as follows. In Section 2, based on the l_q block NSP, we first give the l_q stable block NSP and derive its an equivalent form, and show it is weaker than the block RIP. Then we further consider the l_q robust block NSP and the l_{p, q} robust block NSP for 0 < q ≤ p, respectively. Using these properties, we effectively characterize reconstruction results for the l₂/l_q-minimization when the observations are corrupted by noise. In Section 3, we give the block instance optimality, the block quotient property and the simultaneous block quotient property. These properties combining with the l_{p, q} robust block NSP will yield two important lemmas, which are the cores of the proof of the main results. In addition, we also show that Gaussian random matrices satisfy the block quotient property with high probability. In Section 4, we give our two main results, Theorems 7 and 8. Section 5 is a discussion and Section 6 includes some conclusions. Finally, we relegate the proofs of the main results, lemmas and proposition, i.e., Theorems 1–Theorem 5, Theorem 7, Theorem 8, Lemma 1, Lemma 2 and Proposition 1 to Appendix A.