Preconditioning for Orthogonal Matching Pursuit with Noisy and Random Measurements: The Gaussian Case

Abstract

The success of orthogonal matching pursuit (OMP) in the sparse signal recovery heavily depends on its ability for correct support recovery. Based on a support recovery guarantee for OMP expressed in terms of the mutual coherence, and a result about the concentration of the extreme singular values of a Gaussian random matrix, this paper proposes a preconditioning method for increasing the recovery rate of OMP from random and noisy measurements. Compared to several existing preconditionings, the proposed method can reduce the mutual coherence with a proven high probability. Simultaneously, the proposed preconditioning can also succeed with a high probability in providing slight signal-to-noise ratio reduction, which is empirically shown to be less severe than that caused by a recently suggested technique for the noisy case. The simulations show the advantages of the proposed preconditioning over other currently relevant ones in terms of both the performance improvement for OMP, and computation time.

Appendix: Some Necessary Results for the Proofs

For derivations in Sect. 3, we need several results summarized in the theorems below.

Theorem 6

(Theorem 3.4 in [11]) Let $\mathbf {A}\in {\mathbb {R}}^{m\times m}$ be a symmetric positive definite matrix with eigenvalues $\lambda _{1}\ge \cdots \ge \lambda _{m}>0$. For any two nonzero independent vectors $\mathbf {x},\mathbf {y}\in {\mathbb {R}}^{m}$, $\theta \in (0,\pi /2]$ is the angle between the lines corresponding to $\mathbf {x}$ and $\mathbf {y}$ such that $\Vert \mathbf {x}\Vert _{2}\Vert \mathbf {y}\Vert _{2}\cos \,\theta =\left| \mathbf {x}^{\mathrm {T}}\mathbf {y}\right|$. Then,

$$\begin{aligned} \frac{\left| \mathbf {x}^{\mathrm {T}}\mathbf {Ay}\right| }{\left( \mathbf {x}^{\mathrm {T}}\mathbf {Ax}\right) ^{1/2}\cdot \left( \mathbf {y}^{\mathrm {T}}\mathbf {Ay}\right) ^{1/2}} \le \frac{\chi +\cos \,\theta }{1+\chi \cos \,\theta } =\frac{\chi +\left| \mathbf {x}^{\mathrm {T}}\mathbf {y}\right| / \left( \Vert \mathbf {x}\Vert _{2}\cdot \Vert \mathbf {y}\Vert _{2}\right) }{1+\chi \cdot \left| \mathbf {x}^{\mathrm {T}}\mathbf {y}\right| /\left( \Vert \mathbf {x}\Vert _{2} \cdot \Vert \mathbf {y}\Vert _{2}\right) }, \end{aligned}$$

where $\chi \doteq (\lambda _{1}/\lambda _{m}-1)/(\lambda _{1}/\lambda _{m}+1)$.

Theorem 7

(The concentration of the singular values of a Gaussian random matrix in [7]) Suppose that $\mathbf {\Phi }\in {\mathbb {R}}^{m\times n}$, $m<n$, is a Gaussian random matrix whose entries $\varphi _{ij}\buildrel {\textit{i.i.d.}}\over {\sim }N(0,m^{-1})$, the singular values of the matrix $\mathbf {\Phi }$ satisfy

$$\begin{aligned}&\Pr \,\left\{ \sqrt{n/m}(1-\epsilon )-1\le \sigma _{i}\le 1 +\sqrt{n/m}(1+\epsilon ),i\in [m]\right\} \ge 1-2\exp \,(-n\epsilon ^{2}/2),\\&\quad \forall \,\epsilon >0. \end{aligned}$$

Theorem 8

(Appendix 1 in [5]) Suppose that $\mathbf {\Phi }$ is an $m\times n$ Gaussian measurement matrix, for any $0<\epsilon <1$, and a constant $a\in (0,1)$,

$$\begin{aligned} \Pr \,\{\mu \,(\mathbf {\Phi })\ge \epsilon \} \le n(n-1) \left[ \exp \,\left( -\frac{ma^{2}\epsilon ^{2}}{4(1+a\epsilon /2)} \right) +\exp \,\left( -\frac{m}{4}(1-a)^{2}\right) \right] . \end{aligned}$$

Theorem 9

(Appendix D in [19]) Given a centralized $\chi ^{2}$-variable Z with m degrees of freedom, then $\forall \,t>0$, there are

$$\begin{aligned} \Pr \, \{ Z-m\le -2\sqrt{mt} \} \le \exp \,(-t), \quad \text {and}\quad \Pr \,\{Z-m\ge 2\sqrt{mt}+2t\} \le \exp \,(-t). \end{aligned}$$