Oblique Projection Matching Pursuit

Abstract

Recent theory of compressed sensing (CS) tells us that sparse signals can be reconstructed from a small number of random samples. In reconstruction of sparse signals, greedy algorithms, such as the orthogonal matching pursuit (OMP), have been shown to be computationally efficient. In this paper, the performance of OMP is shown to be dependent on how well information of the underlying signals is preserved in the residual vector. Further, to improve the information preservation, we present a modification of OMP, called oblique projection matching pursuit (ObMP), which updates the residual in a oblique projection manor. Using the restricted isometric property (RIP), we build a solid yet very intuitive grasp of the more accurate phenomenon of ObMP. We also show from empirical experiments that the ObMP achieves improved reconstruction performance over the conventional OMP algorithm in terms of support detection ratio and mean squared error (MSE).

Appendices

Appendix A: Proof of Proposition 1

Proof

Let Ω = T ^k∪Λ _α . Then, the projection of y onto span (Φ _Ω) can be given by

$$ \mathbf{P}_{\Omega} \mathbf{y} = \boldsymbol{\Phi}_{\Omega} \boldsymbol{\Phi}^{\dag}_{\Omega} \mathbf{y}. $$

(40)

Following the argument in [17], we can show that

$$\begin{array}{@{}rcl@{}} \mathbf{P}_{\Omega} \mathbf{y} = \boldsymbol{\Phi}_{T^{k}} (\mathbf{P}_{{\Lambda}_{\alpha}}^{\bot} \boldsymbol{\Phi}_{T^{k}})^{\dag} \mathbf{y} + \boldsymbol{\Phi}_{{\Lambda}_{\alpha}} (\mathbf{P}_{T^{k}}^{\bot} \boldsymbol{\Phi}_{{\Lambda}_{\alpha}})^{\dag} \mathbf{y}. \end{array} $$

(41)

Since P _Ω y lies in the column span of $\boldsymbol {\Phi }_{\Omega } = [\boldsymbol {\Phi }_{T^{k}}, \boldsymbol {\Phi }_{{\Lambda }_{\alpha }}]$ where $\boldsymbol {\Phi }_{T^{k}}$ and $\boldsymbol {\Phi }_{{\Lambda }_{\alpha }}$ are disjoint, it has a unique linear representation with $\boldsymbol {\Phi }_{T^{k}}$ and $\boldsymbol {\Phi }_{{\Lambda }_{\alpha }}$. That is,

$$\begin{array}{@{}rcl@{}} \mathbf{P}_{\Omega} \mathbf{y} &=& [\boldsymbol{\Phi}_{T^{k}}, \boldsymbol{\Phi}_{{\Lambda}_{\alpha}}] \left[ \begin{array}{c} \widehat{\mathbf{x}}_{T^{(k)}}\\ \widehat{\mathbf{x}}_{\Lambda} \end{array} \right] \\ &=& \boldsymbol{\Phi}_{T^{k}} \mathbf{x}_{T^{k}}^{k} + \boldsymbol{\Phi}_{{\Lambda}_{\alpha}} \mathbf{x}_{{\Lambda}_{\alpha}}^{k}. \end{array} $$

(42)

From Eqs. 41 and 42, we have

$$ \mathbf{x}_{T^{k}}^{k} = (\mathbf{P}_{{\Lambda}_{\alpha}}^{\bot} \boldsymbol{\Phi}_{T^{k}})^{\dag}) \mathbf{y}. $$

(43)

Furthermore, from Table 1, the residual vector of ObMP becomes

$$\begin{array}{@{}rcl@{}} \mathbf{r}_{1}^{k} & = & \mathbf{y} - \boldsymbol{\Phi}_{T^{k}} \mathbf{x}^{k}_{T^{k}} \\ & = & \mathbf{y} - \boldsymbol{\Phi}_{T^{k}} (\mathbf{P}_{{\Lambda}_{\alpha}}^{\bot} \boldsymbol{\Phi}_{T^{k}})^{\dag}) \mathbf{y} \\ & = & (\mathbf{I} - \boldsymbol{\Phi}_{T^{k}} (\mathbf{P}_{{\Lambda}_{\alpha}}^{\bot} \boldsymbol{\Phi}_{T^{k}})^{\dag}) \mathbf{y}, \end{array} $$

(44)

which completes the proof. □

Appendix B: Proof of Proposition 2

Proof

First, for the OMP algorithm, $\| \mathbf {r}^{k} - \mathbf {r}_{0}^{k} \|_{2}$ can be lower bounded as

$$\begin{array}{@{}rcl@{}} \| \mathbf{r}^{k} - \mathbf{r}_{0}^{k} \|_{2} & = & \| \mathbf{P}_{T^{k}} \boldsymbol{\Phi}_{T \backslash T^{k}} \mathbf{x}_{T \backslash T^{k}} \|_{2} \\ & = & \| \boldsymbol{\Phi}_{T^{k}} (\boldsymbol{\Phi}^{\prime}_{T^{k}} \boldsymbol{\Phi}_{T^{k}})^{- 1} \boldsymbol{\Phi}^{\prime}_{T^{k}} \boldsymbol{\Phi}_{T \backslash T^{k}} \mathbf{x}_{T \backslash T^{k}} \|_{2} \\ & = & \| (\boldsymbol{\Phi}^{\dag}_{T^{k}})^{\prime} \boldsymbol{\Phi}^{\prime}_{T^{k}} \boldsymbol{\Phi}_{T \backslash T^{k}} \mathbf{x}_{T \backslash T^{k}} \|_{2} \\ & \overset{(a)}{\geq} & \frac{\| \boldsymbol{\Phi}^{\prime}_{T^{k}} \boldsymbol{\Phi}_{T \backslash T^{k}} \mathbf{x}_{T \backslash T^{k}} \|_{2}}{\sqrt{1 + \delta_{K}}} \\ & {\geq} & \frac{\sigma_{\min} (\boldsymbol{\Phi}_{T^{k}}) \| \boldsymbol{\Phi}_{T \backslash T^{k}} \mathbf{x}_{T \backslash T^{k}} \|_{2}}{\sqrt{1 + \delta_{K}}} \\ & {\geq} & \sqrt{\frac{1 - \delta_{K}}{1 + \delta_{K}}} \| \boldsymbol{\Phi}_{T \backslash T^{k}} \mathbf{x}_{T \backslash T^{k}} \|_{2} \\ & \overset{\text{RIP}}{\geq} & \frac{1 - \delta_{K}}{\sqrt{1 + \delta_{K}}} \| \mathbf{x}_{T \backslash T^{k}} \|_{2}. \end{array} $$

(45)

where (a) is from Lemma 5.

We next consider the ObMP algorithm. $\| \mathbf {r}_{1}^{k} - \mathbf {r}_{0}^{k} \|_{2}$ can be upper bounded as follows:

$$\begin{array}{@{}rcl@{}} {\| \mathbf{r}_{1}^{k} \,-\, \mathbf{r}_{0}^{k} \|_{2}}\!\!& = & \| \mathbf{Q}_{T^{k}} \boldsymbol{\Phi}_{T \backslash T^{k}} \mathbf{x}_{T \backslash T^{k}} \|_{2} \\ & \overset{(20)}{=} & \| \boldsymbol{\Phi}_{T^{k}} (\mathbf{P}_{{\Lambda}_{\alpha}}^{\bot} \boldsymbol{\Phi}_{T^{k}})^{\dag} \boldsymbol{\Phi}_{T \backslash T^{k}} \mathbf{x}_{T \backslash T^{k}} \|_{2} \\ & \overset{\text{RIP}}{\leq} & \sqrt{1 + \delta_{K}} \| (\mathbf{P}_{{\Lambda}_{\alpha}}^{\bot} \boldsymbol{\Phi}_{T^{k}})^{\dag} \boldsymbol{\Phi}_{T \backslash T^{k}} \mathbf{x}_{T \backslash T^{k}} \|_{2} \\ & \overset{(a)}{\leq} &\! \frac{\sqrt{1 + \delta_{K}}}{1 - \delta_{K}} \| \boldsymbol{\Phi}^{\prime}_{T^{k} \backslash {\Lambda}_{\alpha}} \mathbf{P}_{{\Lambda}_{\alpha}}^{\bot} \boldsymbol{\Phi}_{T \backslash (T^{k} \cup {\Lambda}_{\alpha})} \mathbf{x}_{T \backslash (T^{k} \cup {\Lambda}_{\alpha})} \|_{2} \\ & \overset{(b)}{\leq} & \frac{\delta_{K + \lceil \alpha K \rceil} \sqrt{1 + \delta_{K}}}{1 - \delta_{K}} \| \mathbf{x}_{T \backslash (T^{k} \cup {\Lambda}_{\alpha})} \|_{2} \\ & \overset{(c)}{\leq} & \frac{\delta_{2K} \sqrt{1 + \delta_{K}}}{1 - \delta_{K}} \| \mathbf{x}_{T \backslash (T^{k} \cup {\Lambda}_{\alpha})} \|_{2}, \end{array} $$

(46)

where (a) is from Lemma 3, (b) is due to Lemma 4, and (c) is because α∈(0,1].

The proof is now complete. □