A Simple Compressive Sensing Algorithm for Parallel Many-Core Architectures

Abstract

In this paper we consider the l ¹-compressive sensing problem. We propose an algorithm specifically designed to take advantage of shared memory, vectorized, parallel and many-core microprocessors such as the Cell processor, new generation Graphics Processing Units (GPUs) and standard vectorized multi-core processors (e.g. quad-core CPUs). Besides its implementation is easy. We also give evidence of the efficiency of our approach and compare the algorithm on the three platforms, thus exhibiting pros and cons for each of them.

Appendix: Proof of Convergence

We briefly show the standard elements of the proof of the convergence of the proximal iterations. Recall that the approach is quite standard and some proof can be adapted for [32, 35] for instance.

First, let us note that the series $\{E_\mu\left(p(u^{(k)})\right)\}$ is clearly non-increasing and bounded by below, and thus converges toward some value referred to as η.

Then, let us recall a standard convex optimality result. Assume that $g:\mathbb{\bf{R}}^N\rightarrow \mathbb{\bf{R}}$ is convex and differentiable and $h:\mathbb{\bf{R}}^N \rightarrow \mathbb{\bf{R}}$ is convex, then u ^⋆ is a global minimizer of (g + h) if and only if the following holds:

$$ \label{eq.opt-sum-convex} \forall u \in \mathbb{\bf{R}}^N\,\, \langle \nabla g(u^\star), u - u^\star \rangle + h(u) - h(u^\star) \geq 0\enspace. $$

(6)

For our case, let us have $g(\cdot) = \frac{1}{2}\| \cdot - u ^{(k)}\|_M^2$ and h(·) = E _μ (·) and recall that $p_\mu(u ^{(k)})$ is the global minimum of the inf-convolution when it is fed with u ^(k):

$$\begin{array}{rll} \label{eq.opt-sum-convex-for-our-case} \forall u \in \mathbb{\bf{R}}^N\,\, \left\langle p\left(u ^{(k)}\right) - u ^{(k)} , u - p\left(u ^{(k)}\right) \right\rangle_M \nonumber\\ + E_\mu(u) - E_\mu\left(p\left((u ^{(k)}\right)\right) \geq 0\enspace. \end{array} $$

(7)

Now, we consider this inequality for the two points $\hat{u}$ and $\bar{u}$ with associated proximal points $p(\hat{u})$ and $p(\bar{u})$, respectively. Some simple calculus lead to:

$$\begin{array}{rll} &&{\kern-7pt}\left\langle p(\hat{u}) - \hat{u}, p(\bar{u}) - p(\hat{u}) \right\rangle_M \\&&+ \left\langle p(\bar{u}) - \bar{u}, p(\hat{u}) - p(\bar{u}) \right\rangle_M \geq 0 \enspace, \end{array}$$

and thus we get:

$$ \left\langle \bar{u} - \hat{u}, p(\bar{u}) - p(\hat{u}) \right\rangle_M \geq \| p(\hat{u}) - p(\bar{u})\|_M^2 \enspace. $$

The latter is equivalent to:

$$ \left\| \hat{u} - \bar{u} \right\|_M^2 - \| p(\hat{u}) - \hat{u} - p(\bar{u}) + \bar{u} \|_M^2 \geq \| p(\hat{u}) - p(\bar{u}) \|_M^2 \enspace\!. $$

Now, let us set $\bar{u}$ to a global minimizer of R _μ , i.e., $\bar{u} = u^\star$, and $\hat{u} = u^{(k)}$, in the previous inequality. Note that $p\left(u^\star\right) = u^\star$, and recall that $u^{(k+1)} = p\left(u^{(k)}\right)$. The following inequality holds:

$$ \label{eq.ineq-wanted-for-convergence} \left\| u ^{(k)} - u^\star \right\|_M^2 - \left\| u^{(k+1)} - u^{(k)} \right\|_M^2 \geq \left\| u^{(k+1)} - u^\star \right\|_M^2 $$

(8)

With this inequality we can conclude using the following points.

The series $\| u^{(k)} - u^\star\|_M^2$ is non-increasing and bounded by below (by $E_\mu(u ^\star))$) and thus does converge. Thus, we deduce that $\lim_{k \rightarrow \infty} \left( \| u^{(k+1)} - u^\star \|_M^2 - \| u^{(k)} -\right.$ $\left.u^\star \|_M^2 \right) = 0\,.$ Using this result and Eq. 7 we get that $\lim_{k\rightarrow \infty} \| u^{(k+1)} - u^{(k)}\|_M^2 = 0\,.$

Note that by the convexity of the energy E _μ , we have:

$$ \label{eq.convergence-comvexite-optim} E_\mu\left(u ^\star\right) \geq E_\mu\left(u ^{(k+1)}\right) + \left\langle \partial E_\mu (u ^{(k+1)}) , u^{\star} - u^{(k+1)}\right\rangle $$

(9)

Since u ^(k + 1) is the global minimizer of F _μ , we have that:

$$\begin{array}{rll} \label{eq.convergence-optim-prox} &&{\kern-8pt}\left\langle \partial E_\mu (u ^{(k+1)}) , u^{\star} - u^{(k+1)}\right\rangle\notag\\&& + \left\langle u^{(k+1)} - u^{(k)}, u^\star - u^{(k+1)} \right\rangle \geq 0\enspace. \end{array} $$

(10)

Recall that, as it has been shown above, $\lim_{k\rightarrow + \infty} \| u^{(k+1)} - u^{(k)}\|^2 = 0$, and since ||u ^(k + 1) − u ^⋆||² is bounded we have that $\lim \inf_{k\rightarrow + \infty} \left\langle \partial E_\mu(u ^{(k+1)}),{\kern-4.5pt} \right.$ $\left.u^{\star} - u^{(k+1)}\right\rangle \geq 0\,.$ Injecting this information into Eq. 8, we obtain that $\lim_{k \rightarrow + \infty} E_\mu\left(u^{(k)}\right) \leq \eta$, and thus we conclude that $\lim_{k\rightarrow + \infty} E_\mu(u^{(k)}) = \eta\,.$