Model based Bayesian compressive sensing via Local Beta Process

Highlights

• This paper is dealing with the recovery problem for model based compressive sensing.

• This paper has proposed a hierarchical Bayesian model to describe the model based compressive sensing.

• Local Beta Process has been applied to describe the inherent structures of the sparse signals.

• Variational Bayesian approach has been exploited to implement the Bayesian inference.

Abstract

In the framework of Compressive Sensing (CS), the inherent structures underlying sparsity patterns can be exploited to promote the reconstruction accuracy and robustness. And this consideration results in a new extension for CS, called model based CS. In this paper, we propose a general statistical framework for model based CS, where both sparsity and structure priors are considered simultaneously. By exploiting the Latent Variable Analysis (LVA), a sparse signal is split into weight variables representing values of elements and latent variables indicating labels of elements. Then the Gamma-Gaussian model is exploited to describe weight variables to induce sparsity, while the beta process is assumed on each of the local clusters to describe inherent structures. Since the complete model is an extension of Bayesian CS and the process is for local properties, it is called Model based Bayesian CS via Local Beta Process (MBCS-LBP). Moreover, the beta process is a Bayesian conjugate prior to the Bernoulli Process, as well as the Gamma to Gaussian distribution, thus it allows for an analytical posterior inference through a variational Bayes inference algorithm and hence leads to a deterministic VB-EM iterative algorithm.

Introduction

Compressive Sensing (CS) provides a new sampling paradigm that allows signals to be sampled with sub-Nyquist rate (2 times of signals׳ Fourier bandwidth) [1]. In the framework of CS, samples are collected via a linear projection satisfying restricted isometry property, and provided that signals are sparse (itself or under transform), the exact recovery can be guaranteed theoretically [2]. The reconstruction of signals is often casted as the following regularized optimization problem:

$${\mathit{\theta }}^{\star }=\underset{\theta }{\arg \min }\frac{1}{2}{\Vert \mathit{y}-\Phi \mathit{\theta }\Vert }_2^2+\lambda ·\psi \left(\mathit{\theta }\right)$$

with $\mathit{\theta }\in {\mathbb{R}}^n$ being the sparse signals, $\Phi \in {\mathbb{R}}^{m\times n}$ the sensing matrix and $\mathit{y}\in {\mathbb{R}}^m$ the collected samples. $\psi :{\mathbb{R}}^n\to {\mathbb{R}}_{+}$ is a regularization term that induces the sparsity of solutions, e.g. the ℓ_p norm with $p\in [0,1]$. $\lambda >0$ is a constant that balances distortion and sparsity.

Exploiting sparse regularization term, lots of algorithms have been proposed recently such as Linear Programming (Basis Pursuit) methods [3] and iterative soft thresholding algorithm [4] with p=1, greedy methods [5], [6] and iterative hard thresholding algorithm [7] with p=0, iterative re-weighted least squares regression [8] and Bayesian methods [9], [10] that can be considered to solve the regularized optimization problem with $p\in (0,1]$.

On the other hand, besides the sparse prior, structural constraint on the support of the sparse signals has been found in many practical applications. The most common example is the wavelet coefficients that can be considered as a tree-structured sparse vector [11], [12] due to the relationship between two conjuncted wavelet scales. Another application is ECG telemonitoring where Zhang et al. [13] exploited the block structure on the ECG signal to improve the compression performance. Similar block/cluster structure has also been exploited in ISAR imaging where the support constraints are due to the continuity of the target scene [14], [15], [16] and image denoising where the similar patches share the same sparsity patterns [17].

Theoretically, it can be proven that the introduction of structural priori largely improves the reconstruction performance [18], [19]. Considering structured sparse signal model leads to an extension to CS, i.e. Model-based Compressive Sensing (MCS) [19]. Accordingly, many new algorithms are proposed by imposing structural sparse regularization on (1), for instance the mixed ${\ell }_{1,2}$ norm that has been widely employed to cope with problems with block structured sparsity [20] or multiple measurement vector model [17]. Other type of approaches has been exploited to cope with different structures. Group-structured extension to OMP [21], Block-OMP [22], Tree-based OMP [19], etc. are OMP-based approaches that proposed recently. However, the above methods are parametric approaches that require to set structure-related parameters at first. Another type of approaches is based on Bayesian CS framework [9], including Tree-based Bayesian CS [11], Cluss-MCMC/-VB [23], [24], BSBL [25] and so on. The Bayesian approaches are non-parametric, but only specific to one type of structures. More recently, a general model for structured sparse signals is proposed via the Boltzmann machine [26], [27], while the interaction matrix for the structural model should be set or pre-estimated.

In this paper, we propose a general statistical framework for model based CS, where both sparsity and structure priors are considered simultaneously. By exploiting the Latent Variable Analysis (LVA), a sparse signal is split into weight variables representing values of elements and latent variables indicating labels of elements. Then we assume that weight variables obey a Gamma-Gaussian model to induce the sparsity. On the other hand, according to the inherent structures, the latent variables can be described by a graph with local clusters, thus a beta-Bernoulli process is assumed on each of the local clusters to describe the properties of structures. Since the complete model is an extension of Bayesian CS and the process is for local properties, it is called Model based Bayesian CS via Local Beta Process (MBCS-LBP). Moreover, the beta process is a Bayesian conjugate prior to the Bernoulli Process, as well as the Gamma to Gaussian distribution, thus it allows for an analytical posterior inference through a variational Bayes inference algorithm and hence leads to a deterministic VB-EM iterative algorithm.

The rest of paper is organized as follows. In Section 2, we briefly review the framework of Bayesian CS. Then the proposed structured sparsity model is presented in Section 3. After that, the posterior inference through the variational Bayes approach is given in Section 4. Experiments are carried out in Section 5 to verify superior performance of the proposed algorithm. The paper ends up with a conclusion.