Compressive image sensing for fast recovery from limited samples: A variation on compressive sensing
Introduction
In this section, we describe the background of compressed sensing in Section 1.1, discuss related work in Section 1.2, and describe the contributions and overview of proposed method in Section 1.3, before providing the outline of this paper in Section 1.4.
Compressed/Compressive sensing (CS) has received considerable attention recently due to its revolutionary development in simultaneously sensing and compressing signals with certain sparsity. Moreover, the architecture of the so-called single-pixel camera [19], [30] has promoted the practicality of compressed sensing for images. CS is mainly composed of two steps. Let x denote a k-sparse signal of length n to be sensed, let ϕ of dimensionality m × n represent a sampling matrix, and let y be the measurement of length m. At the encoder, a signal x simultaneously is sensed and compressed via random projection, and the obtained samples are called measurements y in the context of compressed sensing. They are related via random projection as: The measurement rate is defined as or which indicates the compression ratio (without quantization) without storing the original signal of length n. At the decoder, the original signal x to be sensed can be perfectly recovered by means of convex optimization or greedy algorithms if the relationship between m and k, i.e., is satisfied [7].
For convex optimization-based CS algorithms, sparse signal recovery will be time-consuming and intractable if ℓ0-minimization is adopted. ℓ0-minimization seeks to find k non-zero entries of a signal if the signal is k-sparse in either the time/space or transform (e.g., DCT or wavelet) domain. The solution can become more tractable if the constraint of ℓ0-minimization is relaxed and ℓ1-minimization is used instead. Several algorithms relying on ℓ1-minimization have been presented in the literature.
In addition to convex optimization, non-convex programming (or greedy) algorithms, like Orthogonal Matching Pursuit (OMP) [41], are an alternative for sparse signal recovery. Basically, OMP has been recognized as a “fast” algorithm with time complexity O(kmn) with reasonable reconstruction quality in some cases.
On the other hand, in the context of compressed sensing (CS) [15], the constraint of sparsity enables the possibility of sparse signal recovery to use the measurements with the number (far) fewer than the original signal length. Moreover, the measurements generated from random projection of the original signal via a sampling matrix are weighted equally; i.e., no measurement is more significant than the others. Thus, CS inherently is weakened in handling less-sparse signals, such as highly textured images. The problem here is if we can yield weighted measurements so that less sparse signals can be quickly reconstructed while maintaining good reconstruction quality. Namely, we seek to find approximate reconstruction instead of an exact reconstruct for multimedia data that permit certain content loss.
In the compressed sensing literature, many studies have explored the structure or correlation inherent in the transformed coefficients in order to better reconstruct the signal from its corresponding measurement vector. Inspired by the concept of JPEG2000 compression, the tree structure of wavelet transform has been exploited popularly.
In [16], [17], instead of capturing non-adaptive or universal measurements, the authors propose attaining adaptive transform coefficients by exploiting the tree structure of the Haar wavelet. In terms of image quality and recovery speed, the so-called adaptive compressed sensing framework demonstrates its superiority over its non-adaptive counterparts.
In [24], a tree-structured Bayesian compressed sensing framework is proposed, wherein the hierarchical statistical models of wavelet and DCT were adopted and Markov chain Monte Carlo (MCMC) inference was employed. The computationally inefficient MCMC mechanism later is replaced with variational analysis in [25] to speed up recovery. Results show that their methods can achieve both accurate and fast CS recovery. The paradigm in [24], [25] belongs to probabilistic structured sparsity [2].
Moreover, the concept of clustered sparsity has received considerable attention in compressed sensing. As summarized in [2] and Table I of [46], many existing CS algorithms [3], [11], [12], [20], [21], [26], [40] exploiting clustered sparsity need to know some pre-defined information, such as numbers, sizes, and positions of clusters, along with the degree of sparsity. In [46], the proposed Bayesian compressed sensing method, a kind of nonparametric recovery algorithm, could make use of clustered sparsity without relying on prior information. Basically, [46] is inspired by [25] in that variational analysis was used in place of MCMC for Bayesian inference in order to guarantee convergence within finite iterations. The major difference between [25] and [46] is that the former employs a directional graphical model for the tree structure of wavelet coefficients, while the latter uses an undirectional graphical model. Furthermore, in order to target the problem of reconstructing structured-sparse signals, belief propagation is employed in [39], which resembles turbo equalization from digital communications. The clustered sparsity-based compressed sensing methods mentioned above belong to deterministic structured sparsity [2].
It should be noted that, in [3], both tree structure and structured sparsity are considered and incorporated into two state-of-the-art CS algorithms, which are CoSaMP [34] and iterative hard thresholding (IHT) [4].
Recently, a so-called N-BOMP (N-way block OMP) method [5] has been developed based on exploiting Kronecker product and block sparsity. The authors prove the equivalence between the Tucker model and Kronecker representation for multiway arrays, thus, Kronecker structure can be used to solve the Tucker model-based underdetermined linear systems within compressive sensing. N-BOMP outperforms the existing tensor-based CS algorithms in that block sparsity of tensor is exploited such that the Kronecker dictionary can be used to speed recovery and improve reconstruction quality. Nevertheless, these advantages come from (also indicated in Subsection 7.2.1 of [5]) the assumption that, for a 2D image, it is pre-processed in advance to possess a perfect block sparsity pattern in that the important/significant coefficients in some transform domains fall within the specified block sparsity pattern while other insignificant coefficients are removed entirely. Therefore, N-BOMP is able to obtain reconstruction quality far better than the existing tensor CS algorithms under the prerequisite/restriction. Later, without making any assumptions about the sparsity pattern, Caiafa and Cichocki [6] present a fast non-iterative tensor compressive sensing method. It, however, assumes that the signal to be sensed and recovered has low multilinear-rank, leading to redundant sensing. This means that, under the same measurement rate, the reconstructed quality is (remarkably) lower than other CS algorithms.
In [29], we propose the use of tree structure sparsity pattern (TSSP) in tensor compressive sensing. TSSP can help to quickly find significant wavelet coefficients and save the execution time to calculate the maximum correlations in greedy algorithms. Its weakness is that there is no fast recovery algorithm that can exploit TSSP.
In addition to the aforementioned sparsity patterns, including the tree structure and clustered/block sparsity, other models of transform coefficients, including Laplacian scale mixtures [8], piecewise autoregressive model [44], Laplace prior [1], and Gaussian Mixture Models [49] also have been employed within the compressed sensing framework. The nice property of structured sparsity mentioned above has been applied to a number of image processing applications beyond reconstruction. In [47], the inverse problems of inpainting and deblurring are solved via the proposed structured sparse model selection algorithm. The key is that the sparsity of local windows partitioned from an image can be better controlled. Basically, stable inversion can be achieved because the degree of freedom in selecting models is equal to the number of bases, and is considerably lower than overcomplete dictionary methods. Further, a work, called piecewise linear estimator (PLE), extended from [47] is presented in [48].
On the other hand, the paradigm of partially known support-based CS methods can be found in [9], [10], [42]. The idea is that the reconstruction quality can be improved if the prior knowledge of the support of the signal to be sensed is involved in the so-called modified CS framework. Typically, the known support of a signal in the wavelet transform domain can be the low-frequency subbands, which approximate/capture the original signal well. Specifically, Vaswani and Lu’s work [42] is basis pursuit (BS) with partially known support while Carrillo et al.’s works [9], [10] belong to greedy algorithms with partially known support.
In this paper, we address the problems of accomplishing fast and accurate recovery of images in the framework of compressed sensing. The goal is to achieve so-called compressed image sensing (CIS) with approximate reconstruction. Our method basically is analogous to the aforementioned CS methods exploiting either known sparsity patterns or partially known support of a signal. Nevertheless, the methodology is fundamentally different in that we focus on the design of a new sampling matrix while the others, in essence, use some prior knowledge in either convex optimization-based or greedy-based CS algorithms.
Our first contribution is to investigate an elaborate design of the sampling matrix ϕ that can directly capture “important” measurements. With these important measurements, the quality original signal can be reconstructed sparsely based on the important (corresponding to low-frequency) components in a transform domain. In our CIS method, the qualities of reconstructed images mimic those of JPEG compressed images. The designed sampling matrix can be embedded readily into mobile devices with camera functionality to simultaneously capture and compress signals, and the sampled measurements can be transmitted efficiently to the decoder or remote server for turbo fast recovery of the captured signals. In particular, our CIS algorithm can be applied to a scenario where mobile device to mobile device (M2M) is considered. In addition, the bottleneck of distributed compressive video sensing (DCVS) [18], [27], [35] at the decoder now can be solved if the proposed method is used.
The second contribution of this paper is to study different sensing strategies for image reconstruction. More specifically, in addition to the commonly adopted 1D sensing, we also propose a 2D sensing strategy and demonstrate its impact on compressed image sensing and reconstruction. We find that 2D sensing indeed can benefit sensing and reconstruction of images. In addition, compressed sensing of large images is a challenge for compressed sensing of signals in a 1D form. We show that better reconstruction quality and speed can be achieved by means of 2D separate sensing.
The rest of this paper is organized as follows. The use of compressed sensing for capturing natural images is called compressed image sensing (CIS) and is discussed in Section 2. In Section 3, the idea behind our method and the proposed fast CIS recovery algorithm are described. Some characteristics of our method are discussed in Section 4. In Section 5, we provide extensive simulations to verify the proposed method in terms of reconstruction quality and computation speed. Finally, conclusions are given in Section 6.
Section snippets
Compressed sensing of digital images: problem statement
Inspired by the development of compressed sensing [15] and single-pixel cameras [19], [30], it is possible to sense and recover an image with as few measurements as possible if the image to be sensed is sufficiently sparse. In order to achieve the goal of compressed image sensing (CIS) for fast image reconstruction from limited samples, the critical problems described in the following subsections need to be solved.
Proposed method: compressed image sensing with low-frequency measurement sampling and fast recovery
In this section, we will present a new compressed image sensing algorithm via an elaborate design of sampling matrices and the study of different sensing strategies. We first describe the motivations behind our method in Section 3.1. Then, the proposed CIS algorithm, based on an elaborately designed sampling matrix, is presented in Section 3.2. We investigate how it can be conducted via 1D block sensing and 2D separate sensing. In Section 3.3, we discuss and compare two strategies for
Discussions
In this section, we will discuss the following issues in order to better reveal the characteristics of the proposed compressed image sensing method: A. the difference between our method and frequency/space-frequency transforms, B. the computational complexity of CS recovery, C. the mutual incoherence between the sampling matrix and dictionary, D. sparsity vs. reconstruction quality, and E. approximate recovery vs. perfect reconstruction.
Simulation results
Several simulations were conducted to verify the performance of the proposed compressed image sensing method along with different sensing strategies in terms of reconstruction quality and speed.
State-of-the-art CS algorithms [15], including orthogonal matching pursuit (OMP) [41], Lasso, TS-BCS-MCMC [24], TS-BCS-VB [25], and model-based CS (MCS) [3],1 were chosen for
Conclusions and future work
Fast and accurate compressed sensing recovery is still a challenging issue, and has received considerable attention in the literature. In this paper, we do not follow the tradition of imposing certain sparsity patterns on a compressed sensing recovery algorithm. On the contrary, we propose a novel sampling matrix for the purpose of preserving important measurements in compressed sensing of images. Under this circumstance, extremely fast compressed image sensing recovery with a closed-form
Acknowledgment
This work was supported by National Science Council, Taiwan, under grants NSC 97-2628-E-001-011-MY3 and NSC 100-2628-E-001-005-MY2.
References (49)
- et al.
Iterative hard thresholding for compressed sensing
Appl. Comput. Harmonic Anal.
(2009) - et al.
Fast compressive sensing recovery with transform-based sampling
Proceedings of the Workshop on Signal Processing with Adaptive Sparse Structured Representations, June 27–30, Edinburgh, UK
(2011) - et al.
Distributed video coding using compressive sampling
Proceedings of the Picture Coding Symposium
(2009) - et al.
Image modeling and enhancement via structured sparse model selection
Proceedings of the IEEE International Conference on Image Processing (ICIP), Hong Kong
(2010) - et al.
Bayesian compressive sensing using laplace priors
IEEE Trans. Image Process.
(2010) - et al.
Low-dimensional models for dimensionality reduction and signal recovery: a geometric perspective
Proc. IEEE
(2010) - et al.
Model-based compressive sensing
IEEE Trans. Inf. Theory
(2010) - et al.
Computing sparse representations of multidimensional signals using Kronecker bases
Neural Comput. J.
(2013) - et al.
Stable, robust and super fast reconstruction of tensors using multi-way projections
IEEE Trans. Signal Process.
(2015) - et al.
Sparsity and incoherence in compressive sampling
Inverse Probl.
(2007)
Ph. D. dissertation
Iterative algorithms for compressed sensing with partially known support
Proceedings of the IEEE ICASSP
Iterative hard thresholding for compressed sensing with partially known support
Proceedings of the IEEE ICASSP
Sparse signal recovery using Markov random fields
Proceedings of the Workshop on Neural Information Processing Systems
Recovery of clustered sparse signals from compressive measurements
Proceedings on Sampling Theory and Applications (SAMPTA)
Collaborative compressive spectrum sensing in a UAV environment
Proceedings of the Military Communications Conference (MILCOM)
Adaptive compressed image sensing based on wavelet modeling and direct sampling
Proceedings of the SAMPTA
Distributed compressed video sensing
Proceedings of the IEEE International Conference on Image Processing
Single-pixel imaging via compressive sampling
IEEE Signal Process. Mag.
Robust recovery of signals from a structured union of subspaces
IEEE Trans. Inf. Theory
Block-sparse signals: uncertainty relations and efficient recovery
IEEE Trans. Signal Process.
Block compressed sensing of natural images
Proceedings of the Conference on Digital Signal Processing (DSP)
Cited by (17)
Deterministic construction of sparse binary matrices via incremental integer optimization
2018, Information SciencesA theoretical perspective of solving phaseless compressive sensing via its nonconvex relaxation
2017, Information SciencesTensor compressed video sensing reconstruction by combination of fractional-order total variation and sparsifying transform
2017, Signal Processing: Image CommunicationCitation Excerpt :More recently, a fast low rank tensor CS (LRTCS) method was proposed in [15], which no longer assumed certain sparsity pattern and did not involve iterations, making it suitable for large-scale problems. However, it assumed that the signal to be sensed and recovered must have low multi-linear-rank, leading to redundant sensing, which meant that under the same measurement rate the reconstructed quality was remarkably lower than other CVS algorithms [14]. TV regularization model, proposed by Rudin et al. [16], plays a key role in image/video CS reconstruction, this model enforces sparsity for piecewise constant signals and has advantage of preserving edges [12,17,18].
SIFT-guided multi-resolution video inpainting with innovative scheduling mechanism and irregular patch matching
2016, Information SciencesCitation Excerpt :Video inpainting is an important technique to achieve content restoration [4,12,13,20].
Compressive-sensing recovery of images by context extraction from random samples
2024, Multimedia Tools and Applications