Keywords

1 Introduction

In block compressive sensing (BCS) [5], an image \(\mathbf {X}\) is divided into a certain number of image blocks with size \(B \times B\). For the ith block,

$$\begin{aligned} \mathbf {y}_{i}=\varvec{\varPhi }_{B} \mathbf {x}_{i} + \mathbf {n}_{i} \end{aligned}$$
(1)

where \(\varvec{\varPhi }_{B} \in R^{M_{B}\times B^{2}}\) represents the sensing matrix, \(\mathbf {y}_{i} \in R^{M_{B}}\) is the observed vector for the ith block, \(\mathbf {x}_{i} \in R^{B^{2}}\) denotes the ith vectorized block and \(\mathbf {n}_{i}\) denotes the additive noise.

Given \(\varvec{\varPhi }_{B}\), designing efficient approaches to recover \(\mathbf {x}_{i}\) from \(\mathbf {y}_{i}\) is very critical. It is well-known that most of existing BCS are based on the iterative approaches [2,3,4, 6]. To further improve the quality of the reconstructed image, more comprehensive models, such as the bilevel optimization model [10], multi-objective optimization models [11] and sparsity-enhanced models [2, 6] are proposed. Although proper modeling could contribute a bit in achieving very competitive results, these models are usually not easy to solve, which requires different iterative approaches, e.g. iterative thresholding shrinkage methods, gradient-based methods or some evolutionary algorithms. To obtain the satisfactory results, it is often the case that more number of iterations are preferred, increasing the computational complexity. Moreover, the initial solution is also very important, which is currently obtained either by random initialization or by the least squares (LS) method. When the sampling rate (SR) is small (less than 20%), which makes the problem seriously ill-posed, these methods may not guide the iterative algorithm to converge to a high quality solution fast enough.

In this paper, we propose an efficient method which takes both the advantages of deep learning framework and the iterative approaches. First, a deep multi-layer perceptron (DMLP) is trained to find the mapping between BCS measurements and the original imagery blocks. Then, an efficient iterative approach is applied on the whole image to further reduce the block artifacts between the adjacent blocks. The proposed method is compared with several state-of-the-art competitors on the benchmark database and demonstrates its efficacy. The rest of our paper is organized as follows. Section 2 introduces our proposed BCS reconstruction approach, the experimental studies are presented in Sect. 3, and the conclusion is finally made in Sect. 4.

2 The Proposed Method

2.1 Deep MLP

Following the suggestions in [1], the structure of deep MLP (DMLP) is shown in Fig. 1, which consists of give layers.

Fig. 1.
figure 1

Structure of the proposed DMLP

Full size image
Fig. 2.
figure 2

Flowchart of our proposed approach

Full size image

For the input layer, the number of neurons is set equal to the length of vectorized image block, \(B^{2}\). The connection weights between the input layer and the sensing layer construct the sensing matrix, where the signal \(\mathbf {x}_{i} \in R^{B^{2}}\) is mapped to \(\mathbf {y}_{i} \in R^{M_{B}}\) (\(M_{B} \ll B^{2}\)). There are two reconstruction layers that perform recovery of \(\mathbf {x}_{i}\) from \(\mathbf {y}_{i}\) and each followed by an activation function in terms of rectified linear units (ReLU) [7]. The length of the reconstruction layer is usually determined by \(B^{2} \times T\), where T denotes the redundancy parameter for reconstruction. The output layer has the same size with input layer, which denotes the reconstructed signal, \(\hat{\mathbf {x}}_{i}\). In overall, the mathematical expression of the DMLP can be written as:

$$\begin{aligned} f(\mathbf {x}) = \mathbf {W}_{4} \cdot ReLU\left( \mathbf {W}_{3} \cdot ReLU \left( \mathbf {W}_{2} \left( \mathbf {W}_{1}\mathbf {x}+\mathbf {b}_{1} \right) + \mathbf {b}_{2} \right) + \mathbf {b}_{3} \right) + \mathbf {b}_{4} \end{aligned}$$
(2)

where \(\cdot \) denotes the product between matrix and vector and \(ReLU(\mathbf {x})=max(\mathbf {0},\mathbf {x})\) is the activation function based on an element-wise comparison.

Table 1. Statistical results of PSNR and SSIM for BCS considering the overlap
Full size table

2.2 Smoothed Projected Landweber Algorithm

Landweber iteration algorithm is a classical approach to solve the ill-posed linear inverse problems. The projected version converts the original signal into a sparse or compressive domain where the sparsity could be enhanced, so it could be intuitively applied in CS image reconstruction. Considering removing the block artifacts, the procedure of BCS combined with smoothed projected Landweber (SPL) algorithm is given in Algorithm 1, where \(\varvec{\varPsi }\) denotes the sparse basis Dual-tree Discrete Wavelet Transform (DDWT) [6] applied in our method.

figure a

2.3 The Flowchart of Our Approach

As presented in Fig. 2, the proposed method is composed of two stages: at first, we establish an end-to-end deep neural network based on MLP and obtain the reconstructed image block by block based on inference. In the second stage, the reconstructed image in the first stage is used as the initial solution, \({\mathbf {x}}^{(0)}\), the SPL iteration method is applied to get the ultimately recovered image.

3 Experimental Studies

3.1 Parameter Settings

In our experiment, we randomly choose 5,000,000 image blocks from 50,000 images in LabelME database [8]. The learning rate is set to 0.005, the batch size is 16 with a block size \(16\times 16\) and the redundancy parameter R is equal to 8. All the training images for DMLP are converted into grayscale images.

3.2 Numerical and Visual Results

In Table 1, the peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) [9] for 10 test images when SR is equal to 0.1, 0.125, 0.15 and 0.175 are given and each image is tested for 20 times. For simplicity, we fixed the overlap equal to 8 in all the simulations. It is demonstrated that our proposed method could achieve significantly higher PSNR and SSIM. In comparison to the DL-based method, the proposed approach could have a maximum 0.9dB gain in PSNR and the improvement is more significant when the SR is smaller. Our method utilizes the powerful regression ability of DNN in the first stage and have a very good estimated initial solution for the iterative process in the second stage especially under a very low SR.

Table 2. Average computational time for different BCS methods
Full size table

3.3 Computational Time

To evaluate the efficiency of our proposed method, the computational time for different methods with and without overlap between the adjacent blocks is recorded in Table 2. BCS-DNN only needs the fast inference, which is able to be completed in around 3 s. In our method, the initial solution obtained by DNN is usually near-optimal, which largely reduces the iterative times. So, its computational time is comparative to BCS-DNN and could achieve a higher reconstruction quality.

4 Conclusion

In this paper, we proposed an efficient CS recover method. At first, a DMLP is built where the recovered image block is obtained based on the inference from the learned regression parameters and its input measurements. Then, SPL is applied on the whole image to further reduce the block artifacts and improve the pixel consistence and smoothness between the adjacent blocks. Experimental results demonstrate that our method could not only achieve competitive numerical results but also outperform the compared methods in visual quality.