Single-image super-resolution reconstruction based on global non-zero gradient penalty and non-local Laplacian sparse coding

Highlights

• Global non-zero gradient penalty is developed to reconstruct the edge component.

• Non-local Laplacian sparse coding is developed to reconstruct the texture component.

• Global and local function is developed to further improve the imageʼs quality.

Abstract

Methods based on sparse coding have been successfully used in single-image super-resolution reconstruction. However, they tend to reconstruct incorrectly the edge structure and lose the difference among the image patches to be reconstructed. To overcome these problems, we propose a new approach based on global non-zero gradient penalty and non-local Laplacian sparse coding. Firstly, we assume that the high resolution image consists of two components: the edge component and the texture component. Secondly, we develop the global non-zero gradient penalty to reconstruct correctly the edge component and the non-local Laplacian sparse coding to preserve the difference among texture component patches to be reconstructed respectively. Finally, we develop a global and local optimization on the initial image, which is composed of the reconstructed edge component and texture component, to remove possible artifacts. Experimental results demonstrate that the proposed approach can achieve more competitive single-image super-resolution quality compared with other state-of-the-art methods.

Introduction

High resolution (HR) images are desired in most electronic imaging applications such as biometrics identification, medical imaging, military surveillance and so on. Unfortunately, due to the physical limitation of relevant imaging devices, the images we observed are usually noisy, blurred and downsampled. To obtain the HR image, we can either reduce the pixel size by sensor manufacturing techniques or increase the chip size of charge-coupled device sensors, which are both severely limited in increasing the cost of digital imaging systems and reducing the processing efficiency of real-time environment [1]. Therefore, the signal processing methods are selected to reconstruct potential details and features hidden in the low resolution (LR) image.

Generally, the existing methods can be classified into three categories: interpolation-based methods [2], [3], [4], [5], regularization-based methods [6], [7], [8], [9], [10], [11], [12] and example-based methods [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32]. However, the interpolation-based methods are usually prone to yield overly smooth images with ringing and jagged artifacts when a larger magnification ratio (such as a factor of more than double) is performed. The regularization-based methods are limited in modeling the visual complexity of the real images and selecting correct regularization parameters. The focus of this paper is the example-based methods because the methods are of stronger capability of image super-resolution reconstruction as the magnification becomes larger.

In recent years, the example-based methods have been explored. This kind of methods presumes that the high-frequency details lost in the LR image can be predicted by learning the co-occurrence relationship between LR training patches and their corresponding HR patches. Freeman et al. [13] first proposed a relation model between the local regions of images and scenes by using the Markov network. However, this approach depends heavily on a large training data set. Chang et al. [14] introduced locally linear embedding from manifold learning to process the image super resolution task. Although this method has advantages over Freemanʼs, the problems of the number of neighbor and feature representation of LR and HR image patches remain unsettled. Others based on learning primal sketch prior are proposed [15], [16], [17]. However, due to the lack of priors of textures and details, they are weak in hallucinating both textures and details. Recently, Yang et al. [20] proposed a sparse coding to reconstruct HR images. In their works, HR image patches are sparsely coded under over-complete dictionary learned with coupled pattern. Considering that there are different types of image patches (such as smooth regions, texture regions and edge regions) in image, Jing et al. [21] proposed a multi-space sparse representation method, which first decomposes image into structural and textural components, and then the HR image is recovered by coding the structural and textural components respectively. And Yang et al. [22] also proposed a multiple-geometric-dictionaries-based clustered sparse coding scheme, which first trains the geometric dictionaries of geometric clusters, and then HR image patches are sparsely coded under different geometric dictionaries. Recently, the geometric structure information of image patches has been successfully used in various image processing applications [33], [34], [35]. Some research works have pointed out that the reconstruction quality greatly depends on geometrical structures of the data [31]. Hence, it is important to explore these potential geometrical structures to enhance existing sparse coding stability. By transferring the non-local information of images patches into the sparse coefficients, the non-local sparse coding methods [30], [31], [32] are widely proposed for image reconstruction. However, the methods lose the difference among the image patches. Moreover, they are not effective in reconstructing images which contain the patterns with strong edge and reconstruct incorrectly the edge structures (such as continuity and orientation) [23].

To resolve the above problem, we propose a new approach based on global non-zero gradient penalty and non-local Laplacian sparse coding. The overall framework of proposed approach is illustrated in Fig. 1. As shown in Fig. 1, firstly, by exploring the global non-zero gradient penalty (GGP) which can globally sharpen major edges and preserve their geometric structure by increasing the steepness of transition in a sparsity-control manner, the HR edge component can be reconstructed. Meanwhile, by exploring the non-local Laplacian sparse coding (NLSC) which can preserve the difference by exploring the one-to-one relationship of the image patches and the non-local prior simultaneously, the HR texture component can be reconstructed. Then, the global and local optimization (GLO) is applied on the initial image for removing the possible artifacts and making the final image more natural. Figs. 2(c)–(d) show the edge component and the texture component of “Butterfly” image. Fig. 3 shows the decomposition process. Due to the different image components reconstructed by different methods, it makes the obtaining of desired HR image possible. The performance of the approach is tested by various typical experiments in terms of visual evaluation, peak signal-to-noise ratio (PSNR) and structural similarity (SSIM). Compared with the related single image super resolution (SISR) approaches, the proposed approach has the following characteristics:

• GGP is proposed for reconstructing edge component.

• NLSC is proposed for reconstructing texture component.

• GLO is applied on the initial HR image to further improve the imageʼs quality.

The rest of the paper is organized as follows. In Section 2, we present our SISR approach in detail. The experimental results together with relevant discussions are given in Section 3. Finally, conclusions are discussed in Section 4. In addition, the descriptions of the acronyms used in this paper are listed in Table 1.