Elsevier

Neurocomputing

Volume 202, 19 August 2016, Pages 49-66
Neurocomputing

Development of robust neighbor embedding based super-resolution scheme

https://doi.org/10.1016/j.neucom.2016.04.013Get rights and content

Abstract

In this paper, we propose a robust neighbor embedding super-resolution (RNESR) scheme to generate a super-resolution (SR) image from a single low-resolution (LR) image. It utilizes histogram matching for selection of best training pair of images. This helps to learn co-occurrence prior to high-resolution (HR) image reconstruction. The global neighborhood size is computed from local neighborhood size, which avoids the over-fitting and under-fitting problem during neighbor embedding. Robust locally linear embedding (RLLE) is used in place of locally linear embedding (LLE) to generate HR image. To validate the scheme, exhaustive simulation has been carried out on standard images. Comparative analysis with respect to different measures like peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and feature similarity index (FSIM) reveals that the RNESR scheme generates high-quality SR image from a LR image as compared to existing schemes.

Introduction

Image analysis is an important direction of research in the field of image processing and computer vision. Image resolution plays a significant role during image analysis. The higher the resolution of an image, the more accurate is its analysis. However, during image acquisition due to some unfavorable conditions we get a low-resolution (LR) image with loss of information. Hence, achieving high-resolution (HR) image from a low-resolution image becomes a necessity. To our favor, there exists a technique called super-resolution (SR) to achieve HR images from corresponding LR ones. SR uses one or multiple LR images to produce a HR image with high spatial resolution. Due to loss of missing frequency components, there is a possibility of information loss in a SR image during the process of conversion. Hence, the major challenge in SR process is to enhance the quality of the LR image by preserving the missing high-frequency components. The primary task of SR process is to produce an image having alias free, up-sampled, and high spatial frequency from a LR image [1]. Over the past years, the SR algorithms have been extensively used in computer vision applications like remote sensing, astronomical imaging, medical imaging, and video surveillance. SR is broadly divided into two categories namely, reconstruction-based SR and recognition-based SR. Reconstruction-based SR refers to generation of HR image from a degraded LR image through traditional upscaling methods [2]. Recognition-based SR utilizes learning algorithms as it identifies pre-configured patterns hidden in LR images. Hence, recognition based SR is also known as learning based SR and it has been widely used in detection, recognition, and identification.

This paper proposes a robust learning based SR algorithm to generate a HR image from a single LR image. The scheme utilizes neighbor embedding approach and suitably named as robust neighbor embedding based super-resolution (RNESR). RNESR is trained using known LR–HR image pairs to generate the information with respect to local geometry and neighborhood. Further, it uses histogram matching to select the best LR–HR image pairs for training. Subsequently, the scheme is validated using LR images selected from training pairs as well as images not used during training to generate their corresponding HR image. The proposed RNESR scheme is simulated along with other competent schemes are also simulated, and results are compared with respect to peak signal-to-noise ratio (PSNR), structural similarity index (SSIM) [3], and feature similarity index (FSIM) [4]. It is observed that RNESR scheme outperforms others with respect to qualitative and quantitative parameters.

The rest of the paper is organized as follows. Section 2 presents the related work. The proposed RNESR algorithm is described detail in Section 3. The experimental results and discussion are presented in Section 4. Finally, Section 5 deals with the concluding remarks.

Section snippets

Related work

Existing single image SR approaches can be classified into two general classes, including frequency-domain methods and spatial-domain methods. Furthermore, the spatial domain-based approaches can be classified into three categories containing interpolation-based methods [5], [6], reconstruction-based techniques [7], [8], and learning-based approach. The proposed scheme is based on learning based scheme. An exhaustive literature on learning based approaches discussed below. Basically, the

Robust neighbor embedding based super-resolution (RNESR) algorithm

The proposed RNESR scheme generates a HR image from a given LR image. Normally, a LR image is a down-sampled degraded image of the corresponding HR image as shown in Fig. 1. The degradations are due to blur and warping (rotation and translation) in the LR image during acquisition. Mathematically, a LR image can be modeled asLR=DBR(HR),where D: Down sampling factor, B: Gaussian blur, and R: Warping operator.

It may be noted that the operations rotation, blurring, and down-sampling occurs in the

Simulation and results

To validate the efficacy of the proposed RNESR scheme, extensive simulations have been carried out on a sample image database created using standard images like Lena, Peppers, Butterfly, Baby, Flower, and face images available on public domain website [42]. The sample images are shown in Fig. 4. The face images constitute faces from different age groups and genders. Initially, all face images are standardized to 300×300 dimensions and represent as HR images. The scheme is verified on LR images

Conclusion

This paper proposes learning based SR algorithm to generate a HR image from a single LR image. The LR images with outlier and without outlier have been taken into account for HR image generation. The method uses histogram matching to select best training image pair. A global neighborhood size is selected from a set of local neighborhood size for neighbor embedding. To generate the HR patches from LR patches, it uses RLLE which in turn utilizes RPCA. The scheme is named as robust neighbor

Deepasikha Mishra is now pursuing Ph.D. in Computer Science and Engineering at National Institute of Technology, Rourkela, India. Her research interests include image processing, computer vision and pattern recognition.

References (42)

  • Z. Wang et al.

    Image quality assessmentfrom error visibility to structural similarity

    IEEE Trans. Image Process.

    (2004)
  • L. Zhang et al.

    Fsima feature similarity index for image quality assessment

    IEEE Trans. Image Process.

    (2011)
  • X. Li et al.

    New edge-directed interpolation

    IEEE Trans. Image Process.

    (2001)
  • L. Zhang et al.

    An edge-guided image interpolation algorithm via directional filtering and data fusion

    IEEE Trans. Image Process.

    (2006)
  • Z. Lin et al.

    Fundamental limits of reconstruction-based superresolution algorithms under local translation

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2004)
  • K. Zhang et al.

    Single image super-resolution with non-local means and steering kernel regression

    IEEE Trans. Image Process.

    (2012)
  • W.T. Freeman et al.

    Example-based super-resolution

    IEEE Comput. Graph. Appl.

    (2002)
  • H. Chang, D.-Y. Yeung, Y. Xiong, Super-resolution through neighbor embedding, in: IEEE Computer Society Conference on...
  • T.-M. Chan et al.

    Improved super-resolution through residual neighbor embedding

    J. Guangxi Norm. Univ.

    (2006)
  • M. Cao, Z. Gan, X. Zhu, Super-resolution algorithm through neighbor embedding with new feature selection and example...
  • W. Fan, D.-Y. Yeung, Image hallucination using neighbor embedding over visual primitive manifolds, in: IEEE Conference...
  • Cited by (0)

    Deepasikha Mishra is now pursuing Ph.D. in Computer Science and Engineering at National Institute of Technology, Rourkela, India. Her research interests include image processing, computer vision and pattern recognition.

    Banshidhar Majhi is a senior professor in the Department of Computer Science and Engineering at NIT Rourkela, India. He has published more than 60 papers in referred journals and guided 10 scholars so far. His research interests include image processing, data compression, cryptographic protocols, parallel computing, and biometrics.

    Pankaj Kumar Sa is presently working as an assistant professor in the Department of Computer Science and Engineering at NIT Rourkela, India. He has published more than 10 papers in referred journals. His research areas include computer vision and image processing.

    Ratnakar Dash is presently working as an assistant professor in the Department of Computer Science and Engineering at NIT Rourkela, India. He has published more than 7 papers in referred journals. His research areas include image processing and pattern recognition.

    View full text