Development of robust neighbor embedding based super-resolution scheme
Introduction
Image analysis is an important direction of research in the field of image processing and computer vision. Image resolution plays a significant role during image analysis. The higher the resolution of an image, the more accurate is its analysis. However, during image acquisition due to some unfavorable conditions we get a low-resolution (LR) image with loss of information. Hence, achieving high-resolution (HR) image from a low-resolution image becomes a necessity. To our favor, there exists a technique called super-resolution (SR) to achieve HR images from corresponding LR ones. SR uses one or multiple LR images to produce a HR image with high spatial resolution. Due to loss of missing frequency components, there is a possibility of information loss in a SR image during the process of conversion. Hence, the major challenge in SR process is to enhance the quality of the LR image by preserving the missing high-frequency components. The primary task of SR process is to produce an image having alias free, up-sampled, and high spatial frequency from a LR image [1]. Over the past years, the SR algorithms have been extensively used in computer vision applications like remote sensing, astronomical imaging, medical imaging, and video surveillance. SR is broadly divided into two categories namely, reconstruction-based SR and recognition-based SR. Reconstruction-based SR refers to generation of HR image from a degraded LR image through traditional upscaling methods [2]. Recognition-based SR utilizes learning algorithms as it identifies pre-configured patterns hidden in LR images. Hence, recognition based SR is also known as learning based SR and it has been widely used in detection, recognition, and identification.
This paper proposes a robust learning based SR algorithm to generate a HR image from a single LR image. The scheme utilizes neighbor embedding approach and suitably named as robust neighbor embedding based super-resolution (RNESR). RNESR is trained using known LR–HR image pairs to generate the information with respect to local geometry and neighborhood. Further, it uses histogram matching to select the best LR–HR image pairs for training. Subsequently, the scheme is validated using LR images selected from training pairs as well as images not used during training to generate their corresponding HR image. The proposed RNESR scheme is simulated along with other competent schemes are also simulated, and results are compared with respect to peak signal-to-noise ratio (PSNR), structural similarity index (SSIM) [3], and feature similarity index (FSIM) [4]. It is observed that RNESR scheme outperforms others with respect to qualitative and quantitative parameters.
The rest of the paper is organized as follows. Section 2 presents the related work. The proposed RNESR algorithm is described detail in Section 3. The experimental results and discussion are presented in Section 4. Finally, Section 5 deals with the concluding remarks.
Section snippets
Related work
Existing single image SR approaches can be classified into two general classes, including frequency-domain methods and spatial-domain methods. Furthermore, the spatial domain-based approaches can be classified into three categories containing interpolation-based methods [5], [6], reconstruction-based techniques [7], [8], and learning-based approach. The proposed scheme is based on learning based scheme. An exhaustive literature on learning based approaches discussed below. Basically, the
Robust neighbor embedding based super-resolution (RNESR) algorithm
The proposed RNESR scheme generates a HR image from a given LR image. Normally, a LR image is a down-sampled degraded image of the corresponding HR image as shown in Fig. 1. The degradations are due to blur and warping (rotation and translation) in the LR image during acquisition. Mathematically, a LR image can be modeled aswhere D: Down sampling factor, B: Gaussian blur, and R: Warping operator.
It may be noted that the operations rotation, blurring, and down-sampling occurs in the
Simulation and results
To validate the efficacy of the proposed RNESR scheme, extensive simulations have been carried out on a sample image database created using standard images like Lena, Peppers, Butterfly, Baby, Flower, and face images available on public domain website [42]. The sample images are shown in Fig. 4. The face images constitute faces from different age groups and genders. Initially, all face images are standardized to 300×300 dimensions and represent as HR images. The scheme is verified on LR images
Conclusion
This paper proposes learning based SR algorithm to generate a HR image from a single LR image. The LR images with outlier and without outlier have been taken into account for HR image generation. The method uses histogram matching to select best training image pair. A global neighborhood size is selected from a set of local neighborhood size for neighbor embedding. To generate the HR patches from LR patches, it uses RLLE which in turn utilizes RPCA. The scheme is named as robust neighbor
Deepasikha Mishra is now pursuing Ph.D. in Computer Science and Engineering at National Institute of Technology, Rourkela, India. Her research interests include image processing, computer vision and pattern recognition.
References (42)
- et al.
Neighbor embedding based super-resolution algorithm through edge detection and feature selection
Pattern Recognit. Lett.
(2009) - et al.
Non-local neighbor embedding for image super-resolution through FoE features
Neurocomputing
(2014) - et al.
Single image super-resolution via subspace projection and neighbor embedding
Neurocomputing
(2014) - et al.
Nonlinear neighbor embedding for single image super-resolution via kernel mapping
Signal Process.
(2014) - et al.
Dual-sparsity regularized sparse representation for single image super-resolution
Inf. Sci.
(2015) - et al.
Image super-resolution using sparse coding over redundant dictionary based on effective image representations
J. Vis. Commun. Image Represent.
(2015) - et al.
Robust locally linear embedding
Pattern Recognit.
(2006) - et al.
Global and local choice of the number of nearest neighbors in locally linear embedding
Pattern Recognit. Lett.
(2011) - et al.
Single frame image super-resolutionshould we process locally or globally?
Multidimens. Syst. Signal Process.
(2007) - et al.
Limits on super-resolution and how to break them
IEEE Trans. Pattern Anal. Mach. Intell.
(2002)
Image quality assessmentfrom error visibility to structural similarity
IEEE Trans. Image Process.
Fsima feature similarity index for image quality assessment
IEEE Trans. Image Process.
New edge-directed interpolation
IEEE Trans. Image Process.
An edge-guided image interpolation algorithm via directional filtering and data fusion
IEEE Trans. Image Process.
Fundamental limits of reconstruction-based superresolution algorithms under local translation
IEEE Trans. Pattern Anal. Mach. Intell.
Single image super-resolution with non-local means and steering kernel regression
IEEE Trans. Image Process.
Example-based super-resolution
IEEE Comput. Graph. Appl.
Improved super-resolution through residual neighbor embedding
J. Guangxi Norm. Univ.
Cited by (0)
Deepasikha Mishra is now pursuing Ph.D. in Computer Science and Engineering at National Institute of Technology, Rourkela, India. Her research interests include image processing, computer vision and pattern recognition.
Banshidhar Majhi is a senior professor in the Department of Computer Science and Engineering at NIT Rourkela, India. He has published more than 60 papers in referred journals and guided 10 scholars so far. His research interests include image processing, data compression, cryptographic protocols, parallel computing, and biometrics.
Pankaj Kumar Sa is presently working as an assistant professor in the Department of Computer Science and Engineering at NIT Rourkela, India. He has published more than 10 papers in referred journals. His research areas include computer vision and image processing.
Ratnakar Dash is presently working as an assistant professor in the Department of Computer Science and Engineering at NIT Rourkela, India. He has published more than 7 papers in referred journals. His research areas include image processing and pattern recognition.