Skip to main content
SpringerLink
Log in

Image super-resolution by estimating the enhancement weight of self example and external missing patches

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Image super-resolution (SR) is the process of generating a high-resolution (HR) image using one or more low-resolution (LR) inputs. Many SR methods have been proposed, but generating the small-scale structure of an SR image remains a challenging task. We hence propose a single-image SR algorithm that combines the benefits of both internal and external SR methods. First, we estimate the enhancement weights of each LR-HR image patch pair. Next, we multiply each patch by the estimated enhancement weight to generate an initial SR patch. We then employ a method to recover the missing information from the high-resolution patches and create that missing information to generate a final SR image. We then employ iterative back-projection to further enhance visual quality. The method is compared qualitatively and quantitatively with several state-of-the-art methods, and the experimental results indicate that the proposed framework provides high contrast and better visual quality, particularly for non-smooth texture areas.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Aharon M, Elad M, Bruckstein A (2006) K-SVD: An algorithm for designing over-complete dictionaries for sparse representation. IEEE T Signal Process 54:4311–4322

    Article  MATH  Google Scholar 

  2. Babacan SD, Molina R, Katsaggelos AK (2011) Variational Bayesian super resolution. IEEE T Image Process 20:984–999

    Article  MathSciNet  MATH  Google Scholar 

  3. Cao F, Cai M, Tan Y, Zhao J (2016) Image super-resolution via adaptive regularization and sparse representation. IEEE T Neural Netw Learn Syst 27:1550–1561

    Article  MathSciNet  Google Scholar 

  4. Dong C, Loy C, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. CVPR:184–199

  5. Farsiu S, Robinson D, Elad M, Milanfar P (2003) Fast and robust super resolution. Proc. IEEE Int’l Conf. Image Processing. IEEE, Barcelona, pp 14–1–217

  6. Fattal R (2007) Image upsampling via imposed edge statistics. ACM T Graph 26:1–95

    Google Scholar 

  7. Freeman WT, Jones TR, Pasztor EC (2002) Example-based super-resolution. IEEE Comput Graph 22:56–65

    Article  Google Scholar 

  8. Glasner D, Bagon S, Irani M (2009) Super-resolution from a single image. IEEE ICCV:349–356

  9. Gong W, Hu L, Li J, Li W (2015) Combining sparse representation local rank constraint for single image superresolution. Inform Sci 325:1–19

    Article  MathSciNet  Google Scholar 

  10. Hou HS, Andrews HC (1978) Cubic splines for image interpolation and digital filtering. IEEE T Acoust Speech 26:508–517

    Article  MATH  Google Scholar 

  11. Keys RG (1981) Cubic convolution interpolation for digital image processing. IEEE T Acoust Speech ASSP-29:1153–1160

    Article  MathSciNet  MATH  Google Scholar 

  12. Kim K, Kwon Y (2010) Single-image super resolution using sparse regression and natural image prior. IEEE T Pattern Anal 32:1127–1133

    Article  Google Scholar 

  13. Li X, Orchard MT (2001) New edge-directed interpolation. IEEE T Image Process 10:1521–1527

    Article  Google Scholar 

  14. Lin FJ (2015) Super-resolution from learning the enhancement ratio and texture/residual dictionary. Proc. IEEE Int’l Conf. Image Processing. IEEE, Quebec City, pp 2135–2139

  15. Park SC, Park MK, Kang MG (2003) Super resolution image reconstruction: A technical overview. IEEE Signal Process Mag 20:21–36

    Article  Google Scholar 

  16. Permuter H, Francos J, Jermyn I (2006) A study of Gaussian mixture models of color and texture features for image classification and segmentation. Pattern Recog 39:695–706

    Article  MATH  Google Scholar 

  17. Protter M, Elad M, Takeda H, Milanfar P (2009) Generalizing the nonlocal-means to super-resolution reconstruction. IEEE T Image Process 18:36–51

    Article  MathSciNet  MATH  Google Scholar 

  18. Shan Q, Li Z, Jia J, Tang CK (2008) Fast image/video upsampling. ACM T Graph 27:1–5

    Google Scholar 

  19. Thanh MN, Wu QMJ, Ahuja S (2010) An extension of the standard mixture model for image segmentation. IEEE T Neural Networ 21:1326–1338

    Article  Google Scholar 

  20. Timofte R, De Smet V, Van Gool L (2013) Anchored neighborhood regression for fast example-based super resolution. ICCV. https://doi.org/10.1109/ICCV.2013.241

  21. Timofte R, De Smet V, Van Gool L (2014) A+: Adjusted anchored neighborhood regression for fast super-resolution. ACCV. https://doi.org/10.1007/978-3-319-16817-3_8

  22. Timofte R, Rothe R, Van Gool L (2016) Seven ways to improve example-based single image super resolution. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2016), June 2016, US. pp 1865–1873

  23. Yang J, Wright J, Huang T, Ma Y (2010) Image super-resolution via sparse representation. IEEE T Image Process 19:2861–2873

    Article  MathSciNet  MATH  Google Scholar 

  24. Yang J, Wang Z, Lin Z, Cohen S, Huang TS (2012) Coupled dictionary training for image superresolution. IEEE T Image Process 21:3467–3478

    Article  MATH  Google Scholar 

  25. Zeyde R, Elad M, Protter M (2012) On Single Image Scale-Up Using Sparse-Representations. In: Boissonnat J-D, Chenin P, Cohen A, Gout C, Lyche T, Mazure M-L, Schumaker L (eds) Curves and Surfaces. Lecture Notes in Computer Science, vol 6920. Springer, Berlin, Heidelberg, pp 711–730

  26. Zhang H, Zhang Y, Li H, Huang TS (2012) Generative Bayesian image super resolution with natural image prior. IEEE T Image Process 21:4054–4067

    Article  MathSciNet  MATH  Google Scholar 

  27. Zhang K, Tao D, Gao X et al (2015) Learning multiple linear mappings for efficient single image super-resolution. IEEE T Image Process 24:846–861

    Article  MathSciNet  Google Scholar 

  28. Zibetti M, Mayer J (2007) A robust and computationally efficient simultaneous superresolution scheme for image sequences. IEEE T Circ Syst Vid 17:1288–1300

    Article  Google Scholar 

  29. Zuo W, Zhang L, Song C, Zhang D, Gao H (2014) Gradient histogram estimation and preservation for texture enhanced image denoising. IEEE T Image Process 23:2459–2472

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Science, National Chiao Tung University, 1001 University Road, Hsinchu, Taiwan

    Fang-Ju Lin & Jen-Hui Chuang

Authors
  1. Fang-Ju Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jen-Hui Chuang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fang-Ju Lin.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lin, FJ., Chuang, JH. Image super-resolution by estimating the enhancement weight of self example and external missing patches. Multimed Tools Appl 77, 19071–19087 (2018). https://doi.org/10.1007/s11042-017-5350-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-5350-1

Keywords

Search

Navigation