Skip to main content
Springer Nature Link
Log in

Single image super-resolution via self-similarity and low-rank matrix recovery

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

We propose a novel single-image super resolution (SISR) approach using self-similarity of image and the low-rank matrix recovery (LRMR). The method performs multiple upsampling steps with relatively small magnification factors to recover a desired high resolution image. Each upsampling process includes the following steps: First, a set of low/high resolution (LR/HR) patch pairs is generated from the pyramid of the input low resolution image. Next, for each patch of the unknown HR images, similar HR patches are found from the set of LR/HR patch pairs by the corresponding LR patch and are stacked into a matrix with approximately low rank. Then, the LRMR technique is exploited to estimate the unknown HR image patch. Finally, the back-projection technique is used to perform the global reconstruction. We tested the proposed method on fifteen images including humans, animals, plants, text, and medical images. Experimental results demonstrate the effectiveness of the proposed method compared with several representative methods for SISR in terms of quantitative metrics and visual effect.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Arya S, Mount DM (1993) Approximate nearest neighbor queries in fixed dimensions. In: Proceedings of the fourth annual ACM-SIAM symposium on discrete algorithms - SODA '93. Austin, Texas, USA, 25–27 January 1993, pp 271–280

  2. Bagon S (2009) Matlab class for ANN. http://www.wisdom.weizmann.ac.il/~bagon/matlab.html

  3. Bevilacqua M, Roumy A, Guillemot C, Alberi-Morel ML (2012) Low complexity single image super-resolution based on nonnegative neighbor embedding. In: Proceedings British Machine Vision Conference 135:1–10

    MATH  Google Scholar 

  4. Chang H, Yeung D-Y, Xiong Y (2004) Super-resolution through neighbor embedding. IEEE Comput Soc Conf Comput Vis Pattern Recognit 1:275–282

    Google Scholar 

  5. Chen X, Qi C (2014) Low-rank neighbor embedding for single image super resolution. IEEE Signal Processing Letters. 21(1):79–82

    Article  Google Scholar 

  6. Dabov K, Foi A, Katkovnik V, Egiazarian K (2007) Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans. Image Process 16(8):2080–2095

    Article  MathSciNet  Google Scholar 

  7. Ding C, Zhou D, He X, Zha H (2006) R1-PCA: rotational invariant l1 -norm principal component analysis for robust subspace. In: Proceedings of the 23rd international conference on Machine learning - ICML '06. Pittsburgh, Pennsylvania, USA, 25–29 June 2006, pp 281–288

  8. Dong W, Shi G, Li X (2013) Nonlocal image restoration with bilateral variance estimation: a low-rank approach. IEEE Trans Image Process 22(2):700–711

    Article  MathSciNet  MATH  Google Scholar 

  9. Dong C, Loy CC, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. In: Proceedings of the 13th European Conference on Computer Vision - ECCV 2014. Lecture Notes in Computer Science, vol 8692. Zurich, Switzerland, 6–12 September 2014, pp 184–199

  10. Fan N (2009) Wavelet-based compressive super-resolution. In: 2009 Workshop on Applications of Computer Vision -WACV 2009. Snowbird, UT, USA, 7–8 December 2009, pp 1–6

  11. Freedman G, Fattal R (2011) Image and video upscaling from local self-examples. ACM Trans Graph 30(2):1–11

    Article  Google Scholar 

  12. Freeman WT, Pasztor EC (1999) Learning to estimate scenes from images. In: Kearns MS, Solla SA, Cohn DA (eds) Adv. Neural Information Processing Systems, vol. 11, MIT Press, Cambridge, pp 775–781

  13. Freeman WT, Jones TR, Pasztor EC (2002) Example learning-based super-resolution. IEEE Comput Graphic Application 22(02):56–65

    Article  Google Scholar 

  14. Glasner D, Bagon S, Irani M (2009) Super-resolution from a single image. IEEE Int Conf Comput Vision 30(2):349–356

    Google Scholar 

  15. Keys RG (1981) Cubic convolution interpolation for digital image processing. IEEE Trans Acoust Speech Signal Process 29(6):1153–1160

    Article  MathSciNet  MATH  Google Scholar 

  16. Lin Z, Shum H (2004) Fundamental limits of reconstruction-based super-resolution algorithms under local translation. IEEE Transac Pattern Anal Mach Intell 26(01):83–97

    Article  Google Scholar 

  17. Lin Z, Chen M, Wu L, Ma Y (2009) The augmented Lagrange multiplier method for exact recovery of corrupted low-rank matrices. UIUC technical report UILU-ENG-09-2215, November

  18. Liu D, Wang Z, Wen B, Yang J, Han W, Huang T (2016) Robust single image super-resolution via deep networks with sparse prior. IEEE Transact Image Process. 25(7):1–14

    Article  MathSciNet  Google Scholar 

  19. Mairal J, Bach F, Ponce J, Sapiro G, Zisserman A (2009) Non-local sparse models for image restoration. IEEE international conference computer vision. Pp 2272–2279

  20. Mount DM, Arya S (1998) ANN: a library for approximate nearest neighbor searching. In: Proceedings of IEEE CGC Workshop on Computational Geometry, Providence, RI, USA 1998, pp 33–40

  21. Nie F, Huang H, Cai X, Ding CH (2010) Efficient and robust feature selection via joint l2,1-norms minimization. In: Proceedings of the 23rd International Conference on Neural Information Processing Systems - NIPS’10. Vancouver, British Columbia, Canada, 6–9 December 2010, pp 1813–1821

  22. Park S, Park M, Kang MG (2003) Super-resolution image reconstruction: a technical overview. IEEE Signal Process Mach 20(03):21–36

    Article  Google Scholar 

  23. Ren CX, Dai DQ, Yan H (2012) Robust classification using l21-norm based regression model. Pattern Recogn 45(7):2708–2718

    Article  MATH  Google Scholar 

  24. Romano Y, Isidoro JR, Milanfar P (2017) RAISR: rapid and accurate image super resolution. IEEE Transac Computat Imaging 3(1):110–125

    Article  MathSciNet  Google Scholar 

  25. Sen P, Darabi S (2009) Compressive image super-resolution. In: Proceedings of the 43rd Asilomar Conference on Signals, Systems and Computers -Asilomar'09. Pacific Grove, CA, USA, 1–4 November 2009, pp 1235–1242

  26. Timofte R, De Smet V, Van Gool L (2013) Anchored neighborhood regression for fast example-based super-resolution. In: Proceedings of the 2013 IEEE International Conference on Computer Vision - ICCV 2013. Sydney, Australia, 1–8 December 2013, pp 1920–1927

  27. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Transact Image Process 13(4):600–612

    Article  Google Scholar 

  28. Wright J, Ganesh A, Rao S, Ma Y (2009) Robust principal component analysis: exact recovery of corrupted low-rank matrices via convex optimization. Adv Neural Inf Proces Syst 87(4):1–44

    Google Scholar 

  29. Yang MC, Wang YCF (2013) A self-learning approach to single image super-resolution. IEEE Trans Multimedia 15(3):498–508

    Article  MathSciNet  Google Scholar 

  30. Yang CY, Yang MH (2013) Fast direct super-resolution by simple functions. In: Proceedings of the 2013 IEEE International Conference on Computer Vision -ICCV2013, vol 00. Sydney, NSW, Australia, 1–8 December 2013, pp 561–568

  31. Yang J, Wright J, Huang T, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873

    Article  MathSciNet  MATH  Google Scholar 

  32. Yang CY, Huang JB, Yang MH (2010) Exploiting self-similarities for single frame super-resolution. In: Kimmel R, Klette R, Sugimoto A (eds) Computer Vision. Proceedings of the 10th Asian Conference on Computer Vision –ACCV 2010. Lecture Notes in Computer Science, vol 6494. Springer. Queenstown, New Zealand, 8–12 November 2010, pp 497–510

  33. Yang CY, Ma C, Yang MH (2014) Single-image super-resolution: a benchmark. In: Fleet D, Pajdla T, Schiele B, Tuytelaars T (eds) Computer Vision. Proceedings of the 13th European Conference on Computer Vision - ECCV 2014. Lecture Notes in Computer Science, vol 8692. Zurich, Switzerland, 6–12 September 2014, pp 372–386

  34. Zeyde R, Elad M, Protter M (2010) On single image scale-up using sparse representations. In: Boissonnat JD et al (eds) Curves and Surfaces. Proceedings of the 7th international conference on Curves and Surfaces. Lecture Notes in Computer Science, vol 6920. Avignon, France, 24–30 June, 2010, pp 711–730

  35. Zhang K, Gao X, Tao D, Li X (2012) Multiscale dictionary for single image super-resolution. In: Proceedings of the 2012 IEEE conference on Computer Vision and Pattern Recognition -CVPR '12, vol 00. Providence, RI USA, 16–21 June 2012, pp 1114–1121

  36. Zhang K, Tao D, Gao X, Li X, Xiong X (2015) Learning multiple linear mappings for efficient single image super-resolution. IEEE Trans Image Process 24(3):846–861

    Article  MathSciNet  Google Scholar 

  37. Zhou F, Yuan T, Yang W, Liao Q (2015) Single image super-resolution based on compact KPCA coding and kernel regression. IEEE Signal Process Lett 22(3):336–340

    Article  Google Scholar 

  38. Zontak M, Irani M (2011) Internal statistics of a single natural image. IEEE Conf Comput Vision Pattern Recog 42(7):977–984

    Google Scholar 

Download references

Acknowledgments

The authors would like to thank the anonymous reviewers for their helpful suggestions that have led to great improvement on this paper. This work was supported by the National Science Foundation of China under Grant No. 61271374 and China Scholarship Council.

Author information

Authors and Affiliations

  1. School of Mathematics, Tianjin University, Tianjin, 300072, China

    Hong Wang

  2. Beijing Key Laboratory of Intelligent Information Technology, School of Computer Science and Technology, Beijing Institute of Technology, Beijing, 100081, China

    Jianwu Li

  3. Department of Psychiatry, Columbia University, NY10032, New York, USA

    Zhengchao Dong

  4. New York State Psychiatric Institute, New York, NY, 10032, USA

    Zhengchao Dong

Authors
  1. Hong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jianwu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhengchao Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jianwu Li.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, H., Li, J. & Dong, Z. Single image super-resolution via self-similarity and low-rank matrix recovery. Multimed Tools Appl 77, 15181–15199 (2018). https://doi.org/10.1007/s11042-017-5098-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-5098-7

Keywords