Skip to main content
SpringerLink
Log in

Single image super-resolution via low-rank tensor representation and hierarchical dictionary learning

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Super-resolution (SR) has been widely studied due to its importance in real applications and scenarios. In this paper, we focus on generating an SR image from a single low-resolution (LR) input image by employing the multi-resolution structures of an input image. By taking the LR image and its downsampled resolution (DR) and upsampled resolution (UR) versions as inputs, we propose a hierarchical dictionary learning approach to learn the latent UR-LR dictionary pair by preserving the internal structure coherence with the LR-DR dictionary pair. Note that an imposed restriction involved in this process is that the pairwise resolution images are jointly trained to obtain more compact patterns of image patches. In particular, to better explore the underlying structures of tensor data spanned by image patches, we propose a low-rank tensor approximation (LRTA) algorithm based on nuclear-norm regularization to embed input image patches into a low-dimensional space. Experimental results from publicly used images show that our proposed method achieves performance comparable with that of other state-of-the-art SR algorithms, even without using any external training databases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Aharon M, Elad M, Bruckstein A (2006) k-svd: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans Signal Process 54 (11):4311–4322

    Article  Google Scholar 

  2. Bevilacqua M, Roumy A, Guillemot C, Alberi-Morel ML (2012) Low-complexity single-image super-resolution based on nonnegative neighbor embedding 1–9

  3. Cai J-F, Candès EJ, Shen Z (2010) A singular value thresholding algorithm for matrix completion. SIAM J Optim 20(4):1956–1982

    Article  MathSciNet  Google Scholar 

  4. Chang H, Yeung D-Y, Xiong Y (2004) Super-resolution through neighbor embedding. In: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, vol 1, pp 275–282

  5. Chen X, Qi C (2014) Low-rank neighbor embedding for single image super-resolution. IEEE Signal Process Lett 21(1):79–82

    Article  Google Scholar 

  6. Cheng Z, Shen J (2016) On very large scale test collection for landmark image search benchmarking. Signal Process 124:13–26

    Article  Google Scholar 

  7. Cheng Z, Chang X, Zhu L, Kanjirathinkal RC, Kankanhalli M (2019) Mmalfm: explainable recommendation by leveraging reviews and images. ACM Trans Inf Syst 37(2):16

    Article  Google Scholar 

  8. Dong C, Loy CC, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. In: Proceedings of European Conference on Computer Vision, pp 184–199

  9. Dong W, Li G, Shi G, Li X, Ma Y (2015) Low-rank tensor approximation with laplacian scale mixture modeling for multiframe image denoising. In: Proceedings of IEEE International Conference on Computer Vision, pp 442–449

  10. Gao X, Zhang K, Tao D, Li X (2012) Image super-resolution with sparse neighbor embedding. IEEE Trans Image Process 21(7):3194–3205

    Article  MathSciNet  Google Scholar 

  11. Glasner D, Bagon S, Irani M (2009) Super-resolution from a single image. In: Proceedings of IEEE International Conference on Computer Vision, pp 349–356

  12. Greenspan H (2009) Super-resolution in medical imaging. The Comput J 52 (1):43–63

    Article  Google Scholar 

  13. Huang J-B, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp 5197–5206

  14. Irani M, Peleg S (1991) Improving resolution by image registration. CVGIP: Graph Model Image Process 53(3):231–239

    Google Scholar 

  15. Jia C, Zhong G, Fu YR (2014) Low-rank tensor learning with discriminant analysis for action classification and image recovery. In: AAAI, pp 1228–1234

  16. Jia K, Wang X, Tang X (2013) Image transformation based on learning dictionaries across image spaces. IEEE Trans Pattern Anal Mach Intell 35(2):367–380

    Article  Google Scholar 

  17. Jing X-Y, Zhu X, Wu F, You X, Liu Q, Yue D, Hu R, Xu B (2015) Super-resolution person re-identification with semi-coupled low-rank discriminant dictionary learning. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp 695–704

  18. Hardie RC, Barnard KJ, Armstrong EE (1997) Joint map registration and high resolution image estimation using a sequence of undersampled images. IEEE Trans Image Process 6:12

    Article  Google Scholar 

  19. Nguyen N, Milanfar P, Golub G (2001) A computationally efficient superresolution image reconstruction algorithm. IEEE Trans Image Process 10 (4):573–583

    Article  Google Scholar 

  20. Sun J, Zhu J, Tappen MF (2010) Context-constrained hallucination for image super-resolution. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp 231–238

  21. Kawakami R, Matsushita Y, Wright J, Ben-Ezra M, Tai Y-W, Ikeuchi K (2011) High-resolution hyperspectral imaging via matrix factorization. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp 2329–2336

  22. Kim KI, Kwon Y (2010) Single-image super-resolution using sparse regression and natural image prior. IEEE Trans Pattern Anal Mach Intell 32(6):1127–1133

    Article  Google Scholar 

  23. Kolda TG, Bader BW (2009) Tensor decompositions and applications. SIAM review 51(3):455–500

    Article  MathSciNet  Google Scholar 

  24. Liu J, Musialski P, Wonka P, Ye J (2013) Tensor completion for estimating missing values in visual data. IEEE Trans Pattern Anal Mach Intell 35(1):208–220

    Article  Google Scholar 

  25. Liu X, Song M, Tao D, Zhou X, Chen C, Bu J (2014) Semi-supervised coupled dictionary learning for person re-identification. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp 3550–3557

  26. Nie L, Song X, Chua T-S (2016) Learning from multiple social networks. Synthesis Lectures on Inform Concepts Retrieval Services 8(2):1–118

    Article  Google Scholar 

  27. Liu M, Nie L, Wang M, Chen B (2017) Towards micro-video understanding by joint sequential-sparse modeling. In: Proceedings of ACM Conference on Multimedia, pp 970–978

  28. Lu C, Feng J, Chen Y, Liu W, Lin Z, Yan S (2016) Tensor robust principal component analysis: exact recovery of corrupted low-rank tensors via convex optimization. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition

  29. Marcia RF, Willett RM (2008) Compressive coded aperture superresolution image reconstruction. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, pp 833–836

  30. Qi N, Shi Y, Sun X, Yin B (2016) Tensr: Multi-dimensional tensor sparse representation. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp 5916–5925

  31. Shi F, Cheng J, Wang L, Yap P-T, Shen D (2015) Lrtv: Mr image super-resolution with low-rank and total variation regularizations. IEEE Trans Med Imag 34(12):2459–2466

    Article  Google Scholar 

  32. Singh A, Porikli F, Ahuja N (2014) Super-resolving noisy images. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp 2846–2853

  33. Timofte R, De Smet V, Van Gool L (2013) Anchored neighborhood regression for fast example-based super-resolution. In: Proceedings of IEEE International Conference on Computer Vision, pp 1920–1927

  34. Veganzones MA, Simoes M, Licciardi G, Yokoya N, Bioucas-Dias JM, Chanussot J (2016) Hyperspectral super-resolution of locally low rank images from complementary multisource data. IEEE Trans Image Process 25(1):274–288

    Article  MathSciNet  Google Scholar 

  35. Wang H, Nie F, Huang H (2014) Low-rank tensor completion with spatio-temporal consistency. In: Proceedings of AAAI International Conference, pp 2846–2852

  36. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13 (4):600–612

    Article  Google Scholar 

  37. Wang Z, Liu D, Yang J, Han W, Huang T (2015) Deep networks for image super-resolution with sparse prior. In: Proceedings of IEEE International Conference on Computer Vision, pp 370–378

  38. Yang C-Y, Ma C, Yang M-H (2014) Single-image super-resolution: a benchmark. In: Proceedings of European Conference on Computer Vision, pp 372–386

  39. Yang J, Lin Z, Cohen S (2013) Fast image super-resolution based on in-place example regression. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp 1059–1066

  40. Yang J, Wright J, Huang T, Ma Y (2008) Image super-resolution as sparse representation of raw image patches. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp 1–8

  41. Yang J, Wright J, Huang TS, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873

    Article  MathSciNet  Google Scholar 

  42. Zeyde R, Elad M, Protter M (2010) On single image scale-up using sparse-representations. In: Proceedings of International conference on curves and surfaces, pp 711–730

  43. Grasedyck L, Kressner D, Tobler C (2013) A literature survey of low-rank tensor approximation techniques. GAMM-Mitteilungen 36(1):53–78

    Article  MathSciNet  Google Scholar 

  44. De Lathauwer L, De Moor B, Vandewalle J (2000) A multilinear singular value decomposition. SIAM J Matrix Anal Appl 21(4):1253–1278

    Article  MathSciNet  Google Scholar 

  45. Zhang X, Yuan X, Carin L (2018) Nonlocal low-rank tensor factor analysis for image restoration. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp 8232–8241

  46. Tucker LR (1966) Some mathematical notes on three-mode factor analysis. Psychometrika 31(3):279–311

    Article  MathSciNet  Google Scholar 

  47. Liu J, Musialski P, Wonka P, Ye J (2013) Tensor completion for estimating missing values in visual data. IEEE Trans Pattern Anal Mach Intell 35(1):208–220

    Article  Google Scholar 

  48. Jia C, Zhong G, Fu YR (2014) Low-rank tensor learning with discriminant analysis for action classification and image recovery. In: Proceedings of AAAI Conference on Artificial Intelligence, pp 1228–1234

  49. Zhang K, Gao X, Tao D, Li X (2012) Multi-scale dictionary for single image super-resolution. In: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, pp 1114–1121

  50. Zhang K, Tao D, Gao X, Li X, Xiong Z (2015) Learning multiple linear mappings for efficient single image super-resolution. IEEE Trans Image Process 24(3):846–861

    Article  MathSciNet  Google Scholar 

  51. Zheng J, Jiang Z, Chellappa R (2016) Cross-view action recognition via transferable dictionary learning. IEEE Trans Image Process 25(6):2542–2556

    Article  MathSciNet  Google Scholar 

  52. Zontak M, Irani M (2011) Internal statistics of a single natural image. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp 977–984

Download references

Author information

Authors and Affiliations

  1. School of Electrical and Information Engineering, Tianjin University, Tianjin, China

    Peiguang Jing, Xu Bai, Hongbin Guo & Yuting Su

  2. Hewlett Packard Enterprise Singapore, Singapore, Singapore

    Weili Guan

Authors
  1. Peiguang Jing
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Weili Guan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xu Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hongbin Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yuting Su
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hongbin Guo.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jing, P., Guan, W., Bai, X. et al. Single image super-resolution via low-rank tensor representation and hierarchical dictionary learning. Multimed Tools Appl 79, 11767–11785 (2020). https://doi.org/10.1007/s11042-019-08259-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-08259-9

Keywords

Search

Navigation