Skip to main content
SpringerLink
Log in

Single-image super-resolution via joint statistic models-guided deep auto-encoder network

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Recent researches on super-resolution (SR) with deep learning networks have achieved amazing results. However, most of the existing studies neglect the internal distinctiveness of an image and the output of most methods tends to be of blurring, smoothness and implausibility. In this paper, we proposed a unified model which combines the deep model with the image restoration model for single-image SR. This model can not only reconstruct the SR image, but also keep the distinct fine structures for the low-resolution image. Two statistic priors are used to guide the updating of the output of the deep neural network: One is the non-local similarity and the other is the local smoothness. The former is modeled as the non-local total variation regularization, and the latter as the steering kernel regression total variation regularization. For this unified model, a new optimization function is formulated under a regularization framework. To optimize the total variation problem, a novel algorithm based on split Bregman iteration is developed with the theoretical proof of convergence. The experimental results demonstrate that the proposed unified model improves the peak signal-to-noise ratio of the deep SR model. Quantitative and qualitative results on four benchmark datasets show that the proposed model achieves better performance than the deep SR model without regularization terms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Bevilacqua M, Roumy A, Guillemot C, Alberi-Morel ML (2012) Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: British Machine Vision Conference (BMVC), 2012 

  2. Buades A, Coll B, Morel JM (2005) A non-local algorithm for image denoising. In: IEEE Computer Society conference on computer vision and pattern recognition, 2005. CVPR 2005, vol 2, pp 60–65. IEEE

  3. Chang H, Yeung DY, Xiong Y (2004) Super-resolution through neighbor embedding. In: Proceedings of the 2004 IEEE Computer Society conference on computer vision and pattern recognition, 2004. CVPR 2004, vol 1, p I. IEEE

  4. Cui Z, Chang H, Shan S, Zhong B, Chen X (2014) Deep network cascade for image super-resolution. Springer, Berlin, pp 49–64

    Google Scholar 

  5. Dong C, Loy CC, He K, Tang X (2016) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307

    Article  Google Scholar 

  6. Dong C, Loy CC, Tang X (2016) Accelerating the super-resolution convolutional neural network. Springer, Berlin, pp 391–407

    Google Scholar 

  7. Dong W, Zhang L, Lukac R, Shi G (2013) Sparse representation based image interpolation with nonlocal autoregressive modeling. IEEE Trans Image Process 22(4):1382–1394

    Article  MathSciNet  Google Scholar 

  8. Freeman WT, Jones TR, Pasztor EC (2002) Example-based super-resolution. IEEE Comput Graph Appl 22(2):56–65

    Article  Google Scholar 

  9. Goldstein T, Osher S (2009) The split Bregman method for l1-regularized problems. SIAM J Imaging Sci 2(2):323–343

    Article  MathSciNet  Google Scholar 

  10. Huang JB, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp 5197–5206

  11. Kim J, Kwon Lee J, Mu Lee K (2016) Accurate image super-resolution using very deep convolutional networks. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp 1646–1654

  12. Kim J, Kwon Lee J, Mu Lee K (2016) Deeply-recursive convolutional network for image super-resolution. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp 1637–1645

  13. Kim KI, Kwon Y (2010) Single-image super-resolution using sparse regression and natural image prior. IEEE Trans Pattern Anal Mach Intell 32(6):1127–1133

    Article  Google Scholar 

  14. Lai WS, Huang JB, Ahuja N, Yang MH (2017) Deep Laplacian pyramid networks for fast and accurate super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 624–632

  15. Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z et al (2016) Photo-realistic single image super-resolution using a generative adversarial network. arXiv preprint

  16. Li J, Qu Y, Li C, Xie Y, Wu Y, Fan J (2015) Learning local Gaussian process regression for image super-resolution. Neurocomputing 154:284–295

    Article  Google Scholar 

  17. Lim B, Son S, Kim H, Nah S, Lee KM (2017) Enhanced deep residual networks for single image super-resolution. In: The IEEE conference on computer vision and pattern recognition (CVPR) Workshops, vol 1, p 3

  18. Martin D, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Eighth IEEE international conference on computer vision, 2001. ICCV 2001. Proceedings, vol 2, pp 416–423. IEEE

  19. Ni KS, Nguyen TQ (2007) Image superresolution using support vector regression. IEEE Trans Image Process 16(6):1596–1610

    Article  MathSciNet  Google Scholar 

  20. Shi W, Caballero J, Huszár F, Totz J, Aitken AP, Bishop R, Rueckert D, Wang Z (2016) Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp 1874–1883

  21. Takeda H, Farsiu S, Milanfar P (2007) Kernel regression for image processing and reconstruction. IEEE Trans Image Process 16(2):349–366

    Article  MathSciNet  Google Scholar 

  22. Timofte R, De V, Van Gool L (2013) Anchored neighborhood regression for fast example-based super-resolution. In: 2013 IEEE international conference on computer vision (ICCV), pp 1920–1927. IEEE

  23. Wang R, Tao D (2016) Non-local auto-encoder with collaborative stabilization for image restoration. IEEE Trans Image Process 25(5):2117–2129

    Article  MathSciNet  Google Scholar 

  24. Wang Z, Liu D, Yang J, Han W, Huang T (2015) Deep networks for image super-resolution with sparse prior. In: Proceedings of the IEEE international conference on computer vision, pp 370–378

  25. Yang J, Wang Z, Lin Z, Cohen S, Huang T (2012) Coupled dictionary training for image super-resolution. IEEE Trans Image Process 21(8):3467–3478

    Article  MathSciNet  Google Scholar 

  26. Yang J, Wright J, Huang TS, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873

    Article  MathSciNet  Google Scholar 

  27. Zeng K, Yu J, Wang R, Li C, Tao D (2017) Coupled deep autoencoder for single image super-resolution. IEEE Trans Cybern 47(1):27–37

    Article  Google Scholar 

  28. Zeyde R, Elad M, Protter M (2010) On single image scale-up using sparse-representations. Springer, Berlin, pp 711–730

    MATH  Google Scholar 

  29. Zhang H, Cao X, Ho JKL, Chow TWS (2017) Object-level video advertising: an optimization framework. IEEE Trans Ind Inform 13(2):520–531

    Article  Google Scholar 

  30. Zhang H, Ji Y, Huang W, Liu L (2018) Sitcom-star-based clothing retrieval for video advertising: a deep learning framework.  Neural Comput Appl 1–20

  31. Zhang K, Gao X, Tao D, Li X (2012) Single image super-resolution with non-local means and steering kernel regression. IEEE Trans Image Process 21(11):4544–4556

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China under Grant 61876161, Grant 61772524, Grant 61601389 and Grant 61866022 and in part by the Beijing Natural Science Foundation under Grant 4182067.

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Xiamen University, Xiamen, China

    Rong Chen, Yanyun Qu & Cuihua Li

  2. College of Electronic Science and Technology, Xiamen University, Xiamen, China

    Kun Zeng

  3. Research Center of Precision Sensing and Control, Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Yuan Xie

  4. College of Information Engineering, Xizang Minzu University, Xianyang, China

    Rong Chen

  5. New Energy School, Lanzhou University of Technology, Lanzhou, China

    Ce Li

Authors
  1. Rong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yanyun Qu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Cuihua Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kun Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yuan Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ce Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yanyun Qu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, R., Qu, Y., Li, C. et al. Single-image super-resolution via joint statistic models-guided deep auto-encoder network. Neural Comput & Applic 32, 4885–4896 (2020). https://doi.org/10.1007/s00521-018-3886-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-018-3886-2

Keywords

Search

Navigation