Skip to main content
SpringerLink
Log in

An improved method for single image super-resolution based on deep learning

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

This paper strives for presenting an improved method for single image super-resolution based on deep learning, and therefore, a well-designed network structure is proposed by simultaneously considering the merits of convolutional sparse coding (CSC) and deep convolutional neural networks (CNN). In our model, contrary to most existing methods that directly operate on the raw input, we first perform a global decomposition on the input based on CSC for the purpose of extracting two specific components from it. Since the generated components are designed to have predefined physical meanings (i.e., residual or smooth), they can be discriminatively super-resolved according to their distinctive appearances. Specifically, a strong preference is given to the residual one as it is much more crucial to our task, while the other should just provide a quick reference. Based on this analysis, deep CNN and plain interpolation are selected to map them, respectively. In all, the proposed model integrates the above procedures into a completely end-to-end trainable deep network. Thorough experimental results demonstrate that our proposed network is able to gain considerable accuracy from this deep and delicate architecture, thereby outperforming many recently published baselines in terms of both objective evaluation and visual fidelity.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Nguyen, K., Fookes, C., Sridharan, S., Tistarelli, M., Nixon, M.: Super-resolution for biometrics: a comprehensive survey. Pattern Recogn. 78, 23–42 (2018)

    Article  Google Scholar 

  2. Nasrollahi, K., Escalera, S., Rasti, P., Anbarjafari, G., Baro, X., Escalante, H.J., Moeslund, T. B.: Deep learning based super-resolution for improved action recognition. In: Proceedings of 2015 International Conference on Image Processing Theory, Tools and Applications (IPTA), pp. 67–72. IEEE (2015)

  3. Zeng, W., Lu, X.: A generalized DAMRF image modeling for superresolution of license plates. IEEE Trans. Intell. Transp. Syst. 13(2), 828–837 (2012)

    Article  Google Scholar 

  4. Park, S.C., Park, M.K., Kang, M.G.: Super-resolution image reconstruction: a technical overview. IEEE Signal Process. Mag. 20(3), 21–36 (2003)

    Article  Google Scholar 

  5. Nasrollahi, K., Moeslund, T.B.: Super-resolution: a comprehensive survey. Mach. Vis. Appl. 25(6), 1423–1468 (2014)

    Article  Google Scholar 

  6. Zhu, S., Li, Y.: Single image super-resolution under multi-frame method. In: Signal, Image and Video Processing, pp. 1–9 (2018)

  7. Freeman, W.T., Pasztor, E.C., Carmichael, O.T.: Learning low-level vision. Int. J. Comput. Vis. 40(1), 25–47 (2000)

    Article  MATH  Google Scholar 

  8. Freeman, W.T., Jones, T.R., Pasztor, E.C.: Example-based super-resolution. IEEE Comput. Gr. Appl. 22(2), 56–65 (2002)

    Article  Google Scholar 

  9. Gao, X., Zhang, K., Tao, D., Li, X.: Image super-resolution with sparse neighbor embedding. IEEE Trans. Image Process. 21(7), 3194–3205 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  10. Chang, H., Yeung, D., Xiong, Y.: Super-resolution through neighbor embedding. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. I–I. IEEE (2004)

  11. Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  12. Candes, E.J., Wakin, M.B.: An introduction to compressive sampling. IEEE Signal Process. Mag. 25(2), 21–30 (2008)

    Article  Google Scholar 

  13. Yang, J., Wright, J., Huang, T.S., Ma, Y.: Image super-resolution via sparse representation. IEEE Trans. Image Process. 19(11), 2861–2873 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  14. Ahmed, J., Waqas, M., Ali, S., Memon, R.A., Klette, R.: Coupled dictionary learning in wavelet domain for Single-Image Super-Resolution. SIViP 12(3), 453–461 (2018)

    Article  Google Scholar 

  15. Chaudhry, A.M., Riaz, M.M., Ghafoor, A.: Super-resolution based on self-example learning and guided filtering. In: Signal, Image and Video Processing, pp. 1–8 (2018)

  16. Rasti, P., Nasrollahi, K., Orlova, O., Tamberg, G., Ozcinar, C., Moeslund, T.B., Anbarjafari, G.: A new low-complexity patch-based image super-resolution. IET Comput. Vision 11(7), 567–576 (2017)

    Article  Google Scholar 

  17. Rasti, P., Nasrollahi, K., Orlova, O., Tamberg, G., Moeslund, T.B., Anbarjafari, G.: Reducible dictionaries for single image super-resolution based on patch matching and mean shifting. J. Electron. Imaging 26(2), 23024 (2017)

    Article  Google Scholar 

  18. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Proceedings of Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

  19. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Proceedings of International Conference on Learning Representations (2015)

  20. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)

    Article  Google Scholar 

  21. Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. In: Curves and Surfaces, pp. 711–730. Springer (2012)

  22. Zeiler, M.D., Krishnan, D., Taylor, G.W., Fergus, R.: Deconvolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2528–2535. IEEE (2010)

  23. Liu, D., Wang, Z., Wen, B., Yang, J., Han, W., Huang, T.S.: Robust single image super-resolution via deep networks with sparse prior. IEEE Trans. Image Process. 25(7), 3194–3207 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  24. Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016)

    Article  Google Scholar 

  25. Chen, Y., Pock, T.: Trainable nonlinear reaction diffusion: a flexible framework for fast and effective image restoration. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1256–1272 (2017)

    Article  Google Scholar 

  26. Zhang, K., Zuo, W., Gu, S., Zhang, L.: Learning deep CNN denoiser prior for image restoration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017)

  27. Peleg, T., Elad, M.: A statistical prediction model based on sparse representations for single image super-resolution. IEEE Trans. Image Process. 23(6), 2569–2582 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  28. Gu, S., Zuo, W., Xie, Q., Meng, D., Feng, X., Zhang, L.: Convolutional sparse coding for image super-resolution. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1823–1831 (2015)

  29. Timofte, R., De Smet, V., Van Gool, L.: A + : adjusted anchored neighborhood regression for fast super-resolution. In: Proceedings of Asian Conference on Computer Vision, pp. 111–126. Springer (2014)

  30. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)

  31. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  32. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  33. Pascanu, R., Mikolov, T., Bengio, Y.: On the difficulty of training recurrent neural networks. In: Proceedings of International Conference on Machine Learning, pp. 1310–1318 (2013)

  34. Bevilacqua, M., Roumy, A., Guillemot, C., Morel, M.A.: Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: Proceedings British Machine Vision Conference (2012)

  35. Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 416–423. IEEE (2001)

  36. Huang, J., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: Proceedings of Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5197–5206 (2015)

  37. He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceeding of the IEEE International Conference on Computer Vision, pp. 1026–1034 (2015)

  38. Vedaldi, A., Lenc, K.: Matconvnet: convolutional neural networks for matlab. In: Proceedings of the 23rd ACM International Conference on Multimedia, pp. 689–692. ACM (2015)

  39. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (No. 61374194, No. 61403081), the National Key Science & Technology Pillar Program of China (No. 2014BAG01B03), the Key Research and Development Program of Jiangsu Province (No. BE2016739), and a Project Funded by the Priority Academic Program Development of Jiangsu Higher Education Institutions.

Author information

Authors and Affiliations

  1. College of Mechanical and Electronic Engineering, Nanjing Forestry University, Nanjing, 210037, China

    Chao Xie & Ying Liu

  2. College of Civil Aviation, Nanjing University of Aeronautics and Astronautics, Nanjing, 210016, China

    Weili Zeng

  3. School of Automation, Southeast University, Nanjing, 210096, China

    Xiaobo Lu

Authors
  1. Chao Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ying Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Weili Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaobo Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chao Xie.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xie, C., Liu, Y., Zeng, W. et al. An improved method for single image super-resolution based on deep learning. SIViP 13, 557–565 (2019). https://doi.org/10.1007/s11760-018-1382-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-018-1382-x

Keywords

Search

Navigation