Skip to main content
SpringerLink
Log in

Gradient information distillation network for real-time single-image super-resolution

  • Special Issue Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

In recent years, deep convolutional neural networks have played an increasingly important role in single-image super-resolution (SR). However, with the increase of the depth and width of networks, the super-resolution methods based on convolution neural networks are facing training difficulties, memory consumption, running slowness and other problems. Furthermore, most of the methods do not make full use of the image gradient information which leads to the loss of geometric structure information of the image. To solve these problems, we propose a gradient information distillation network in this paper. On the one hand, the advantages of fast and lightweight are maintained through information distillation. On the other hand, the SR performance is improved by gradient information. Our network has two branches named gradient information distillation branch (GIDB) and image information distillation branch. To combine features in both branches, we also introduce a residual feature transfer mechanism (RFT). Under the function of GIDB and RFT, our network can retain the rich geometric structure information which can make the edge details of the reconstructed image sharper. The experimental results show that our method is superior to the existing methods while well limits the parameters, computation and running time of the model. It provides the possibility for real-time image processing and mobile applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Agustsson, E., Timofte, R.: Ntire 2017 challenge on single image super-resolution: dataset and study. CVPR Workshops, pp. 1122–1131 (2017)

  2. Anoosheh, A., Sattler, T., Timofte, R., Pollefeys, M., Gool, V.L.: Night-to-day image translation for retrieval-based localization. In: International Conference on Robotics and Automation (2019)

  3. Bevilacqua, M., Roumy, A., Guillemot, C., Alberi-Morel, M.L.: Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: BMVC, pp. 1–10 (2012)

  4. Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. ECCV 4, 184–199 (2014)

    Google Scholar 

  5. Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016)

    Article  Google Scholar 

  6. Dong, C., Loy, C.C., Tang, X.: Accelerating the super-resolution convolutional neural network. In: ECCV (2016)

  7. Fattal, R.: Image upsampling via imposed edge statistics. In: ACM Trans. Graph. p. 95-es (2007)

  8. Huang, J.B., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: IEEE Conference on Computer Vision and Pattern Recognition (2015)

  9. Hui, Z., Wang, X., Gao, X.: Fast and accurate single image super-resolution via information distillation network. In: CVPR (2018)

  10. Kim, J., Lee, K.J., Lee, M.K.: Accurate image super-resolution using very deep convolutional networks. In: CVPR (2016)

  11. Kim, J., Lee, K.J., Lee, M.K.: Deeply-recursive convolutional network for image super-resolution. In: CVPR (2016)

  12. Kingma, P.D., Ba, L.J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations (2015)

  13. Lai, W.S., Huang, J.B., Ahuja, N., Yang, M.H.: Deep Laplacian pyramid networks for fast and accurate super-resolution. In: CVPR (2017)

  14. Luan, F., Paris, S., Shechtman, E., Bala, K.: Deep photo style transfer. In: CVPR (2017)

  15. Ma, C., Rao, Y., Cheng, Y., Chen, C., Lu, J., Zhou, J.: Structure-preserving super resolution with gradient guidance. In: CVPR, pp. 7766–7775 (2020)

  16. Mao, X.J., Shen, C., Yang, Y.B.: Image restoration using very deep convolutional encoder–decoder networks with symmetric skip connections. In: Lee, D., Sugiyama, M., Luxburg, U., Guyon, I., Garnett, R. (eds) Advances in Neural Information Processing Systems 29 (NIPS 2016). Curran Associates, Inc. (2016)

  17. Martin, R.D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Computer Vision, 2001. ICCV 2001. Proceedings. Eighth IEEE International Conference, Vol. 2, pp. 416–423 (2001)

  18. Matsui, Y., Ito, K., Aramaki, Y., Yamasaki, T., Aizawa, K.: Sketch-based manga retrieval using manga109 dataset. Multimed. Tools Appl. 76, 21811–21838 (2017)

    Article  Google Scholar 

  19. Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., Lerer, A.: Automatic differentiation in pytorch. In: Proceedings of the 31st conference on neural information processing systems (NIPS 2017), Long Beach, CA, USA (2017)

  20. Sajjadi, S.M.M., Schölkopf, B., Hirsch, M.: Enhancenet: single image super-resolution through automated texture synthesis. In: International Conference On Computer Vision (2017)

  21. Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, P.A., Bishop, R., Rueckert, D., Wang, Z.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: CVPR (2016)

  22. Sun, J., Sun, J., Xu, Z., Shum, H.Y.: Gradient profile prior and its applications in image super-resolution and enhancement. IEEE Trans. Image Process. 20, 1529–1542 (2011)

    Article  MathSciNet  Google Scholar 

  23. Tai, Y., Yang, J., Liu, X.: Image super-resolution via deep recursive residual network. In: CVPR, pp. 2790–2798 (2017)

  24. Tai, Y., Yang, J., Liu, X., Xu, C.: Memnet: A persistent memory network for image restoration. In: ICCV, pp. 4549–4557 (2017)

  25. Wang, Z., Bovik, C.A., Sheikh, R.H., Simoncelli, P.E.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13, 600–612 (2004)

    Article  Google Scholar 

  26. Yan, Q., Xu, Y., Yang, X., Nguyen, Q.T.: Single image super-resolution based on gradient profile sharpness. IEEE Trans. Image Process. Publ. IEEE Signal Process. Soc. 24, 3187–202 (2015)

    MATH  Google Scholar 

  27. Yang, W., Feng, J., Yang, J., Zhao, F., Liu, J., Guo, Z., Yan, S.: Deep edge guided recurrent residual learning for image super-resolution. IEEE Trans. Image Process. 26, 5895–5907 (2017)

    Article  MathSciNet  Google Scholar 

  28. Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. In: Curves and Surfaces, pp. 711–730 (2010)

  29. Zhang, K., Zuo, W., Zhang, L.: Learning a single convolutional super-resolution network for multiple degradations. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 18–23 June 2018. IEEE, Salt Lake City, UT, USA (2018). https://doi.org/10.1109/CVPR.2018.00344

  30. Zhu, Y., Zhang, Y., Bonev, B., Yuille, L.A.: Modeling deformable gradient compositions for single-image super-resolution. In: IEEE Conference on Computer Vision and Pattern Recognition (2015)

Download references

Acknowledgements

This work is sponsored by the National Key R&D Program of China, under Grant No. 2020AAA0104500. The funding is from Sichuan University under Grant No. 2020SCUNG205.

Author information

Authors and Affiliations

  1. College of Electronics and Information Engineering, Sichuan University, Chengdu, 610064, Sichuan, China

    Bin Meng, Lining Wang, Zheng He & Xiaomin Yang

  2. School of Aeronautics & Astronautics, Sichuan University, Chengdu, 610064, Sichuan, China

    Bin Meng, Lining Wang, Zheng He & Xiaomin Yang

  3. Department of Embedded Systems Engineering, Incheon National University, Incheon, 22012, Korea

    Gwanggil Jeon

  4. The Center of Gerontology and Geriatrics, West China Hospital, Sichuan University, Chengdu, 610064, Sichuan, China

    Qingyu Dou

Authors
  1. Bin Meng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lining Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zheng He
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gwanggil Jeon
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Qingyu Dou
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Xiaomin Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Gwanggil Jeon or Qingyu Dou.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Meng, B., Wang, L., He, Z. et al. Gradient information distillation network for real-time single-image super-resolution. J Real-Time Image Proc 18, 333–344 (2021). https://doi.org/10.1007/s11554-021-01083-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-021-01083-1

Keywords

Search

Navigation