Skip to main content
SpringerLink
Log in

Densely Multi-path Network for Single Image Super-Resolution

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2020)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12532))

Included in the following conference series:

  • 2254 Accesses

Abstract

Recently, deep convolutional neural networks (CNNs) make many breakthroughs in accuracy and speed for single image super-resolution (SISR). However, we observe that the fusion of information on different receptive fields have not been fully exploited in current SR methods. In this paper, we propose a novel deep densely multi-path network (DMPN) for SISR that introduces densely multi-path blocks (DMPBs). A DMPB contains several multi-path subnets (MPSs) with dense skip connections, and concatenates the outputs of MPSs that are fed into the next DMPB. A MPS uses convolution kernels of different sizes in each path, and exchanges information through cross-path skip connections. Such a multi-path fusion strategy allows the network to make full use of different levels of information and better adapt for extracting high-frequency features. Quantitative and qualitative experimental results indicate the effectiveness of the proposed DMPN, which achieves better restoration performance and visual effects than state-of-the-art algorithms.

Supported by the Natural Science Foundation of the Jiangsu Higher Education Institutions of China under Grant No. 19KJA550002, the Six Talent Peak Project of Jiangsu Province of China under Grant No. XYDXX-054, and the Priority Academic Program Development of Jiangsu Higher Education Institutions.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Agustsson, E., Timofte, R.: Ntire 2017 challenge on single image super-resolution: dataset and study. In: Computer Vision and Pattern Recognition Workshops, pp. 1122–1131. IEEE (2017). https://doi.org/10.1109/CVPRW.2017.150

  2. Ahn, N., Kang, B., Sohn, K.-A.: Fast, accurate, and lightweight super-resolution with cascading residual network. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11214, pp. 256–272. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01249-6_16

    Chapter  Google Scholar 

  3. Bevilacqua, M., Roumy, A., Guillemot, C., Alberi Morel, M.L.: Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: British Machine Vision Conference, pp. 135.1-135.10. BMVA (2012). https://doi.org/10.5244/C.26.135

  4. Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 184–199. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10593-2_13

    Chapter  Google Scholar 

  5. Freeman, W.T., Jones, T.R., Pasztor, E.C.: Example-based super-resolution. IEEE Comput. Graphics Appl. 22(2), 56–65 (2002)

    Article  Google Scholar 

  6. Greenspan, H.: Super-resolution in medical imaging. Comput. J. 52(1), 43–63 (2008)

    Article  Google Scholar 

  7. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Computer Vision and Pattern Recognition, pp. 770–778. IEEE (2016). https://doi.org/10.1109/CVPR.2016.90

  8. He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: International Conference on Computer Vision, pp. 1026–1034. IEEE (2015). https://doi.org/10.1109/ICCV.2015.123

  9. Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Computer Vision and Pattern Recognition, pp. 2261–2269. IEEE (2017). https://doi.org/10.1109/CVPR.2017.243

  10. Huang, J., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: Computer Vision and Pattern Recognition, pp. 5197–5206. IEEE (2015). https://doi.org/10.1109/CVPR.2015.7299156

  11. Huang, Y., Shao, L., Frangi, A.F.: Simultaneous super-resolution and cross-modality synthesis of 3d medical images using weakly-supervised joint convolutional sparse coding. In: Computer Vision and Pattern Recognition, pp. 5787–5796. IEEE (2017). https://doi.org/10.1109/CVPR.2017.613

  12. Sun, J., Xu, Z, Shum, H.Y.: Image super-resolution using gradient profile prior. In: Computer Vision and Pattern Recognition, pp. 1–8. IEEE (2008). https://doi.org/10.1109/CVPR.2008.4587659

  13. Keys, R.: Cubic convolution interpolation for digital image processing. IEEE Trans. Acoust. Speech Signal Process. 29(6), 1153–1160 (1981)

    Article  MathSciNet  Google Scholar 

  14. Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. In: Computer Vision and Pattern Recognition, pp. 1646–1654. IEEE (2016). https://doi.org/10.1109/CVPR.2016.182

  15. Kim, J., Lee, J.K., Lee, K.M.: Deeply-recursive convolutional network for image super-resolution. In: Computer Vision and Pattern Recognition, pp. 1637–1645. IEEE (2016). https://doi.org/10.1109/CVPR.2016.181

  16. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. In: International Conference on Learning Representations. arXiv:1412.6980 (2015)

  17. Lai, W.S., Huang, J.B., Ahuja, N., Yang, M.H.: Deep laplacian pyramid networks for fast and accurate super-resolution. In: Computer Vision and Pattern Recognition, pp. 5835–5843. IEEE (2017). https://doi.org/10.1109/CVPR.2017.618

  18. Li, J., Fang, F., Mei, K., Zhang, G.: Multi-scale residual network for image super-resolution. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11212, pp. 527–542. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01237-3_32

    Chapter  Google Scholar 

  19. Lim, B., Son, S., Kim, H., Nah, S., Lee, K.M.: Enhanced deep residual networks for single image super-resolution. In: Computer Vision and Pattern Recognition Workshops, pp. 1132–1140. IEEE (2017). https://doi.org/10.1109/CVPRW.2017.151

  20. Liu, D., Wen, B., Fan, Y., Loy, C., Huang, T.: Non-local recurrent network for image restoration. In: Neural Information Processing Systems, pp. 1673–1682. Curran (2018)

    Google Scholar 

  21. Lobanov, A.P.: Resolution limits in astronomical images. arXiv:astro-ph/0503225 (2005)

  22. Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: International Conference on Computer Vision. ICCV 2001, vol. 2, pp. 416–423. IEEE (2001). https://doi.org/10.1109/ICCV.2001.937655

  23. Rasti, P., Uiboupin, T., Escalera, S., Anbarjafari, G.: Convolutional neural network super resolution for face recognition in surveillance monitoring. In: Perales, F.J.J., Kittler, J. (eds.) AMDO 2016. LNCS, vol. 9756, pp. 175–184. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-41778-3_18

    Chapter  Google Scholar 

  24. Shi, W., et al.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Computer Vision and Pattern Recognition, pp. 1874–1883. IEEE (2016). https://doi.org/10.1007/978-3-319-41778-3_18

  25. Swaminathan, A., Wu, M., Liu, K.J.R.: Digital image forensics via intrinsic fingerprints. IEEE Trans. Inf. Forensics Secur. 3(1), 101–117 (2008)

    Article  Google Scholar 

  26. Tai, Y., Yang, J., Liu, X.: Image super-resolution via deep recursive residual network. In: Computer Vision and Pattern Recognition, pp. 2790–2798. IEEE (2017). https://doi.org/10.1109/CVPR.2017.298

  27. Tai, Y., Yang, J., Liu, X., Xu, C.: Memnet: a persistent memory network for image restoration. In: International Conference on Computer Vision, pp. 4549–4557. IEEE (2017). https://doi.org/10.1109/ICCV.2017.486

  28. Timofte, R., Agustsson, E., Gool, L.V.: Ntire 2017 challenge on single image super-resolution: Methods and results. In: Computer Vision and Pattern Recognition Workshops, pp. 1110–1121. IEEE (2017). https://doi.org/10.1109/CVPRW.2017.149

  29. Tong, T., Li, G., Liu, X., Gao, Q.: Image super-resolution using dense skip connections. In: International Conference on Computer Vision, pp. 4809–4817. IEEE (2017). https://doi.org/10.1109/ICCV.2017.514

  30. Xiong, Z., Sun, X., Wu, F.: Robust web image/video super-resolution. IEEE Trans. Image Process. 19(8), 2017–2028 (2010)

    Article  MathSciNet  Google Scholar 

  31. Yang, J., Wright, J., Huang, T.S., Ma, Y.: Image super-resolution via sparse representation. IEEE Trans. Image Process. 19(11), 2861–2873 (2010)

    Article  MathSciNet  Google Scholar 

  32. Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. In: Boissonnat, J.-D., Chenin, P., Cohen, A., Gout, C., Lyche, T., Mazure, M.-L., Schumaker, L. (eds.) Curves and Surfaces 2010. LNCS, vol. 6920, pp. 711–730. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-27413-8_47

    Chapter  Google Scholar 

  33. Zhang, L., Zhang, H., Shen, H., Li, P.: A super-resolution reconstruction algorithm for surveillance images. Sig. Process. 90(3), 848–859 (2010)

    Article  Google Scholar 

  34. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Technology and Joint International Research Laboratory of Machine Learning and Neuromorphic Computing, Soochow University, Suzhou, 215006, Jiangsu, China

    Shi Xu & Li Zhang

  2. Provincial Key Laboratory for Computer Information Processing Technology, Soochow University, Suzhou, 215006, Jiangsu, China

    Li Zhang

Authors
  1. Shi Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Li Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Li Zhang .

Editor information

Editors and Affiliations

  1. Department of AI, Ping An Life, Shenzhen, China

    Haiqin Yang

  2. Faculty of Information Technology, King Mongkut's Institute of Technology Ladkrabang, Bangkok, Thailand

    Kitsuchart Pasupa

  3. City University of Hong Kong, Kowloon, Hong Kong

    Andrew Chi-Sing Leung

  4. Department of Computer Science and Engineering, Hong Kong University of Science and Technology, Hong Kong, Hong Kong

    James T. Kwok

  5. School of Information Technology, King Mongkut’s University of Technology Thonburi, Bangkok, Thailand

    Jonathan H. Chan

  6. The Chinese University of Hong Kong, New Territories, Hong Kong

    Irwin King

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Xu, S., Zhang, L. (2020). Densely Multi-path Network for Single Image Super-Resolution. In: Yang, H., Pasupa, K., Leung, A.CS., Kwok, J.T., Chan, J.H., King, I. (eds) Neural Information Processing. ICONIP 2020. Lecture Notes in Computer Science(), vol 12532. Springer, Cham. https://doi.org/10.1007/978-3-030-63830-6_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-63830-6_22

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-63829-0

  • Online ISBN: 978-3-030-63830-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation