Skip to main content
SpringerLink
Log in

Image super-resolution network based on a multi-branch attention mechanism

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

As a member of low-level visual tasks, image super-resolution (SR) is now mostly implemented by deep learning. Although the deeper convolution neural network can bring larger receptive field, it will increase the amount of calculation, make the training difficult and reduce efficiency. In addition, the feature information obtained by each channel plays a different and important role in the detail recovery during the SR process. To settle the above problems and improve the performance, we develop a multi-branch attention SR model. The main network contains multiple residual bodies, which are composed of several residual units. Additionally, we construct a multi-branch attention mechanism, which divides all channels into equal parts. Then, the network learns the relationship between channels and focuses more on the high-frequency feature channels. Experimental results show that the proposed algorithm is superior to the state-of-the-art algorithms in terms of subjective visual quality and objective evaluation criteria.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Greenspan, H.: Super-resolution in medical imaging. Comput J 52(1), 43–63 (2009)

    Article  Google Scholar 

  2. Lillesand, T., Kiefer, R.W., Chipman, J.: Remote Sensing and Image Interpretation. Wliley, New York (2014)

    Google Scholar 

  3. Lobanov, A.P.: Resolution limits in astronomical images. arXiv:astro-ph/0503225 (2005)

  4. Harris, J.L.: Diffraction and resolving power (1917–1983). J. Opt. Soc. Am. 54(7), 931–933 (1964)

    Article  Google Scholar 

  5. Goodman, J.W.: Introduction to Fourier Optics. McGraw-Hill, London (1968)

    Google Scholar 

  6. Tsai, R.Y., Huang, T.S.: Multi-frame image restoration and registration. Adv. Comput. Vis. Image Process. 1(2), 317–339 (1984)

    Google Scholar 

  7. Irani, M.: Improving resolution by image registration. GVGIP Graph. Models Image Process. 53, 231–239 (1991)

    Article  Google Scholar 

  8. Schultz, R.R., Stevenson, R.L.: Extraction of high-resolution frames from video sequences. IEEE Trans. Image Process. Publ. IEEE Sig. Process. Soc. 5(6), 996–1011 (1996)

    Article  Google Scholar 

  9. Gao, X., Zhang, K., Tao, D., et al.: Image super-resolution with sparse neighbor embedding. IEEE Trans. Image Process. 21(7), 3194–3205 (2012)

    Article  MathSciNet  Google Scholar 

  10. Dong, C., Loy, C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: ECCV (2014)

  11. Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)

  12. Lim B, Son S, Kim H, et al. Enhanced deep residual networks for single image super-resolution. In: 2017 IEEE conference on computer vision and pattern recognition workshops (CVPRW). IEEE Computer Society (2017)

  13. Zhang, Y., et al.: Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European Conference on Computer Vision (ECCV) (2018)

  14. Kim, J., Kwon Lee, J., Mu Lee, K.: Deeply-recursive convolutional network for image super-resolution. In: CVPR (2016)

  15. Zhang, Y., Tian, Y., Kong, Y., Zhong, B., Fu, Y.: Residual dense network for image super-resolution. In: CVPR (2018)

  16. Wang, X., Yu, K., Wu, S., Gu, J., Liu, Y., Dong, C., Loy, C.C., Qiao, Y., Tang, X.: Esrgan: enhanced super-resolution generative adversarial networks. arXiv:1809.00219 (2018)

  17. Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017)

  18. Ma, C., Rao, Y., Cheng, Y., et al.: Structure-preserving super resolution with gradient guidance. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7769–7778 (2020)

  19. Ren, H., El-Khamy, M., Lee, J.: Image super resolution based on fusing multiple convolution neural networks. In: CVPRW (2017)

  20. Hu, Y., Gao, X., Li, J., Huang, Y., Wang, H.: Single image super-resolution via cascaded multi-scale cross network. arXiv:1802.08808 (2018)

  21. Hui, Z., Wang, X., Gao, X.: Fast and accurate single image super-resolution via information distillation network. In: CVPR (2018)

  22. Hui, Z., et al.: Lightweight Image Super-Resolution with Information Multi-distillation Network. arXiv:1909.11856v1 (2019)

  23. Choi, J., Kim, M.: A deep convolutional neural network with selection units for super-resolution. In: CVPRW (2017)

  24. Kim, J.-H., Choi, J.-H., Cheon, M., Lee, J.-S.: Ram: residual attention module for single image super-resolution. arXiv:1811.12043 (2018)

  25. Dai, T., Cai, J., Zhang, Y., et al.: Second-order attention network for single image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 11065–11074 (2019)

  26. Li, X., Wang, W., Hu, X., et al.: Selective kernel networks In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2020)

  27. Timofte, R., Agustsson, E., Van Gool, L., Yang, M.-H., Zhang, L., Lim, B., Son, S., Kim, H., Nah, S., Lee, K.M. et al.: Ntire 2017 challenge on single image super-resolution: methods and results. In: CVPRW (2017)

  28. Bevilacqua, M., Roumy, A., Guillemot, C., Alberi-Morel, M.L.: Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: Proceedings of the 23rd British Machine Vision Conference (BMVC). BMVA Press, pp. 135.1–135.10 (2012)

  29. Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. In: International Conference on Curves and Surfaces (2010)

  30. Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: ICCV (2001)

  31. Huang, J.-B., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: CVPR (2015)

  32. Fujimoto, A., Ogawa, T., Yamamoto, K., Matsui, Y., Yamasaki, T., Aizawa, K.: Manga109 dataset and creation of metadata. In: International Workshop on coMics ANalysis, Processing and Understanding (2016)

  33. Dong, C., Loy, C.C., Tang, X.: Accelerating the super-resolution convolutional neural network. In: ECCV (2016)

  34. Lai, W.-S., Huang, J.-B., Ahuja, N., Yang, M.-H.: Deep Laplacian pyramid networks for fast and accurate super-resolution. In: CVPR (2017)

  35. Tai, Y., Yang, J., Liu, X., Xu, C.: Memnet: a persistent memory network for image restoration. In: CVPR (2017)

  36. Wang, C., Li, Z., Shi, J.: Lightweight image super-resolution with adaptive weighted learning network. arXiv:1904.02358 (2019)

  37. Haris, M., Shakhnarovich, G., Ukita, N.: Deep back-projection networks for super-resolution. In: CVPR (2018)

  38. Li, Z., et al.: Feedback network for image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2019)

Download references

Acknowledgments

This research was supported by the National Natural Science Foundation of China (61573182) and by the Fundamental Research Funds for the Central Universities (NS2020025).

Author information

Authors and Affiliations

  1. Nanjing, China

    Xin Yang, Yingqing Guo, Zhiqiang Li & Dake Zhou

Authors
  1. Xin Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yingqing Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhiqiang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dake Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xin Yang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (RAR 124 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, X., Guo, Y., Li, Z. et al. Image super-resolution network based on a multi-branch attention mechanism. SIViP 15, 1397–1405 (2021). https://doi.org/10.1007/s11760-021-01870-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-021-01870-0

Keywords

Search

Navigation