Skip to main content
SpringerLink
Log in

Residual attention network using multi-channel dense connections for image super-resolution

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

In recent years, the methods based on deep convolutional neural networks (DCNN) have greatly promoted the development of image super-resolution (SR). However, deeper and wider SR networks are more difficult to train. For the SR task, the low-frequency information contained in low-resolution images is very important, and the neglect of exploring the feature information across channels hinders the representational capability of DCNN. To address these problems, we enhance the representational capability of DCNN by utilizing the interactive relationship among multiple channels. In this paper, a residual attention network using multi-channel dense connections (MCRAN) is proposed to improve the image super-resolution significantly. This method can make full use of multi-channel information for more effective feature expression and learning. In MCRAN, a multi-channel residual attention (MCRA) module is designed to coalesce the features of multiple different channels and the attention mechanism is applied to adjust the channel features adaptively. Accordingly, the channel features own more discriminative representation. In addition, the multi-source residual group (MSRG) structure is developed to construct a deeper network and simplify the training of network, which contains several long non-local skip connections (L-NLSC) to capture global low-frequency information in remote space. Besides, MSRG contains some short local-source skip connections (S-LSSC) to enhance the information interaction of local network. Extensive experimental evaluation on benchmark datasets on single image super-resolution proves the superiority of the proposed MCRAN.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Yıldırım D, Güngör O (2012) A novel image fusion method using ikonos satellite images. J Geodesy Geoinform 1(1):75– 83

    Article  Google Scholar 

  2. Zou WW, Yuen PC (2011) Very low resolution face recognition problem. IEEE Trans Image Process 21(1):327–340

    Article  MathSciNet  Google Scholar 

  3. Shi W, Caballero J, Ledig C, Zhuang X, Bai W, Bhatia K, de Marvao AMSM, Dawes T, O’ Regan D, Rueckert D (2013) Cardiac image super-resolution with global correspondence using multi-atlas patchmatch. In: International conference on medical image computing and computer-assisted intervention, pp 9–16. Springer

  4. Zhang L, Wu X (2006) An edge-guided image interpolation algorithm via directional filtering and data fusion. IEEE Trans Image Process 15(8):2226–2238

    Article  Google Scholar 

  5. Yang CY, Yang MH (2013) Fast direct super-resolution by simple functions. In: Proceedings of the IEEE international conference on computer vision, pp 561–568

  6. Yang J, Wright J, Huang T, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873

    Article  MathSciNet  Google Scholar 

  7. Zhang K, Zuo W, Zhang L (2018) Learning a single convolutional super-resolution network for multiple degradations. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3262–3271

  8. Lai WS, Huang JB, Ahuja N, Yang MH (2017) Deep laplacian pyramid networks for fast and accurate super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 624–632

  9. Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z et al (2017) Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4681–4690

  10. Dong C, Loy CC, He K, Tang X (2015) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307

    Article  Google Scholar 

  11. Park SC, Park MK, Kang MG (2003) Super-resolution image reconstruction: a technical overview. IEEE Signal Process Mag 20(3):21–36

    Article  Google Scholar 

  12. Dong C, Loy CC, Tang X (2016) Accelerating the super-resolution convolutional neural network. In: European conference on computer vision, pp 391–407. Springer

  13. Shi W, Caballero J, Huszár F, Totz J, Aitken AP, Bishop R, Rueckert D, Wang Z (2016) Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1874–1883

  14. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  15. Kim J, Kwon Lee J, Mu Lee K (2016) Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1646–1654

  16. Kim J, Kwon Lee J, Mu Lee K (2016) Deeply-recursive convolutional network for image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1637–1645

  17. Lim B, Son S, Kim H, Nah S, Mu Lee K (2017) Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 136–144

  18. Tai Y, Yang J, Liu X (2017) Image super-resolution via deep recursive residual network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3147–3155

  19. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708

  20. Tong T, Li G, Liu X, Gao Q (2017) Image super-resolution using dense skip connections. In: Proceedings of the IEEE International Conference on Computer Vision, pp 4799–4807

  21. Tai Y, Yang J, Liu X, Xu C (2017) Memnet: a persistent memory network for image restoration. In: Proceedings of the IEEE international conference on computer vision, pp 4539–4547

  22. Zhang Y, Tian Y, Kong Y, Zhong B, Fu Y (2018) Residual dense network for image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2472–2481

  23. Wang Z, Chen J, Hoi SC (2019) Deep learning for image super-resolution: A survey. arXiv:1902.06068

  24. Xie S, Girshick R, Dollár P, Tu Z, He K (2017) Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1492–1500

  25. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  26. Li Z, Yang J, Liu Z, Yang X, Jeon G, Wu W (2019) Feedback network for image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3867–3876

  27. Srivastava RK, Greff K, Schmidhuber J (2015) Training very deep networks. In: Advances in neural information processing systems, pp 2377–2385

  28. Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. arXiv:1409.0473

  29. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141

  30. Cao C, Liu X, Yang Y, Yu Y, Wang J, Wang Z, Huang Y, Wang L, Huang C, Xu W et al (2015) Look and think twice: Capturing top-down visual attention with feedback convolutional neural networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp 2956–2964

  31. Wang F, Jiang M, Qian C, Yang S, Li C, Zhang H, Wang X, Tang X (2017) Residual attention network for image classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3156–3164

  32. Odena A, Dumoulin V, Olah C (2016) Deconvolution and checkerboard artifacts. Distill 1 (10):e3

    Article  Google Scholar 

  33. Yu J, Fan Y, Yang J, Xu N, Wang Z, Wang X, Huang T (2018) Wide activation for efficient and accurate image super-resolution. arXiv:1808.08718

  34. Hu X, Mu H, Zhang X, Wang Z, Tan T, Sun J (2019) Meta-sr: a magnification-arbitrary network for super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1575–1584

  35. Haris M, Shakhnarovich G, Ukita N (2018) Deep back-projection networks for super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1664–1673

  36. Zeng K, Ding S, Jia W (2019) Single image super-resolution using a polymorphic parallel cnn. Appl Intell 49(1):292–300

    Article  Google Scholar 

  37. Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv:1412.6980

  38. Veit A, Wilber MJ, Belongie S (2016) Residual networks behave like ensembles of relatively shallow networks. In: Advances in neural information processing systems, pp 550–558

  39. Timofte R, Agustsson E, Van Gool L, Yang MH, Zhang L (2017) Ntire 2017 challenge on single image super-resolution: Methods and results. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp 114–125

  40. Bevilacqua M, Roumy A, Guillemot C, Alberi-morel ML (2012) Low-complexity single-image super-resolution based on nonnegative neighbor embedding

  41. Zeyde R, Elad M, Protter M (2010) On single image scale-up using sparse-representations. In: International conference on curves and surfaces, pp 711–730. Springer

  42. Martin D, Fowlkes C, Tal D, Malik J et al (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics Iccv Vancouver

  43. Huang JB, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 5197–5206

  44. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP et al (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  45. Han W, Chang S, Liu D, Yu M, Witbrock M, Huang T (2018) Image super-resolution via dual-state recurrent networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1654–1663

Download references

Acknowledgements

This work was supported by Jiangxi provincial Outstanding Youth Talent Project of Science and Technology Innovation (No. 20192BCBL23003) and National Natural Science Foundation of China (61601185).

Author information

Authors and Affiliations

  1. Department of Information Engineering, East China Jiaotong University, Nanchang, China

    Zhiwei Liu, Ji Huang, Chengjia Zhu, Xiaoyu Peng & Xinyu Du

Authors
  1. Zhiwei Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ji Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chengjia Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaoyu Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xinyu Du
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhiwei Liu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, Z., Huang, J., Zhu, C. et al. Residual attention network using multi-channel dense connections for image super-resolution. Appl Intell 51, 85–99 (2021). https://doi.org/10.1007/s10489-020-01723-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-01723-2

Keywords

Search

Navigation