Skip to main content
SpringerLink
Log in

Residual deep attention mechanism and adaptive reconstruction network for single image super-resolution

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The low-level image processing task single image super-resolution (SISR) has a long research history. In recent years, convolutional neural networks (CNNs) have been widely used in single image super-resolution (SISR), and significant performance has been achieved. However, most existing CNN-based SISR methods seldom take into account the feature correlations of the original low-quality images, neglecting to treat spatial and channel-wise features differently. The performance of CNN-based SISR models is often enhanced by deploying very deep networks, which inevitably hinders the representational power of the CNNs and results in many obvious shortcomings. To address these issues, in this paper, we propose a residual deep attention mechanism and adaptive reconstruction network (RAAN) with stronger feature expression and learning abilities. Specifically, to discriminate between large and small gray changes in adjacent areas in LR images, a novel spatial and channel attention processing module (SCAM) is developed that incorporates non-local operations to capture long-distance dependencies between pixels in the spatial domain, and automatically rescales hierarchical features with different weights. Furthermore, we present an enhanced residual attention group (ERAG) structure that not only incorporates some feature processing groups (FPGs), but also contains several source skip connections (SSCs). Similarly, we deploy a global residual long skip connection. With the combination of these skip connections, the low-frequency information flows more effectively to the tail of the network. Moreover, in the upsampling module, we implement four different sizes of convolution kernels (i.e., 3×3, 5×5, 7×7 and 9×9) to extract feature fusion and magnify to the required scale. Experimental results demonstrate the superiority of our RAAN over state-of-the-art SISR methods in terms of both quantitative metrics and visual quality.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Ahn N, Kang B, Sohn KA (2018) Fast, accurate, and lightweight super-resolution with cascading residual network. In: European Conference on Computer Vision, Springer, pp 256–272

  2. Cao Y, Xu J, Lin S, Wei F, Hu H (2019) Gcnet: Non-Local networks meet squeeze-excitation networks and beyond. In: 2019 IEEE/CVF International conference on computer vision workshop (ICCVW). IEEE, pp 1971–1980

  3. Dong C, Loy CC, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. In: European conference on computer vision, Springer, pp 184–199

  4. Dong C, Loy C C, He K, Tang X (2015) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307

    Article  Google Scholar 

  5. Dong C, Loy CC, Tang X (2016) Accelerating the super-resolution convolutional neural network. In: European conference on computer vision. Springer, pp 391–407

  6. Haris M, Shakhnarovich G, Ukita N (2018) Deep back-projection networks for super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1664–1673

  7. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141

  8. Hui Z, Gao X, Yang Y, Wang X (2019) Lightweight image super-resolution with information multi-distillation network. In: Proceedings of the 27th ACM International Conference on Multimedia, pp 2024–2032

  9. Jiang K, Wang Z, Yi P, Jiang J (2020) Hierarchical dense recursive network for image super-resolution. Pattern Recogn 107 :107475

    Article  Google Scholar 

  10. Johnson J, Alahi A, Fei-Fei L (2016) Perceptual losses for real-time style transfer and super-resolution. In: European conference on computer vision. Springer, pp 694–711

  11. Ketkar N (2017) Introduction to pytorch. In: Deep learning with python, Springer, pp 195–208

  12. Kim J, Kwon Lee J, Mu Lee K (2016a) Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1646–1654

  13. Kim J, Kwon Lee J, Mu Lee K (2016b) Deeply-recursive convolutional network for image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1637–1645

  14. Lai WS, Huang JB, Ahuja N, Yang MH (2017) Deep laplacian pyramid networks for fast and accurate super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 624–632

  15. Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z et al (2017) Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4681–4690

  16. Lim B, Son S, Kim H, Nah S, Mu Lee K (2017) Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 136–144

  17. Liu J, Zhang W, Tang Y, Tang J, Wu G (2020) Residual feature aggregation network for image super-resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 2359–2368

  18. Luo X, Xie Y, Zhang Y, Qu Y, Fu Y (2020) Latticenet: Towards Lightweight Image Super-Resolution with Lattice Block. Computer Vision – ECCV 2020

  19. Martin D, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001, vol 2. IEEE, pp 416–423

  20. Mei Y, Fan Y, Zhou Y, Huang L, Huang TS, Shi H (2020) Image super-resolution with cross-scale non-local attention and exhaustive self-exemplars mining. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 5690–5699

  21. Sajjadi MS, Scholkopf B, Hirsch M (2017) Enhancenet: Single image super-resolution through automated texture synthesis. In: Proceedings of the IEEE International Conference on Computer Vision, pp 4491–4500

  22. Shi W, Caballero J, Ledig C, Zhuang X, Bai W, Bhatia K, de Marvao AMSM, Dawes T, O’Regan D, Rueckert D (2013) Cardiac image super-resolution with global correspondence using multi-atlas patchmatch. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 9–16

  23. Shi W, Caballero J, Huszár F, Totz J, Aitken AP, Bishop R, Rueckert D, Wang Z (2016) Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1874–1883

  24. Tai Y, Yang J, Liu X (2017a) Image super-resolution via deep recursive residual network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3147–3155

  25. Tai Y, Yang J, Liu X, Xu C (2017b) Memnet: A persistent memory network for image restoration. In: Proceedings of the IEEE international conference on computer vision, pp 4539–4547

  26. Timofte R, Agustsson E, Van Gool L, Yang MH, Zhang L (2017) Ntire 2017 challenge on single image super-resolution: Methods and results. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 114–125

  27. Tong T, Li G, Liu X, Gao Q (2017) Image super-resolution using dense skip connections. In: Proceedings of the IEEE International Conference on Computer Vision, pp 4799–4807

  28. Wang M, Wang Q (2021) Hypergraph-regularized sparse representation for single color image super resolution. J Vis Commun Image Represent 74:102951

    Article  Google Scholar 

  29. Wang X, Girshick R, Gupta A, He K (2018) Non-local neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7794–7803

  30. Wang Z, Chen J (2020) Hoi SC. A survey. IEEE transactions on pattern analysis and machine intelligence, Deep learning for image super-resolution

    Google Scholar 

  31. Xie T, Yang X, Jia Y, Zhu C, Xiaochuan L (2019) Adaptive densely connected single image super-resolution. In: 2019 IEEE/CVF International conference on computer vision workshop (ICCVW). IEEE, pp 3432–3440

  32. Yang J, Wright J, Huang T, Ma Y (2008) Image super-resolution as sparse representation of raw image patches. In: 2008 IEEE conference on computer vision and pattern recognition. IEEE, pp 1–8

  33. Zhang K, Zuo W, Zhang L (2018a) Learning a single convolutional super-resolution network for multiple degradations. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3262–3271

  34. Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y (2018b) Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 286–301

  35. Zhang Y, Tian Y, Kong Y, Zhong B, Fu Y (2018c) Residual dense network for image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2472–2481

  36. Zhang Y, Li K, Li K, Zhong B, Fu Y (2019) Residual non-local attention networks for image restoration. In: International Conference on Learning Representations

  37. Zou W W, Yuen P C (2011) Very low resolution face recognition problem. IEEE Trans Image Process 21(1):327–340

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Intelligent Equipment, Shandong University of Science and Technology, Tai’an, 271000, China

    Hongjuan Wang

  2. College of Computer Science and Engineering, Shandong University of Science and Technology, Qingdao, 266590, China

    Mingrun Wei, Ru Cheng, Yue Yu & Xingli Zhang

Authors
  1. Hongjuan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mingrun Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ru Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yue Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xingli Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hongjuan Wang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, H., Wei, M., Cheng, R. et al. Residual deep attention mechanism and adaptive reconstruction network for single image super-resolution. Appl Intell 52, 5197–5211 (2022). https://doi.org/10.1007/s10489-021-02568-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02568-z

Keywords

Search

Navigation