Skip to main content
SpringerLink
Log in

Dynamic dual attention iterative network for image super-resolution

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Recently, deep convolution neural networks (DCNNs) have obtained remarkable performance in exploring single image super-resolution (SISR). However, most of the existing CNN-based SISR methods only focus on increasing the width and depth of the network to improve SR performance, which makes them face a heavy computing burden. In this paper, we propose a lightweight dynamic dual attention iteration network (DDAIN) for SISR. Specifically, to better realize the attention of the channel and the convolution kernel, we design a dynamic convolution unit (DYCU) at the head of the network. It improves the SR performance by enhancing the complexity of the model without increasing the width and depth of the network. Compared with the traditional static convolution, it can extract more abundant high and low-frequency image features according to different input images. Moreover, to recover the high-frequency detail features of images with different resolutions as much as possible, we embed multiple dual residual attention (DRA) in the feature refinement unit (FRU). Finally, to alleviate the height discomfort caused by SR, we introduce iterative loss Liter to optimize the training process further. Extensive experimental results on benchmark show that the performance of the DDAIN in different degradation models exceeds some existing classical methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  1. Freeman WT, Pasztor EC, Carmichael OT (2000) Learning low-level vision. Int J Comput Vis 40(1):25–47

    Article  Google Scholar 

  2. Webb S, Flower M (2012) Webb’s physics of medical imaging. Med Phys 40(9):097301

  3. Kon F, Román M, Ping L, Mao J, Campbell RH (2000) Monitoring, security, and dynamic configuration with the dynamictao reflective orb. In: Ifip/acm International Conference on Distributed Systems Platforms

  4. Zhang L, Xiaolin W (2006) An edge-guided image interpolation algorithm via directional filtering and data fusion. IEEE Trans Image Process 15(8):2226–2238

    Article  Google Scholar 

  5. Dong W, Zhang L, Shi G, Xiaolin W (2011) Image deblurring and super-resolution by adaptive sparse domain selection and adaptive regularization. IEEE Trans Image Process 20(7):1838–1857

    Article  MathSciNet  Google Scholar 

  6. Wang S, Zhang L, Liang Y, Pan Q (2012) Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp 2216–2223

  7. Zhang Y, Tian Y, Kong Y, Zhong B, Yun F (2018) Residual dense network for image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2472–2481

  8. Zhang K, Gao X, Tao D, Li X (2012) Single image super-resolution with non-local means and steering kernel regression. IEEE Trans Image Process 21(11):4544–4556

    Article  MathSciNet  Google Scholar 

  9. Dong C, Loy CC, He K, Tang X (2015) Image super-resolution using deep convolutional networks, vol 38

  10. Kim J, Lee JK, Lee KM (2016) Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

  11. Lai W-S, Huang J-B, Ahuja N, Yang M-H (2017) Deep laplacian pyramid networks for fast and accurate super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 624–632

  12. Zhang K, Zuo W, Zhang L (2018) Learning a single convolutional super-resolution network for multiple degradations. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3262–3271

  13. Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y (2018) Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 286–301

  14. Kim J, Lee JK, Lee KM (2016) Deeply-recursive convolutional network for image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1637–1645

  15. Haris M, Shakhnarovich G, Ukita N (2018) Deep back-projection networks for super-resolution. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp 1664–1673

  16. Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z et al (2017) Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern recognition, pp 4681–4690

  17. Tai Y, Yang J, Liu X (2017) Image super-resolution via deep recursive residual network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3147–3155

  18. Tai Y, Yang J, Liu X, Xu C (2017) Memnet: A persistent memory network for image restoration. In: Proceedings of the IEEE international Conference on Computer Vision

  19. Li Z, Yang J, Liu Z, Yang X, Jeon G, Wu W (2019) Feedback network for image super-resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 3867–3876

  20. Guo Y, Chen J, Wang J, Qi C, Cao J, Deng Z, Xu Y, Tan M (2020) Closed-loop matters: Dual regression networks for single image super-resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 5407–5416

  21. Lim B, Son S, Kim H, Nah S, Lee KM (2017) Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern recognition workshops, pp 136–144

  22. Dai T, Cai J, Zhang Y, Xia S-T, Zhang L (2019) Second-order attention network for single image super-resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 11065–11074

  23. Tang Y, Huang J, Zhang F, Gong W (2020) Deep residual networks with a fully connected reconstruction layer for single image super-resolution. Neurocomputing 405:186–199

    Article  Google Scholar 

  24. Liu J, Zhang W, Tang Y, Tang J, Wu G (2020) Residual feature aggregation network for image super-resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 2359–2368

  25. Huang G, Chen H, Li T, Wu F, van der Maaten L, Weinberger KQ (2018) Multi-scale dense networks for resource efficient image classification. In: 6th International Conference on Learning Representations, ICLR 2018. Conference track proceedings. OpenReview.net, Vancouver

  26. Ji L, Rao Y, Lu J, Zhou J (2017) Runtime neural pruning. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, pp 2178–2188

  27. Liu L, Deng J (2018) Dynamic deep neural networks: Optimizing accuracy-efficiency trade-offs by selective execution. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 32

  28. Yu J, Yang L, Xu N, Yang J, Huang TS (2019) Slimmable neural networks. In: 7th International Conference on Learning Representations, ICLR 2019. OpenReview.net, New orleans

  29. Chen Y, Dai X, Liu M, Chen D, Yuan L, Liu Z (2020) Dynamic relu. In European Conference on Computer Vision. Springer, pp 351–367

  30. Kong T, Yao A, Chen Y, Sun F (2016) Hypernet: Towards accurate region proposal generation and joint object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 845–853

  31. Hu J, Li S, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 7132–7141

  32. Chen Y, Dai X, Liu M, Chen D, Lu Y, Liu Z (2020) Dynamic convolution: Attention over convolution kernels. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 11030–11039

  33. Ma N, Zhang X, Huang J, Sun J (2020) Weightnet: Revisiting the design space of weight networks. In: Vedaldi A, Bischof H, Brox T, Frahm J-M (eds) Computer Vision - ECCV 2020 - 16th European Conference, Proceedings, Part XV, volume 12360 of Lecture Notes in Computer Science. Springer, Glasgow, pp 776–792

  34. Wang X, Girshick R, Gupta A, He K (2018) Non-local neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 7794–7803

  35. Fu J, Liu J, Tian H, Li Y, Bao Y, Fang Z, Lu H (2019) Dual attention network for scene segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 3146–3154

  36. Zhang Q-L, Yang Y-B (2021) Sa-net: Shuffle attention for deep convolutional neural networks. In: ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, pp 2235–2239

  37. He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE International Conference on Computer Vision, pp 1026–1034

  38. Agustsson E, Timofte R (2017) Ntire 2017 challenge on single image super-resolution Dataset and study. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp 126–135

  39. Bevilacqua M, Roumy A, Guillemot C, Alberi-Morel ML (2012) Low-complexity single-image super-resolution based on nonnegative neighbor embedding

  40. Zeyde R, Elad M, Protter M (2010) On single image scale-up using sparse-representations. In: International conference on curves and surfaces. Springer, pp 711–730

  41. Martin D, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001, vol 2. IEEE, pp 416–423

  42. Huang J-B, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5197–5206

  43. Matsui Y, Ito K, Aramaki Y, Fujimoto A, Ogawa T, Yamasaki T, Aizawa K (2017) Sketch-based manga retrieval using manga109 dataset. Multimed Tools Appl 76(20):21811–21838

    Article  Google Scholar 

  44. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  45. Kingma DP, Ba J (2015) Adam: A method for stochastic optimization. In: Bengio Y, LeCun Y (eds) 3rd International Conference on Learning Representations, ICLR 2015. Conference Track Proceedings, San Diego

  46. Tong T, Li G, Liu X, Gao Q (2017) Image super-resolution using dense skip connections. In: Proceedings of the IEEE International Conference on Computer Vision, pp 4799–4807

  47. Zhang K, Zuo W, Gu S, Zhang L (2017) Learning deep cnn denoiser prior for image restoration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3929–3938

Download references

Acknowledgements

This research was funded in part by the Natural Science Foundation of Xinjiang Uygur Autonomous Region grant number 2020D01C034, Tianshan Innovation Team of Xinjiang Uygur Autonomous Region grant number 2020D14044, the National Science Foundation of China under Grant U1903213, 61771416 and 62041110, the Creative Research Groups of Higher Education of Xinjiang Uygur Autonomous Region under Grant XJEDU2017T002.

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Xinjiang University, Urumqi, 830046, China

    Hao Feng, Liejun Wang, Shuli Cheng, Anyu Du & Yongming Li

Authors
  1. Hao Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Liejun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shuli Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Anyu Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yongming Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Liejun Wang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Feng, H., Wang, L., Cheng, S. et al. Dynamic dual attention iterative network for image super-resolution. Appl Intell 52, 8189–8208 (2022). https://doi.org/10.1007/s10489-021-02816-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02816-2

Keywords

Search

Navigation