Abstract
Deep convolutional neural networks (CNNs) have enabled significant progress in the single image super-resolution (SISR) field in recent years. However, most super-resolution methods based on deep neural networks obtain spatial information using a single-size convolution kernel, which leads to insufficient local feature extraction. In addition, since convolution serves for local operations, it fails to capture the image’s inherent attributes. This paper tackles these problems by proposing a novel multi-scale dense attention network (MSDAN) based on the attention mechanism and multi-scale feature extraction. The network employs multi-scale dense blocks (MSDB) that utilize convolution kernels of different sizes to extract feature information of distinct scales. Additionally, the attention is used to model global features effectively and strengthen the network’s expressiveness. Hierarchical feature fusion is followed by multi-level feature extraction, which fuses each unit’s feature map output. Finally, the image is reconstructed using up-sampling. The experimental results show that the proposed network outperforms other several state-of-the-art super-resolution reconstruction algorithms regarding objective evaluation indicators and visual effects on datasets Set5, Set14, BSD100 and Urban100.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
Enquiries about data availability should be directed to the authors.
References
Ahn N, Kang B, Sohn K-A (2018) Fast, accurate, and lightweight super-resolution with cascading residual network. In: Proceedings of the European conference on computer vision (ECCV), pp 252–268
Arbelaez P, Maire M, Fowlkes C, Malik J (2010) Contour detection and hierarchical image segmentation. IEEE Trans Pattern Anal Mach Intell 33(5):898–916
Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473
Bevilacqua M, Roumy A, Guillemot C, Alberi-Morel ML (2012) Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: British machine vision conference. BMVA Press, Guildford, pp 1–10
Castro EB, Nakano M, Perez GS, Meana HP (2017) Improvement of image super-resolution algorithms using iterative back projection. IEEE Lat Am Trans 15(11):2214–2219
Chu J, Li X, Zhang J, Lu W (2020) Super-resolution using multi-channel merged convolutional network. Neurocomputing 394:136–145
Ding L, Xu W, Chen Y (2020) Density peaks clustering by zero-pointed samples of regional group borders. Comput Intell Neurosci 2020(2):1–15
Dong C, Loy CC, He K, Tang X (2015) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307
Dong C, Loy CC, Tang X (2016) Accelerating the super-resolution convolutional neural network. In: European conference on computer vision, pp 391–407
Emami H, Aliabadi MM, Dong M, Chinnam RB (2019) SPA-GAN: spatial attention GAN for image-to-image translation. arXiv preprint arXiv:1908.06616
Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141
Huang J-B, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5197–5206
Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017a) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708
Huang Y, Shao L, Frangi AF (2017b) Simultaneous super-resolution and cross-modality synthesis of 3D medical images using weakly-supervised joint convolutional sparse coding. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6070–6079
Jiang K, Wang Z, Yi P, Jiang J, Xiao J, Yao Y (2018) Deep distillation recursive network for remote sensing imagery super-resolution. Remote Sens 10(11):1700
Kim J, Lee JK, Lee KM (2016) Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1646–1654
Kuen J, Wang Z, Wang G (2016) Recurrent attentional networks for saliency detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3668–3677
Lan R, Sun L, Liu Z, Lu H, Su Z, Pang C et al (2020) Cascading and enhanced residual networks for accurate single-image super-resolution. IEEE Trans Cybern 51(1):115–125
Li J, Fang F, Mei K, Zhang G (2018) Multi-scale residual network for image super-resolution. In: Proceedings of the European conference on computer vision (ECCV), pp 517–532
Li Z, Yang J, Liu Z, Yang X, Jeon G, Wu W (2019) Feedback network for image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3867–3876
Lim B, Son S, Kim H, Nah S, Lee KM (2017) Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 136–144
Luo Y, Qin J, Xiang X, Tan Y, Liu Q, Xiang L (2020) Coverless real-time image information hiding based on image block matching and dense convolutional network. J Real-Time Image Proc 17(1):125–135
Qin D, Gu X (2020) Single-image super-resolution with multilevel residual attention network. Neural Comput Appl 32(19):15615–15628
Shi W, Caballero J, Huszár F, Totz J, Aitken AP, Bishop R et al (2016) Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1874–1883
Tai Y, Yang J, Liu X (2017) Image super-resolution via deep recursive residual network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3147–3155
Timofte R, Agustsson E, Van Gool L, Yang M-H, Zhang L (2017) Ntire 2017 challenge on single image super-resolution: methods and results. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 114–125
Tong T, Li G, Liu X, & Gao Q. (2017). Image super-resolution using dense skip connections. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4799–4807.
Wang F, Jiang M, Qian C, Yang S, Li C, Zhang H, et al. (2017). Residual attention network for image classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3156–3164
Wang X, Girshick R, Gupta A, He K (2018) Non-local neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7794–7803
Wang W, Zhu H, Dai J, Pang Y, Shen J, Shao L (2020) Hierarchical human parsing with typed part-relation reasoning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8929–8939
Woo S, Park J, Lee J-Y, Kweon IS (2018) Cbam: convolutional block attention module. In: Proceedings of the European conference on computer vision (ECCV), pp 3–19
Wu Y, Zhang Y, Huang Y, Yang J (2019) Fast deconvolution method for angular super-resolution imaging based on sub-space embedding. J Eng 2019(19):6291–6294
Yan Y, Ren W, Hu X, Li K, Shen H, Cao X (2021) Srgat: single image super-resolution with graph attention network. IEEE Trans Image Process 30:4905–4918
Yang X, Li X, Li Z, Zhou D (2021) Image super-resolution based on deep neural network of multiple attention mechanism. J Vis Commun Image Represent 75:103019
Zeyde R, Elad M, Protter M (2010) On single image scale-up using sparse-representations. In: International conference on curves and surfaces, pp 711–730
Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y (2018a) Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European conference on computer vision (ECCV), pp 286–301
Zhang Y, Tian Y, Kong Y, Zhong B, Fu Y (2018b) Residual dense network for image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2472–2481
Zhang H, Goodfellow I, Metaxas D, Odena A (2019) Self-attention generative adversarial networks. In: International conference on machine learning, pp 7354–7363
Zhou F, Yang W, Liao Q (2012) Interpolation-based image super-resolution using multisurface fitting. IEEE Trans Image Process 21(7):3312–3318
Zou WW, Yuen PC (2011) Very low resolution face recognition problem. IEEE Trans Image Process 21(1):327–340
Funding
This work was supported by the Zhejiang Provincial Natural Science Foundation of China under Grant LY20E050011, and the National Natural Science Foundation of China under Grant 62071161.
Author information
Authors and Affiliations
Contributions
All of the authors contribute to this manuscript, the detail contributions are as follows. FG and YW—conceptualization, methodology, writing, reviewing and editing draft; ZY and QZ—methodology, investigation, supervision; YW and YM—visualization, investigation, computational experiments, analyzing and interpretation of data.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Informed consent
All authors have read this manuscript and are willing to process it for publication.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Gao, F., Wang, Y., Yang, Z. et al. Single image super-resolution based on multi-scale dense attention network. Soft Comput 27, 2981–2992 (2023). https://doi.org/10.1007/s00500-022-07456-3
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-022-07456-3