Skip to main content
SpringerLink
Log in

Remote sensing image reconstruction using an asymmetric multi-scale super-resolution network

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

High-resolution (HR) remote sensing images demonstrate detailed geographical information; however, some remote sensing satellites are incapable of offering HR remote sensing images because of the restriction of hardware resources. The single-image super-resolution (SISR) reconstruction technique is considered as an important method to improve the resolution of remote sensing images; however, most super-resolution (SR) image reconstruction methods are not able to sufficiently extract and utilize image features, and there is much redundancy of network parameters. To address this problem, an asymmetric multi-scale super-resolution network (AMSSRN) is proposed. In this network, a residual multi-scale block (RMSB) and a residual multi-scale dilation block (RMSDB) are designed to extract both shallow and deep features from images. This asymmetric structure enables the deep features to be fully extracted while reducing the redundancy of the modules used to extract shallow features. In this work, a feature-refinement fusion (FRF) module is also established, which can make full use of extracted features to improve network performance. Experimental results indicate that compared with enhanced deep super-resolution (EDSR) network, the proposed AMSSRN can efficiently reduce the redundancy of network parameters and enhance the feature extraction capability: the maximum increases in peak signal-to-noise ratio (PSNR) and the structural similarity (SSIM) index can reach 0.23 dB and 0.9782, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Availability of data and materials

The data presented in this study are available on request from the corresponding author.

Abbreviations

HR:

High resolution

SISR:

Single-image super-resolution

SR:

Super-resolution

AMSSRN:

Asymmetric multi-scale super-resolution network

RMSB:

Residual multi-scale block

RMSDB:

Residual multi-scale dilation block

FRF:

Feature-refinement fusion

EDSR:

Enhanced deep super-resolution

PSNR:

Peak signal-to-noise ratio

SSIM:

Structural similarity

LR:

Low resolution

CNN:

Convolutional neural network

SRCNN:

Super-resolution convolutional neural network

FSRCNN:

Fast super-resolution convolutional neural network

ESPCN:

Efficient sub-pixel convolutional network

VDSR:

Very deep super-resolution

DRCN:

Deeply recursive convolutional network

DRRN:

Deep recursive residual network

SRDenseNet:

Super-resolution dense network

LapSRN:

Laplacian super-resolution network

MSRN:

Multi-scale residual network

IMDN:

Information multi-distillation network

RFDN:

Residual feature distillation network

SMSR:

Sparse masks super-resolution

MPSR:

Multi-perception super-resolution

URNet:

U-shaped residual network

Bicubic:

Bicubic interpolation

RL-NL:

Region-level non-local

MBRB:

Multi-branch residual block

PReLU:

Parametric rectified linear unit

MSE:

Mean square error

DIV2K:

Diverse 2K

AID:

Aerial image dataset

References

  1. Muhammad W, Bhutto Z, Ansari A, Memon ML, Kumar R, Hussain A, Shah SAR, Thaheem I, Ali S (2021) Multi-path deep CNN with residual inception network for single image super-resolution. Electronics 10(16):1979. https://doi.org/10.3390/electronics10161979

    Article  Google Scholar 

  2. Chen Y, Qin K, Gan S, Wu T (2014) Structural feature modeling of high-resolution remote sensing images using directional spatial correlation. IEEE Geosci Remote Sens Lett 11(10):1727–1731. https://doi.org/10.1109/LGRS.2014.2306972

    Article  Google Scholar 

  3. Huang B, He B, Wu L, Guo Z (2021) Deep residual dual-attention network for super-resolution reconstruction of remote sensing images. Remote Sens 13(14):2784. https://doi.org/10.3390/rs13142784

    Article  Google Scholar 

  4. Pan Z, Ma W, Guo J, Lei B (2019) Super-resolution of single remote sensing image based on residual dense backprojection networks. IEEE Trans Geosci Remote Sens 57(10):7918–7933. https://doi.org/10.1109/TGRS.2019.2917427

    Article  Google Scholar 

  5. Ma Y, Lv P, Liu H, Sun X, Zhong Y (2021) Remote sensing image super resolution based on dense channel attention network. Remote Sens 13(15):2966. https://doi.org/10.3390/rs13152966

    Article  Google Scholar 

  6. Liu X, Li J, Lu G (2020) A new volumetric fusion strategy with adaptive weight field for RGB-D reconstruction. Sensors 20(15):4330. https://doi.org/10.3390/s20154330

    Article  Google Scholar 

  7. Yang J, Wright J, Huang TS, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873. https://doi.org/10.1109/TIP.2010.2050625

    Article  MathSciNet  MATH  Google Scholar 

  8. Martín D, Fuentes-Lorenzo D, Bordel B, Alcarria R (2020) Towards outlier sensor detection in ambient intelligent platforms-a low-complexity statistical approach. Sensors 20(15):4217. https://doi.org/10.3390/s20154217

    Article  Google Scholar 

  9. Dong C, Loy CC, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. In: European Conference on Computer Vision, ECCV 2014, Zurich, Switzerland, 6–12 Sept, pp 184–199. https://doi.org/10.1007/978-3-319-10593-2_13

  10. Dong C, Loy CC, Tang X (2016) Accelerating the super-resolution con volutional neural network. In: European Conference on Computer Vision, ECCV 2016, Glasgow, UK, 8–16 Oct, pp 391–407. https://doi.org/10.1007/978-3-319-46475-6_25

  11. Shi W, Caballero J, Huszár F, Totz J, Aitken AP, Bishop R, Rueckert D, Wang Z (2016) Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, 27–30 June, pp 1874–1883. https://doi.org/10.1109/CVPR.2016.207

  12. Kim J, Lee JK, Lee KM (2016) Accurate image super-resolution using very deep convolutional networks. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, 27–30 June, pp 1646–1654. https://doi.org/10.1109/CVPR.2016.182

  13. Kim J, Lee JK, Lee KM (2016) Deeply-recursive convolutional network for image super-resolution. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, 27–30 June, pp 1637–1645. https://doi.org/10.1109/CVPR.2016.181

  14. Tai Y, Yang J, Liu X (2017) Image super-resolution via deep recursive residual network. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, 21–26 July, pp 2790–2798. https://doi.org/10.1109/CVPR.2017.298

  15. Tong T, Li G, Liu X, Gao Q (2017) Image super-resolution using dense skip connections. In: IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, 22–29 Oct, pp 4809–4817. https://doi.org/10.1109/ICCV.2017.514

  16. Lai WS, Huang JB, Ahuja N, Yang MH (2017) Deep Laplacian pyramid networks for fast and accurate super-resolution. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, 21–26 July, pp 5835–5843. https://doi.org/10.1109/CVPR.2017.618

  17. Lim B, Son S, Kim H, Nah S, Lee KM (2017) Enhanced deep resid ual networks for single image super-resolution. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2017, Honolulu, HI, USA, 21–26 July, pp 1132–1140. https://doi.org/10.1109/CVPRW.2017.151

  18. Li J, Fang F, Mei K, Zhang G (2018) Multi-scale residual network for image super-resolution. In: European Conference on Computer Vision, ECCV 2018, Munich, Germany, 8–14 Sept, pp 10. https://doi.org/10.1007/978-3-030-01237-3_32

  19. Hui Z, Gao X, Yang Y, Wang X (2019) Lightweight image super resolution with information multi-distillation network. In: Association for Computing Machinery International Conference on Multimedia, ACMM 2019, Nice, France, 21–25 Oct, pp 2024–2032. https://doi.org/10.1145/3343031.3351084

  20. Liu J, Tang J, Wu G (2020) Residual feature distillation network for lightweight image super-resolution. CoRR. https://arxiv.org/abs/2009.11551

  21. Wang L, Dong X, Wang Y, Ying X, Lin Z, An W, Guo Y (2021) Exploring sparsity in image super-resolution for efficient inference. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, Nashville, TN, USA, 20–25 June, pp 4915–4924. https://doi.org/10.1109/CVPR46437.2021.00488

  22. Dong X, Xi Z, Sun X, Gao L (2019) Transferred multi-perception attention networks for remote sensing image super-resolution. Remote Sens 11(23):2857. https://doi.org/10.3390/rs11232857

    Article  Google Scholar 

  23. Wang Y, Zhao L, Liu L, Hu H, Tao W (2021) URNet: a U-shaped residual network for lightweight image super-resolution. Remote Sens 13(19):3848. https://doi.org/10.3390/rs13193848

    Article  Google Scholar 

  24. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612. https://doi.org/10.1109/TIP.2003.819861

    Article  Google Scholar 

  25. Wu H, Qu Y, Lin S, Zhou J, Qiao R, Zhang Z, Xie Y, Ma L (2021) Contrastive learning for compact single image dehazing. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, Nashville, TN, USA, 20–25 June, pp 10546–10555. https://doi.org/10.1109/CVPR46437.2021.01041

  26. Haris M, Shakhnarovich G, Ukita N (2021) Deep back-projectinetworks for single image super-resolution. IEEE Trans Pattern Anal Mach Intell 43(12):4323–4337. https://doi.org/10.1109/TPAMI.2020.3002836

    Article  Google Scholar 

  27. Ioe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, ICML 2015, Lille, France, 6–11 July, pp 448–456. https://arxiv.org/abs/1502.03167

  28. Johnson J, Alahi A, Li FF (2016) Perceptual losses for real-time style transfer and super-resolution. In: European Conference on Computer Vision, ECCV 2016, Glasgow, UK, 8–16 Oct, pp 694–711. https://doi.org/10.1007/978-3-319-46475-6_43

  29. Sajjadi MSM, Schölkopf B, Hirsch M (2017) EnhanceNet: single image super-resolution through automated texture synthesis. In: IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, 22–29 Oct, pp 4501–4510. https://doi.org/10.1109/ICCV.2017.481

  30. Dai T, Cai J, Zhang Y, Xia ST, Zhang L (2019) Second-order attention network for single image super-resolution. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, 16–20 June, pp 11057–11066. https://doi.org/10.1109/CVPR.2019.01132

  31. Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y (2018) Image super-resolution using very deep residual channel attention networks. In: European Conference on Computer Vision, ECCV 2018, Munich, Germany, 8–14 Sept, pp 294–310. https://doi.org/10.1007/978-3-030-01234-2_18

  32. Agustsson E, Timofte R (2017) Ntire 2017 challenge on single image super-resolution: dataset and study. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2017, Honolulu, HI, USA, 21–26 July, pp 1122–1131. https://doi.org/10.1109/CVPRW.2017.149

  33. Bevilacqua M, Roumy A, Guillemot C, Alberi Morel Ml (2012) Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: British Machine Vision Conference, BMVC 2012, Surrey, UK, 3–7 Sept, pp 135.1–135.10. https://doi.org/10.5244/C.26.135

  34. Zeyde R, Elad M, Protter M (2010) On single image scale-up using sparse-representations. In: International Conference on Curves and Sur faces, Curves and Surfaces 2010, Avignon, France, 24–30 June, pp 711–730. https://doi.org/10.1007/978-3-642-27413-8_47

  35. Arbeláez P, Maire M, Fowlkes C, Malik J (2011) Contour detection and hierarchical image segmentation. IEEE Trans Pattern Anal Mach Intell 33(5):898–916. https://doi.org/10.1109/TPAMI.2010.161

    Article  Google Scholar 

  36. Huang JB, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, 7–12 June, pp 5197–5206. https://doi.org/10.1109/CVPR.2015.7299156

  37. Matsui Y, Ito K, Aramaki Y, Fujimoto A, Ogawa T, Yamasaki T, Aizawa K (2017) Sketch-based manga retrieval using manga109 dataset. Multimedia Tools Appl 76(20):21811–21838. https://doi.org/10.1007/s11042-016-4020-z

    Article  Google Scholar 

  38. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. CoRR. https://arxiv.org/abs/1412.6980

Download references

Funding

This research was funded by the NATIONAL NATURAL SCIENCE FOUNDATION OF CHINA, grant number 62001238.

Author information

Authors and Affiliations

  1. School of Artificial Intelligence, Nanjing University of Information Science and Technology, Nanjing, 210044, China

    Hai Huan

  2. School of Electronic and Information Engineering, Nanjing University of Information Science and Technology, Nanjing, 210044, China

    Nan Zou, Yaqin Xie & Chao Wang

  3. Jiangsu Key Laboratory of Meteorological Observation and Information Processing, Nanjing University of Information Science and Technology, Nanjing, 210044, China

    Chao Wang

  4. Nanjing Nanji Agricultural Machinery Science and Technology Research Institute, Nanjing, 210019, China

    Yi Zhang

Authors
  1. Hai Huan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nan Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yaqin Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

H.H. took part in conceptualization; H.H. and N.Z. involved in methodology; N.Z. took part in software; N.Z. took part in validation; N.Z. involved in formal analysis; H.H. and N.Z. took part in investigation; H.H., C.W., Y.Z. involved in resources; N.Z. involved in writing—original draft preparation; H.H., N.Z., C.W., and Y.X. took part in writing—review and editing; H.H. and N.Z. involved in visualization; H.H., C.W., and Y.X. took part in supervision; H.H. involved in project administration. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Hai Huan.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huan, H., Zou, N., Zhang, Y. et al. Remote sensing image reconstruction using an asymmetric multi-scale super-resolution network. J Supercomput 78, 18524–18550 (2022). https://doi.org/10.1007/s11227-022-04617-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-022-04617-x

Keywords

Search

Navigation