Skip to main content
SpringerLink
Log in

Superresolution reconstruction of optical remote sensing images based on a multiscale attention adversarial network

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Due to the influence of imaging equipment and environmental conditions on optical remote sensing image acquisition, image resolution is generally low. Superresolution reconstruction technology is an important way to improve image quality. However, the existing optical remote sensing image superresolution reconstruction methods have some problems, such as insufficient feature extraction, blurred texture details of reconstructed images, and excessive network accumulation. To solve the above problems, a superresolution reconstruction method for optical remote sensing images based on a multiscale attention adversarial network is proposed in this paper. The method takes a generative adversarial network (GAN) as the basic framework. The generator uses four multiscale attention residual blocks (MSARBs) to extract image multiscale feature information and carries out feature fusion through a binary feature fusion structure (BFFS) to generate more realistic images. The discriminator uses a depth convolution network to distinguish the differences between real images and superresolution images. In the aspect of loss function construction, the perceptual loss and adversarial loss are combined to improve the perceptual quality of the images. Experimental results show that this method is superior to the compared algorithm in regard to the objective evaluation metrics of peak signal-to-noise ratio (PSNR) and structural similarity (SSIM), and its reconstructed images have better visual effect.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  1. Tian Y, Jia RS, Xu SH, Hua R, Deng MD (2019) Super-resolution reconstruction of remote sensing images based on convolutional neural network. J Appl Remote Sens 13(4):046502

    Article  Google Scholar 

  2. Zhang N, Wang Y, Zhang X, Xu D, Wang X (2020) An unsupervised remote sensing single-image super-resolution method based on generative adversarial network. IEEE Access 8:29027–29039

    Article  Google Scholar 

  3. He K, Zhang X, Ren S, Sun J (2016). Deep residual learning for image recognition. In proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778)

  4. Bätz M, Eichenseer A, Seiler J, Jonscher M, Kaup A (2015). Hybrid super-resolution combining example-based single-image and interpolation-based multi-image reconstruction approaches. In 2015 IEEE international conference on image processing (ICIP) (pp. 58-62). IEEE

  5. Zhang K, Zuo W, Zhang L (2018). Learning a single convolutional super-resolution network for multiple degradations. In proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3262-3271)

  6. Timofte R, De Smet V, Van Gool L (2013). Anchored neighborhood regression for fast example-based super-resolution. In proceedings of the IEEE international conference on computer vision (pp. 1920-1927). 1927 (2013)

  7. Irani M, Peleg S (1991) Improving resolution by image registration. CVGIP: Graphical models and image processing 53(3):231–239

    Google Scholar 

  8. Stark H, Oskoui P (1989) High-resolution image recovery from image-plane arrays, using convex projections. JOSA A 6(11):1715–1726

    Article  Google Scholar 

  9. Schultz RR, Stevenson RL (1996) Extraction of high-resolution frames from video sequences. IEEE Trans Image Process 5(6):996–1011

    Article  Google Scholar 

  10. Gao X, Zhang K, Tao D, Li X (2012) Image super-resolution with sparse neighbor embedding. IEEE Trans Image Process 21(7):3194–3205

    Article  MathSciNet  MATH  Google Scholar 

  11. Yang J, Wright J, Huang TS, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873

    Article  MathSciNet  MATH  Google Scholar 

  12. Dong C, Loy CC, He K, Tang X (2014). Learning a deep convolutional network for image super-resolution. In European conference on computer vision (pp. 184-199). Springer, Cham

  13. Dong C, Loy CC, He K, Tang X (2015) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307

    Article  Google Scholar 

  14. Shi W, Caballero J, Huszár F, Totz J, Aitken AP, Bishop R, ... Wang Z (2016). Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1874-1883)

  15. Kim J, Lee JK, Lee KM (2016). Accurate image super-resolution using very deep convolutional networks. In proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1646-1654)

  16. Huang N, Yang Y, Liu J, Gu X, Cai H (2017). Single-image super-resolution for remote sensing data using deep residual-learning neural network. In international conference on neural information processing (pp. 622-630). Springer, Cham

  17. Lei S, Shi Z, Zou Z (2017) Super-resolution for remote sensing images via local–global combined network. IEEE Geosci Remote Sens Lett 14(8):1243–1247

    Article  Google Scholar 

  18. Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, ... Shi W (2017). Photo-realistic single image super-resolution using a generative adversarial network. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4681–4690)

  19. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, ... Bengio Y (2014). Generative adversarial nets. Adv Neural Inf Proces Syst, 27

  20. Lim B, Son S, Kim H, Nah S, Mu Lee K (2017). Enhanced deep residual networks for single image super-resolution. In proceedings of the IEEE conference on computer vision and pattern recognition workshops (pp. 136-144)

  21. Li J, Fang F, Mei K, Zhang G (2018). Multi-scale residual network for image super-resolution. In proceedings of the European conference on computer vision (ECCV) (pp. 517-532)

  22. Jiang K, Wang Z, Yi P, Jiang J, Xiao J, Yao Y (2018) Deep distillation recursive network for remote sensing imagery super-resolution. Remote Sens 10(11):1700

    Article  Google Scholar 

  23. Wang X, Yu K, Wu S, Gu J, Liu Y, Dong C, ... Change Loy C (2018). Esrgan: Enhanced super-resolution generative adversarial networks. In Proceedings of the European conference on computer vision (ECCV) workshops (pp. 0–0)

  24. Gao S, Cheng MM, Zhao K, Zhang XY, Yang MH, Torr PH (2019). Res2net: a new multi-scale backbone architecture. IEEE Trans Pattern Anal Mach Intell

  25. Gu J, Sun X, Zhang Y, Fu K, Wang L (2019) Deep residual squeeze and excitation network for remote sensing image super-resolution. Remote Sens 11(15):1817

    Article  Google Scholar 

  26. Hu J, Shen L, Sun G (2018). Squeeze-and-excitation networks. In proceedings of the IEEE conference on computer vision and pattern recognition (pp. 7132-7141)

  27. Jia Z, Ng MK, Song GJ (2019) Robust quaternion matrix completion with applications to image inpainting. Numerical Linear Algebra with Applications 26(4):e2245

    Article  MathSciNet  MATH  Google Scholar 

  28. Zhang S, Yuan Q, Li J, Sun J, Zhang X (2020) Scene-adaptive remote sensing image super-resolution using a multiscale attention network. IEEE Trans Geosci Remote Sens 58(7):4764–4779

    Article  Google Scholar 

  29. Ran Q, Xu X, Zhao S, Li W, Du Q (2020) Remote sensing images super-resolution with deep convolution networks. Multimed Tools Appl 79(13):8985–9001

    Article  Google Scholar 

  30. Wang X, Wu Y, Ming Y, Lv H (2020) Remote sensing imagery super resolution based on adaptive multi-scale feature fusion network. Sensors 20(4):1142

    Article  Google Scholar 

  31. Dong R, Zhang L, Fu H (2021) RRSGAN: reference-based super-resolution for remote sensing image. IEEE Trans Geosci Remote Sens 59:1618–1633

    Article  Google Scholar 

  32. Huan H, Li P, Zou N, Wang C, Xie Y, Xie Y, Xu D (2021) End-to-end super-resolution for remote-sensing images using an improved multi-scale residual network. Remote Sens 13(4):666

    Article  Google Scholar 

  33. Pang S, Chen Z, Yin F (2021). Lightweight multi-scale aggregated residual attention networks for image super-resolution. Multimedia tools and applications, 1-23

  34. Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y (2018). Image super-resolution using very deep residual channel attention networks. In proceedings of the European conference on computer vision (ECCV) (pp. 286-301)

  35. Muqeet A, Iqbal MTB, Bae SH (2019) HRAN: hybrid residual attention network for single image super-resolution. IEEE Access 7:137020–137029

    Article  Google Scholar 

Download references

Acknowledgements

The authors are grateful for collaborative funding support from the Humanity and Social Science Foundation of Ministry of Education, China (21YJAZH077).

Author information

Authors and Affiliations

  1. College of Computer Science and Engineering, Shandong University of Science and Technology, Qingdao, 266590, China

    Qi Zhang, Rui-Sheng Jia, Zeng-Hu Li, Yong-Chao Li & Hong-Mei Sun

  2. Shandong Province Key Laboratory of Wisdom Mine Information Technology, Shandong University of Science and Technology, Qingdao, 266590, China

    Rui-Sheng Jia & Hong-Mei Sun

Authors
  1. Qi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rui-Sheng Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zeng-Hu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yong-Chao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hong-Mei Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Rui-Sheng Jia or Hong-Mei Sun.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Q., Jia, RS., Li, ZH. et al. Superresolution reconstruction of optical remote sensing images based on a multiscale attention adversarial network. Appl Intell 52, 17896–17911 (2022). https://doi.org/10.1007/s10489-022-03548-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03548-7

Keywords

Search

Navigation