Skip to main content

Advertisement

SpringerLink
Log in

Remote sensing images super-resolution with deep convolution networks

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Remote sensing image data have been widely applied in many applications, such as agriculture, military, and land use. It is difficult to obtain remote sensing images in both high spatial and spectral resolutions due to the limitation of implements in image acquisition and the law of energy conservation. Super-resolution (SR) is a technique to improve the resolution from a low-resolution (LR) to a high-resolution (HR). In this paper, a novel deep convolution network (DCN) SR method (SRDCN) is proposed. Based on hierarchical architectures, the proposed SRDCN learns an end-to-end mapping function to reconstruct an HR image from its LR version; furthermore, extensions of SRDCN based on residual learning and multi scale version are investigated for further improvement,namely Developed SRDCN(DSRDCN) and Extensive SRDCN(ESRDCN). Experimental results using different types of remote sensing data (e.g., multispectral and hyperspectral) demonstrate that the proposed methods outperform the traditional sparse representation based methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. http://www.pytorch.org

  2. http://weegee.vision.ucmerced.edu/datasets/landuse.html

References

  1. Anwer RM, Khan FS, van de Weijer J, Molinier M, Laaksonen J (2018) Binary patterns encoded convolutional neural networks for texture recognition and remote sensing scene classification. ISPRS J Photogramm Remote Sens 138:74–85

    Article  Google Scholar 

  2. Burger HC, Schuler CJ, Harmeling S (2012) Image denoising: Can plain neural networks compete with bm3d? In: 2012 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 2392–2399

  3. Candès EJ, et al (2006) Compressive sampling. In: Proceedings of the international congress of mathematicians, vol 3, Madrid, Spain, pp 1433–1452

  4. Chang H, Yeung D-Y, Xiong Y (2004) Super-resolution through neighbor embedding. In: 2004. CVPR 2004. Proceedings of the 2004 IEEE computer society conference on computer vision and pattern recognition, vol 1. IEEE, pp I–I

  5. Chen C, Fowler J (2012) Single-image super-resolution using multihypothesis prediction. In: Asilomar conference on signals, and computers, Pacific Grove, CA, pp 608–612

  6. Dong C, Loy CC, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. In: European conference on computer vision. Springer, pp 184–199

  7. Dong C, Loy CC, He K, Tang X (2016) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307

    Article  Google Scholar 

  8. Freedman G, Fattal R (2011) Image and video upscaling from local self-examples. ACM Trans Graph (TOG) 30(2):12

    Article  Google Scholar 

  9. Freeman WT, Pasztor EC, Carmichael OT (2000) Learning low-level vision. Int J Comput Vis 40(1):25–47

    Article  Google Scholar 

  10. Glasner D, Bagon S, Irani M (2009) Super-resolution from a single image. In: 2009 IEEE 12th international conference on computer vision. IEEE, pp 349–356

  11. Gou S, Liu S, Yang S, Jiao L (2014) Remote sensing image super-resolution reconstruction based on nonlocal pairwise dictionaries and double regularization. IEEE J Sel Top Appl Earth Observ Remote Sens 7(12):4784–4792

    Article  Google Scholar 

  12. He K, Zhang X, Ren S, Sun J (2015) Deep residual learning for image recognition, arXiv:1512.03385

  13. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  14. Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507

    Article  MathSciNet  Google Scholar 

  15. Jain V, Seung S (2009) Natural image denoising with convolutional networks. In: Advances in neural information processing systems, pp 769–776

  16. Kasetkasem T, Arora MK, Varshney PK (2005) Super-resolution land cover mapping using a markov random field based approach. Remote Sens Environ 96 (3-4):302–314

    Article  Google Scholar 

  17. Keys R (1981) Cubic convolution interpolation for digital image processing. IEEE Trans Acoust Speech Signal Process 29(6):1153–1160

    Article  MathSciNet  Google Scholar 

  18. Kim J, Kwon Lee J, Mu Lee K (2016) Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1646–1654

  19. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

  20. Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken AP, Tejani A, Totz J, Wang Z et al (2017) Photo-realistic single image super-resolution using a generative adversarial network. CVPR 2(3):4

    Google Scholar 

  21. Li F, Jia X, Fraser D, Lambert A (2010) Super resolution for remote sensing images based on a universal hidden markov tree model. IEEE Trans Geosci Remote Sens 48(3):1270–1278

    Article  Google Scholar 

  22. Li M, Nguyen TQ (2008) Markov random field model-based edge-directed image interpolation. IEEE Trans Image Process 17(7):1121–1128

    Article  MathSciNet  Google Scholar 

  23. Pan Z, Yu J, Huang H, Hu S, Zhang A, Ma H, Sun W (2013) Super-resolution based on compressive sensing and structural self-similarity for remote sensing images. IEEE Trans Geosci Remote Sens 51(9):4864–4876

    Article  Google Scholar 

  24. Pouliot D, Latifovic R, Pasher J, Duffe J (2018) Landsat super-resolution enhancement using convolution neural networks and sentinel-2 for training. Remote Sens 10(3):394

    Article  Google Scholar 

  25. Rhee S, Kang MG (1999) Discrete cosine transform based regularized high-resolution image reconstruction algorithm. Opt Eng 38(8):1348–1356

    Article  Google Scholar 

  26. Shen H, Zhang L, Huang B, Li P (2007) A map approach for joint motion estimation, segmentation, and super resolution. IEEE Trans Image Process 16 (2):479–490

    Article  MathSciNet  Google Scholar 

  27. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition, arXiv:1409.1556

  28. Sun W, Yang G, Du B, Zhang L, Zhang L (2017) A sparse and low-rank near-isometric linear embedding method for feature extraction in hyperspectral imagery classification. IEEE Trans Geosci Remote Sens 55(7):4032–4046

    Article  Google Scholar 

  29. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9

  30. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2818–2826

  31. Tsai R, Huang T (1984) Multiframe image restoration and registration. Adv Comput Vis Image Process 1(2):317–339

    Google Scholar 

  32. Wang L, Xiang S, Meng G, Wu H, Pan C (2013) Edge-directed single-image super-resolution via adaptive gradient magnitude self-interpolation. IEEE Trans Circ Syst Video Technol 23(8):1289–1299

    Article  Google Scholar 

  33. Wang Z, Liu D, Yang J, Han W, Huang T (2015) Deep networks for image super-resolution with sparse prior. In: Proceedings of the IEEE international conference on computer vision, pp 370–378

  34. Yang J, Wright J, Huang T, Ma Y (2008) Image super-resolution as sparse representation of raw image patches. In: 2008. CVPR 2008. IEEE conference on computer vision and pattern recognition. IEEE, pp 1–8

  35. Yuan Y, Zheng X, Lu X (2017) Hyperspectral image superresolution by transfer learning. IEEE J Sel Top Appl Earth Observ Remote Sens 10(5):1963–1974

    Article  Google Scholar 

  36. Zeng K, Ding S, Jia W (2018) Single image super-resolution using a polymorphic parallel cnn. Appl Intell 2018:1–9. https://doi.org/10.1007/s10489-018-1270-7

    Article  Google Scholar 

  37. Zhang H, Huang B (2011) Scale conversion of multi sensor remote sensing image using single frame super resolution technology. In: 2011 19th international conference on geoinformatics. IEEE, pp 1–5

  38. Zhang H, Zhang L, Shen H (2012) A super-resolution reconstruction algorithm for hyperspectral images. Signal Process 92(9):2082–2096

    Article  Google Scholar 

  39. Zhang Y, Wu W, Dai Y, Yang X, Yan B, Lu W (2013) Remote sensing images super-resolution based on sparse dictionaries and residual dictionaries. In: 2013 IEEE 11th international conference on dependable, autonomic and secure computing (DASC). IEEE, pp 318–323

Download references

Acknowledgments

This work was supported by the National Natural Science Foundation of China under Grant No. NSFC-61501017, No. NSFC-61571033, and partly by the Fundamental Research Funds for the Central Universities under Grants No. BUCTRC201401, BUCTRC201615, XK1521.

Author information

Authors and Affiliations

  1. College of Information Science and Technology, Beijing University of Chemical Technology, Beijing, 100029, China

    Qiong Ran, Xiaodong Xu, Shizhi Zhao & Wei Li

  2. Department of Electrical and Computer Engineering, Mississippi State University, Mississippi State, MS, 39762, USA

    Qian Du

Authors
  1. Qiong Ran
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaodong Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shizhi Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wei Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Qian Du
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qiong Ran.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ran, Q., Xu, X., Zhao, S. et al. Remote sensing images super-resolution with deep convolution networks. Multimed Tools Appl 79, 8985–9001 (2020). https://doi.org/10.1007/s11042-018-7091-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-7091-1

Keywords

Search

Navigation