Skip to main content
SpringerLink
Log in

Multi-scale Single Image Super-Resolution with Remote-Sensing Application Using Transferred Wide Residual Network

  • Published:
Wireless Personal Communications Aims and scope Submit manuscript

Abstract

Super-resolution (SR) has received extensive attention in recent years for satellite image processing in a wide range of application scenarios, such as land classification, identification of changes, the discovery of resources, etc. Satellite images from satellite sensors are mostly low-resolution (LR) images, so they do not completely fulfill object detection and analysis criteria. SR has multiple residual network frameworks in deep learning that have improved performance and can extend thousands of layers in the system. However, each layer improves accuracy by doubling the number of layers, although training thousands of layers are too expensive, the process is slow, and there are functional recovery issues. We proposed a transferred wide residual Single Image Super-Resolution (SISR) remote sensing deep neural network model (WRSR). By increasing the width and reducing the residual network depth, the proposed approach has dramatically reduced memory costs. As a result, our model reduced memory costs by 21% in Enhanced Deep Residual Super-Resolution (EDSR) and 34% in SRResNet as a direct consequence of the in-depth reduction. The proposed architecture improves the efficiency of training loss by performing weight normalization instead of augmentation technology. We compared our method to five recent existing super-resolution (SR) deep neural network methods, tested over three public satellite image datasets and a standard reference (PRIM) dataset. Experiment analysis is evaluated in peak to signal noise ratio (PSNR) and structural similarity index measure (SSIM).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Availability of Data and Material

All necessary data available in the manuscript, provided that any additional data needed is available upon request.

Code Availability

We have code for these proposed results that we do not currently shared.

References

  1. Huo, X., Tang, R., Ma, L., Shao, K., & Yang, Y. (2019). A novel neural network for super-resolution remote sensing image reconstruction. International Journal of Remote Sensing, 40(5–6), 2375–2385. https://doi.org/10.1080/01431161.2018.1516319.

    Article  Google Scholar 

  2. Deeba, F., Kun, S., Ali Dharejo, F., & Zhou, Y. (2020). Wavelet-based enhanced medical image super resolution. IEEE Access, 8, 37035–37044. https://doi.org/10.1109/ACCESS.2020.2974278.

    Article  Google Scholar 

  3. Deeba, F., She, K., Zhou, Y., & Ali, F. (2020). Sparse representation based computed tomography images reconstruction by coupled dictionary learning algorithm. IET Image Processing. https://doi.org/10.1049/iet-ipr.2019.1312.

    Article  Google Scholar 

  4. Jiang, K., Wang, Z., Yi, P., Wang, G., Lu, T., & Jiang, J. (2019). Edge-enhanced GAN for remote sensing image superresolution. IEEE Transactions on Geoscience and Remote Sensing, 57(8), 5799–5812. https://doi.org/10.1109/TGRS.2019.2902431.

    Article  Google Scholar 

  5. Deeba, F., Kun, S., Wang, W., Ahmed, J., & Qadir, B. (2019). Wavelet integrated residual dictionary training for single image super-resolution. Multimedia Tools and Applications, 78(19), 27683–27701. https://doi.org/10.1007/s11042-019-07850-4.

    Article  Google Scholar 

  6. Fernandez-Beltran, R., Latorre-Carmona, P., & Pla, F. (2017). Single-frame super-resolution in remote sensing: a practical overview. International Journal of Remote Sensing, 38(1), 314–354. https://doi.org/10.1080/01431161.2016.1264027.

    Article  Google Scholar 

  7. Dharejo, F. A., Zhou, Y., Deeba, F., Jatoi, M. A., Du, Y., & Wang, X. (2020). A remote‐sensing image enhancement algorithm based on patch‐wise dark channel prior and histogram equalisation with colour correction. IET Image Processing, ipr2.12004. https://doi.org/10.1049/ipr2.12004

  8. Dharejo, F. A., Zhou, Y., Deeba, F., & Du, Y. (2020). A color enhancement scene estimation approach for single image haze removal. IEEE Geoscience and Remote Sensing Letters. https://doi.org/10.1109/LGRS.2019.2951626.

    Article  Google Scholar 

  9. Merino, M. T., & Nunez, J. (2007). Super-resolution of remotely sensed images with variable-pixel linear reconstruction. IEEE Transactions on Geoscience and Remote Sensing, 45(5), 1446–1457. https://doi.org/10.1109/TGRS.2007.893271.

    Article  Google Scholar 

  10. Li, F., Jia, X., & Fraser, D. (2008). Universal HMT based super resolution for remote sensing images. In 2008 15th IEEE International Conference on Image Processing (pp. 333–336). IEEE. https://doi.org/10.1109/ICIP.2008.4711759

  11. Yang, S., Sun, F., Wang, M., Liu, Z., & Jiao, L. (2011). Novel super resolution restoration of remote sensing images based on compressive sensing and example patches-aided dictionary learning. In 2011 International Workshop on Multi-Platform/Multi-Sensor Remote Sensing and Mapping (pp. 1–6). IEEE. https://doi.org/10.1109/M2RSM.2011.5697375

  12. Yue, L., Shen, H., Li, J., Yuan, Q., Zhang, H., & Zhang, L. (2016). Image super-resolution: The techniques, applications, and future. Signal Processing, 128, 389–408. https://doi.org/10.1016/j.sigpro.2016.05.002.

    Article  Google Scholar 

  13. Pan, Z., Ma, W., Guo, J., & Lei, B. (2019). Super-Resolution of Single Remote Sensing Image Based on Residual Dense Backprojection Networks. IEEE Transactions on Geoscience and Remote Sensing, 57(10), 7918–7933. https://doi.org/10.1109/TGRS.2019.2917427.

    Article  Google Scholar 

  14. Lu, T., Wang, J., Zhang, Y., Wang, Z., & Jiang, J. (2019). Satellite image super-resolution via multi-scale residual deep neural network. Remote Sensing, 11(13), 1588. https://doi.org/10.3390/rs11131588.

    Article  Google Scholar 

  15. Yu, J., Fan, Y., Yang, J., Xu, N., Wang, Z., Wang, X., & Huang, T. (2018). Wide activation for efficient and accurate image super-resolution. Retrieved from http://arxiv.org/abs/1808.08718

  16. Zagoruyko, S., & Komodakis, N. (2016). Wide residual networks. In Procedings of the British Machine Vision Conference 2016 (pp. 87.1–87.12). British Machine Vision Association. https://doi.org/10.5244/C.30.87

  17. Wang, Z., Chen, J., & Hoi, S. C. H. (2020). Deep learning for image super-resolution: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence. https://doi.org/10.1109/TPAMI.2020.2982166.

    Article  Google Scholar 

  18. Kim, J., Lee, J. K., & Lee, K. M. (2016). Accurate image super-resolution using very deep convolutional networks. In 2016 IEEE conference on computer vision and pattern recognition (CVPR) (pp. 1646–1654). IEEE. https://doi.org/10.1109/CVPR.2016.182

  19. Dong, C., Loy, C. C., He, K., & Tang, X. (2014). Learning a deep convolutional network for image super-resolution (pp. 184–199). https://doi.org/10.1007/978-3-319-10593-2_13

  20. Tong, T., Li, G., Liu, X., & Gao, Q. (2017). Image super-resolution using dense skip connections. In 2017 IEEE international conference on computer vision (ICCV) (pp. 4809–4817). IEEE. https://doi.org/10.1109/ICCV.2017.514

  21. Tai, Y., Yang, J., Liu, X., & Xu, C. (2017). MemNet: A persistent memory network for image restoration. In 2017 IEEE international conference on computer vision (ICCV) (pp. 4549–4557). IEEE. https://doi.org/10.1109/ICCV.2017.486

  22. Dong, C., Loy, C. C., & Tang, X. (2016). Accelerating the super-resolution convolutional neural network. Lecture notes in computer science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 9906 LNCS, 391–407. https://doi.org/10.1007/978-3-319-46475-6_25

  23. Shi, W., Caballero, J., Huszar, F., Totz, J., Aitken, A. P., Bishop, R., Wang, Z. (2016). Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In 2016 IEEE conference on computer vision and pattern recognition (CVPR) (pp. 1874–1883). IEEE. https://doi.org/10.1109/CVPR.2016.207

  24. Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Shi, W. (2017). Photo-realistic single image super-resolution using a generative adversarial network. In 2017 IEEE conference on computer vision and pattern recognition (CVPR) (pp. 105–114). IEEE. https://doi.org/10.1109/CVPR.2017.19

  25. Lim, B., Son, S., Kim, H., Nah, S., & Lee, K. M. (2017). Enhanced deep residual networks for single image super-resolution. In IEEE computer society conference on computer vision and pattern recognition workshops, 2017-July (pp. 1132–1140). https://doi.org/10.1109/CVPRW.2017.151

  26. Irani, M., & Peleg, S. (1991). Improving resolution by image registration. CVGIP: Graphical Models and Image Processing, 53(3), 231–239. https://doi.org/10.1016/1049-9652(91)90045-L.

    Article  Google Scholar 

  27. Haris, M., Shakhnarovich, G., & Ukita, N. (2018). Deep back-projection networks for super-resolution. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 1664–1673). IEEE. https://doi.org/10.1109/CVPR.2018.00179

  28. Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017). Densely connected convolutional networks. In 2017 IEEE conference on computer vision and pattern recognition (CVPR) (pp. 2261–2269). IEEE. https://doi.org/10.1109/CVPR.2017.243

  29. Wang, Z., Yi, P., Jiang, K., Jiang, J., Han, Z., Lu, T., & Ma, J. (2019). Multi-memory convolutional neural network for video super-resolution. IEEE Transactions on Image Processing, 28(5), 2530–2544. https://doi.org/10.1109/TIP.2018.2887017.

    Article  MathSciNet  Google Scholar 

  30. Zhang, Y., Tian, Y., Kong, Y., Zhong, B., & Fu, Y. (2018). Residual Dense Network for Image Super-Resolution. In 2018 IEEE/CVF conference on computer vision and pattern recognition (pp. 2472–2481). IEEE. https://doi.org/10.1109/CVPR.2018.00262

  31. Liu, B., & Ait-Boudaoud, D. (2020). Effective image super resolution via hierarchical convolutional neural network. Neurocomputing, 374, 109–116. https://doi.org/10.1016/j.neucom.2019.09.035.

    Article  Google Scholar 

  32. Timofte, R., Agustsson, E., Gool, L. Van, Yang, M.-H., Zhang, L., Lim, B., Guo, Q. (2017). NTIRE 2017 Challenge on single image super-resolution: Methods and results. In 2017 IEEE conference on computer vision and pattern recognition workshops (CVPRW) (pp. 1110–1121). IEEE. https://doi.org/10.1109/CVPRW.2017.149

  33. Ioffe, S., & Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. In 32nd international conference on machine learning, ICML 2015, 1, (pp. 48–456).

  34. Wang, Z., Liu, D., Yang, J., Han, W., & Huang, T. (2015). Deep networks for image super-resolution with sparse prior. In 2015 IEEE international conference on computer vision (ICCV) (pp. 370–378). IEEE. https://doi.org/10.1109/ICCV.2015.50

  35. Kim, J., Lee, J. K., & Lee, K. M. (2016). Deeply-recursive convolutional network for image super-resolution. In 2016 IEEE conference on computer vision and pattern recognition (CVPR) (pp. 1637–1645). IEEE. https://doi.org/10.1109/CVPR.2016.181

  36. Timofte, R., De Smet, V., & Van Gool, L. (2015). A+: Adjusted anchored neighborhood regression for fast super-resolution (pp. 111–126). https://doi.org/10.1007/978-3-319-16817-3_8

  37. Su, H., Wei, S., Yan, M., Wang, C., Shi, J., & Zhang, X. (2019). Object detection and instance segmentation in remote sensing imagery based on precise mask R-CNN. In IGARSS 2019–2019 IEEE international geoscience and remote sensing symposium (pp. 1454–1457). IEEE. https://doi.org/10.1109/IGARSS.2019.8898573

  38. Blau, Y., Mechrez, R., Timofte, R., Michaeli, T., & Zelnik-Manor, L. (2019). The 2018 PIRM challenge on perceptual image super-resolution (pp. 334–355). https://doi.org/10.1007/978-3-030-11021-5_21

  39. https://www.kaggle.com/c/draper-satellite-image-chronology

  40. https://captain-whu.github.io/AID/

Download references

Funding

This work was supported in part by the Key Research Program of Frontier Sciences, CAS, and Grant number ZDBS-LY-DQC016, Beijing Natural Science Foundation under Grant No. 4212030, Beijing Nova Program of Science and Technology under Grant No. Z191100001119090, Natural Science Foundation of China under Grant No. 61836013 and, Youth Innovation Promotion Association CAS.

Author information

Authors and Affiliations

  1. Computer Network Information Center, Chinese Academy of Sciences, University of Chinese Academy of Sciences, Haidian, Beijing, 100190, China

    Farah Deeba, Yuanchun Zhou, Fayaz Ali Dharejo, Yi Du & Xuezhi Wang

  2. Department of Computing, Faculty of Engineering Sciences and Technology, Hamdard University, Karachi, Pakistan

    Farah Deeba

  3. School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, China

    She Kun

Authors
  1. Farah Deeba
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuanchun Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fayaz Ali Dharejo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yi Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xuezhi Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. She Kun
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Farah Deeba proposed the super-resolution (SR) method and has written the manuscript; Xuezhi Wang and Y. Zhou provided a useful guide to the SR method; FA dharejo and She Kun gave guidance on the experimental problem and some details; Yi Du compiled the experimental image data and polished the Language of paper.

Corresponding author

Correspondence to Xuezhi Wang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Deeba, F., Zhou, Y., Dharejo, F.A. et al. Multi-scale Single Image Super-Resolution with Remote-Sensing Application Using Transferred Wide Residual Network. Wireless Pers Commun 120, 323–342 (2021). https://doi.org/10.1007/s11277-021-08460-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11277-021-08460-w

Keywords

Search

Navigation