Abstract
The goal of zero-shot image super-resolution (SR) is to generate high-resolution (HR) images from never-before-seen image distributions. This is challenging, especially, because it is difficult to model the statistics of an image that the network has never seen before. Despite deep convolutional neural networks (CNN) being superior to traditional super-resolution (SR) methods, little attention has been given to generating remote sensing scene-based HR images which do not have any prior ground truths available for training. In this paper, we propose a framework that harnesses the inherent tessellated nature of remotely images using continuity to generate HR images that tackle atmospheric and radiometric condition variations. Our proposed solution utilizes self tessellations to fully harness the image heuristics to generate an SR image from a low resolution (LR) input. The salience of our approach lies in a two-fold data generation in a self-preservation case and a cascaded attention sharing mechanism on the latent space for content preservation while generating SR images. By learning a mapping from LR space to SR space while keeping the content statistics preserved helps in better quality image generation. The attention sharing between content and tessellations aids in learning the overall big picture for super-resolution without losing an eye on the main image to be super-resolved. We showcase our results with the generated images given the low resolution (LR) input images in zero-shot cases comparable to state-of-the-art results on EuroSAT and PatternNet datasets with metrics of SSIM and PSNR. We further show how this architecture can be leveraged for non-remote sensing (RS) applications.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
PerceptX labs: https://sites.google.com/view/perceptx/home.
References
Cheng, X., Fu, Z., Yang, J.: Zero-shot image super-resolution with depth guided internal degradation learning. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12362, pp. 265–280. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58520-4_16
Demiray, B.Z., Sit, M., Demir, I.: D-SRGAN: dem super-resolution with generative adversarial networks. SN Comput. Sci. 2(1), 1–11 (2021). https://doi.org/10.1007/s42979-020-00442-2
Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks (2015)
Helber, P., Bischke, B., Dengel, A., Borth, D.: EuroSAT: a novel dataset and deep learning benchmark for land use and land cover classification (2019)
Ji, H., Gao, Z., Mei, T., Ramesh, B.: Vehicle detection in remote sensing images leveraging on simultaneous super-resolution. IEEE Geosci. Remote Sens. Lett. 17(4), 676–680 (2020). https://doi.org/10.1109/LGRS.2019.2930308
Lanaras, C., Bioucas-Dias, J., Galliani, S., Baltsavias, E., Schindler, K.: Super-resolution of sentinel-2 images: learning a globally applicable deep neural network. ISPRS J. Photogramm. Remote Sens. 146, 305–319 (2018). https://doi.org/10.1016/j.isprsjprs.2018.09.018
Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network (2017)
Lim, B., Son, S., Kim, H., Nah, S., Lee, K.M.: Enhanced deep residual networks for single image super-resolution (2017)
Müller, M.U., Ekhtiari, N., Almeida, R.M., Rieke, C.: Super-resolution of multispectral satellite images using convolutional neural networks. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci. V-1-2020, 33–40 (2020). https://doi.org/10.5194/isprs-annals-v-1-2020-33-2020
Rabbi, J., Ray, N., Schubert, M., Chowdhury, S., Chao, D.: Small-object detection in remote sensing images with end-to-end edge-enhanced GAN and object detector network (2020)
Shocher, A., Cohen, N., Irani, M.: “zero-shot” super-resolution using deep internal learning (2017)
Soh, J.W., Cho, S., Cho, N.I.: Meta-transfer learning for zero-shot super-resolution (2020)
Wang, X., et al.: ESRGAN: enhanced super-resolution generative adversarial networks. In: Leal-Taixé, L., Roth, S. (eds.) ECCV 2018. LNCS, vol. 11133, pp. 63–79. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11021-5_5
Xu, W., Xu, G., Wang, Y., Sun, X., Lin, D., Wu, Y.: High quality remote sensing image super-resolution using deep memory connected network. In: IGARSS 2018–2018 IEEE International Geoscience and Remote Sensing Symposium, July 2018. https://doi.org/10.1109/igarss.2018.8518855
Yang, Y., Newsam, S.: Bag-of-visual-words and spatial extensions for land-use classification. In: ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems (ACM GIS) (2010)
Zhang, J., Wang, Z., Zheng, Y., Zhang, G.: Cascaded convolutional neural network for image super-resolution. In: Sun, X., Zhang, X., Xia, Z., Bertino, E. (eds.) ICAIS 2021. CCIS, vol. 1422, pp. 361–373. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-78615-1_32
Zhou, W., Newsam, S., Li, C., Shao, Z.: PatternNet: a benchmark dataset for performance evaluation of remote sensing image retrieval. ISPRS J. Photogramm. Remote Sens. 145, 197–209 (2018). https://doi.org/10.1016/j.isprsjprs.2018.01.004
Acknowledgements
The authors would like to thank the members of PerceptX labsFootnote 1 for their valuable feedback that contributed to this research.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Bose, R., Rangnekar, V., Banerjee, B., Chaudhuri, S. (2021). Zero-Shot Remote Sensing Image Super-Resolution Based on Image Continuity and Self Tessellations. In: Bauckhage, C., Gall, J., Schwing, A. (eds) Pattern Recognition. DAGM GCPR 2021. Lecture Notes in Computer Science(), vol 13024. Springer, Cham. https://doi.org/10.1007/978-3-030-92659-5_42
Download citation
DOI: https://doi.org/10.1007/978-3-030-92659-5_42
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-92658-8
Online ISBN: 978-3-030-92659-5
eBook Packages: Computer ScienceComputer Science (R0)