Avinash Chouhan
ABSTRACT
Satellite image super resolution is an important task that generates high resolution satellite images from low resolution inputs. Multi-frame super resolution utilizes multiple low-resolution images to generate a single high-resolution image. Multi-frame super resolution methods face difficulty in handling spatial and temporal dependencies of pixels. In this work, we proposed a novel architecture named Multi-context Dense Network (MCDNet) to handle spatial and temporal pixel dependencies using multiple approaches of global average pooling, multiple size kernels, and self-attention. The proposed approach improved the PSNR values by 0.29 % and 0.001 % for super resolution of NIR and RED bands on the benchmark PROBA-V dataset.
- Nour Aburaed, Alavikunhu Panthakkan, Mina Al-Saad, Marwa Chendeb El Rai, Saeed Al Mansoori, Hussain Al-Ahmad, and Stephen Marshall. 2020. Super-resolution of satellite imagery using a wavelet multiscale-based deep convolutional neural network model. In Image and Signal Processing for Remote Sensing XXVI, Lorenzo Bruzzone, Francesca Bovolo, and Emanuele Santi (Eds.). Vol. 11533. International Society for Optics and Photonics, SPIE, 115331J. https://doi.org/10.1117/12.2573991Google Scholar
Cross Ref
- Tai An, Xin Zhang, Chunlei Huo, Bin Xue, Lingfeng Wang, and Chunhong Pan. 2022. TR-MISR: Multiimage Super-Resolution Based on Feature Fusion With Transformers. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 15 (2022), 1373–1388. https://doi.org/10.1109/JSTARS.2022.3143532Google ScholarCross Ref
- J. Anger, T. Ehret, Carlo de Franchis, and Gabriele Facciolo. 2020. FAST AND ACCURATE MULTI-FRAME SUPER-RESOLUTION OF SATELLITE IMAGES. ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences V-1-2020 (08 2020), 57–64. https://doi.org/10.5194/isprs-annals-V-1-2020-57-2020Google ScholarCross Ref
- Daniel Bull, Nick Lim, and Eibe Frank. 2021. Perceptual improvements for Super-Resolution of Satellite Imagery. In 2021 36th International Conference on Image and Vision Computing New Zealand (IVCNZ). 1–6. https://doi.org/10.1109/IVCNZ54163.2021.9653355Google ScholarCross Ref
- Hasan Demirel and Gholamreza Anbarjafari. 2010. Satellite Image Resolution Enhancement Using Complex Wavelet Transform. IEEE Geoscience and Remote Sensing Letters 7, 1 (2010), 123–126. https://doi.org/10.1109/LGRS.2009.2028440Google ScholarCross Ref
- Hasan Demirel and Gholamreza Anbarjafari. 2011. Discrete Wavelet Transform-Based Satellite Image Resolution Enhancement. IEEE Transactions on Geoscience and Remote Sensing 49, 6 (2011), 1997–2004. https://doi.org/10.1109/TGRS.2010.2100401Google ScholarCross Ref
- Michel Deudon, Alfredo Kalaitzis, Israel Goytom, Md Rifat Arefin, Zhichao Lin, Kris Sankaran, Vincent Michalski, Samira E. Kahou, Julien Cornebise, and Yoshua Bengio. 2020. HighRes-net: Recursive Fusion for Multi-Frame Super-Resolution of Satellite Imagery. https://doi.org/10.48550/ARXIV.2002.06460Google ScholarCross Ref
- Francisco Dorr. 2020. Satellite Image Multi-Frame Super Resolution Using 3D Wide-Activation Neural Networks. Remote Sensing 12, 22 (2020). https://doi.org/10.3390/rs12223812Google ScholarCross Ref
- S. Farsiu, M.D. Robinson, M. Elad, and P. Milanfar. 2004. Fast and robust multiframe super resolution. IEEE Transactions on Image Processing 13, 10 (2004), 1327–1344. https://doi.org/10.1109/TIP.2004.834669Google ScholarDigital Library
- M. Hakim, A. Ghazdali, and A. Laghrib. 2020. A multi-frame super-resolution based on new variational data fidelity term. Applied Mathematical Modelling 87 (2020), 446–467. https://doi.org/10.1016/j.apm.2020.06.013Google ScholarCross Ref
- Mohamed Ramzy Ibrahim, Robert Benavente, Felipe Lumbreras, and Daniel Ponsa. 2022. 3DRRDB: Super Resolution of Multiple Remote Sensing Images using 3D Residual in Residual Dense Blocks. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 322–331. https://doi.org/10.1109/CVPRW56347.2022.00047Google ScholarCross Ref
- Kui Jiang, Zhongyuan Wang, Peng Yi, Guangcheng Wang, Tao Lu, and Junjun Jiang. 2019. Edge-Enhanced GAN for Remote Sensing Image Superresolution. IEEE Transactions on Geoscience and Remote Sensing 57, 8 (2019), 5799–5812. https://doi.org/10.1109/TGRS.2019.2902431Google ScholarCross Ref
- Toshiyuki Kato, Hideitsu Hino, and Noboru Murata. 2017. Double sparsity for multi-frame super resolution. Neurocomputing 240 (2017), 115–126. https://doi.org/10.1016/j.neucom.2017.02.043Google ScholarDigital Library
- Michal Kawulok, Pawel Benecki, Daniel Kostrzewa, and Lukasz Skonieczny. 2018. Evolving Imaging Model for Super-Resolution Reconstruction(GECCO ’18). Association for Computing Machinery, New York, NY, USA, 284–285. https://doi.org/10.1145/3205651.3205676Google ScholarDigital Library
- Michal Kawulok, Pawel Benecki, Szymon Piechaczek, Krzysztof Hrynczenko, Daniel Kostrzewa, and Jakub Nalepa. 2020. Deep Learning for Multiple-Image Super-Resolution. IEEE Geoscience and Remote Sensing Letters 17, 6 (2020), 1062–1066. https://doi.org/10.1109/LGRS.2019.2940483Google ScholarCross Ref
- Michal Kawulok, Tomasz Tarasiewicz, Jakub Nalepa, Diana Tyrna, and Daniel Kostrzewa. 2021. Deep Learning for Multiple-Image Super-Resolution of Sentinel-2 Data. In 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS. 3885–3888. https://doi.org/10.1109/IGARSS47720.2021.9553243Google ScholarCross Ref
- Minji Lee, Inyong Koo, Kangwook Ko, and Changick Kim. 2022. Multi-image Super-resolution via Quality Map Associated Temporal Attention Network.Google Scholar
- Huan Liu and Yanfeng Gu. 2022. Deep Joint Estimation Network for Satellite Video Super-Resolution With Multiple Degradations. IEEE Transactions on Geoscience and Remote Sensing 60 (2022), 1–15. https://doi.org/10.1109/TGRS.2022.3163790Google ScholarCross Ref
- Xun Liu, Chenwei Deng, Jocelyn Chanussot, Danfeng Hong, and Baojun Zhao. 2019. StfNet: A Two-Stream Convolutional Neural Network for Spatiotemporal Image Fusion. IEEE Transactions on Geoscience and Remote Sensing 57, 9 (2019), 6552–6564. https://doi.org/10.1109/TGRS.2019.2907310Google ScholarCross Ref
- Xiaomin Liu, Tiantian Feng, Xiaofan Shen, and Rongxing Li. 2022. PMDRnet: A Progressive Multiscale Deformable Residual Network for Multi-Image Super-Resolution of AMSR2 Arctic Sea Ice Images. IEEE Transactions on Geoscience and Remote Sensing 60 (2022), 1–18. https://doi.org/10.1109/TGRS.2022.3151623Google ScholarCross Ref
- Jie Ma, Libao Zhang, and Jue Zhang. 2020. SD-GAN: Saliency-Discriminated GAN for Remote Sensing Image Superresolution. IEEE Geoscience and Remote Sensing Letters 17, 11 (2020), 1973–1977. https://doi.org/10.1109/LGRS.2019.2956969Google ScholarCross Ref
- Emanuele Mandanici, Luca Tavasci, Francesco Corsini, and S. Gandolfi. 2019. A multi-image super-resolution algorithm applied to thermal imagery. Applied Geomatics 11 (02 2019). https://doi.org/10.1007/s12518-019-00253-yGoogle ScholarCross Ref
- M. Märtens, D. Izzo, A. Krzic, and D. Cox. 2019. Super-resolution of PROBA-V images using convolutional neural networks. https://rdcu.be/bPOud. Astrodynamics (8 2019).Google Scholar
- Shaohui Mei, Ruituo Jiang, Xu Li, and Qian Du. 2020. Spatial and Spectral Joint Super-Resolution Using Convolutional Neural Network. IEEE Transactions on Geoscience and Remote Sensing 58, 7 (2020), 4590–4603. https://doi.org/10.1109/TGRS.2020.2964288Google ScholarCross Ref
- Andrea Bordone Molini, Diego Valsesia, Giulia Fracastoro, and Enrico Magli. 2019. Deep Learning For Super-Resolution Of Unregistered Multi-Temporal Satellite Images. In 2019 10th Workshop on Hyperspectral Imaging and Signal Processing: Evolution in Remote Sensing (WHISPERS). 1–5. https://doi.org/10.1109/WHISPERS.2019.8920910Google ScholarCross Ref
- Andrea Bordone Molini, Diego Valsesia, Giulia Fracastoro, and Enrico Magli. 2019. DeepSUM: Deep neural network for Super-resolution of Unregistered Multitemporal images. IEEE Transactions on Geoscience and Remote Sensing 58, 5 (2019), 3644–3656.Google ScholarCross Ref
- Andrea Bordone Molini, Diego Valsesia, Giulia Fracastoro, and Enrico Magli. 2020. Deepsum++: Non-Local Deep Neural Network for Super-Resolution of Unregistered Multitemporal Images. In IGARSS 2020 - 2020 IEEE International Geoscience and Remote Sensing Symposium. 609–612. https://doi.org/10.1109/IGARSS39084.2020.9324418Google ScholarCross Ref
- Marwa Mostafa and Sayed Ahmed. 2021. Satellite Imagery Super-Resolution Using Squeeze-and-Excitation-Based GAN. International Journal of Aeronautical and Space Sciences (06 2021). https://doi.org/10.1007/s42405-021-00396-6Google ScholarCross Ref
- Marcus Märtens, Dario Izzo, Andrej Kržič, and Daniël Cox. 2019. Super-resolution of PROBA-V images using convolutional neural networks. Astrodynamics 3 (08 2019). https://doi.org/10.1007/s42064-019-0059-8Google ScholarCross Ref
- Ngoc Long Nguyen, Jérémy Anger, Axel Davy, Pablo Arias, and Gabriele Facciolo. 2021. Self-supervised multi-image super-resolution for push-frame satellite images. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 1121–1131. https://doi.org/10.1109/CVPRW53098.2021.00123Google ScholarCross Ref
- Muhammed Razzak, Gonzalo Mateo-Garcia, Luis Gómez-Chova, Yarin Gal, and Freddie Kalaitzis. 2021. Multi-Spectral Multi-Image Super-Resolution of Sentinel-2 with Radiometric Consistency Losses and Its Effect on Building Delineation. https://doi.org/10.48550/ARXIV.2111.03231Google ScholarCross Ref
- Md Rifat Arefin, Vincent Michalski, Pierre-Luc St-Charles, Alfredo Kalaitzis, Sookyung Kim, Samira E. Kahou, and Yoshua Bengio. 2020. Multi-Image Super-Resolution for Remote Sensing using Deep Recurrent Networks. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 816–825. https://doi.org/10.1109/CVPRW50498.2020.00111Google ScholarCross Ref
- Francesco Salvetti, Vittorio Mazzia, Aleem Khaliq, and Marcello Chiaberge. 2020. Multi-Image Super Resolution of Remotely Sensed Images Using Residual Attention Deep Neural Networks. Remote Sensing 12, 14 (2020). https://doi.org/10.3390/rs12142207Google ScholarCross Ref
- Changyeop Shin, Sungho Kim, and Youngjung Kim. 2022. Satellite Image Target Super-Resolution With Adversarial Shape Discriminator. IEEE Geoscience and Remote Sensing Letters 19 (2022), 1–5. https://doi.org/10.1109/LGRS.2020.3042238Google ScholarCross Ref
- Tao Sun, Zhengqiang Xiong, Zixian Wei, and Zhengxing Wang. 2022. Infrared image super-resolution method for edge computing based on adaptive nonlocal means. The Journal of Supercomputing 78 (04 2022), 1–22. https://doi.org/10.1007/s11227-021-04141-4Google ScholarDigital Library
- Diego Valsesia and Enrico Magli. 2022. Permutation invariance and uncertainty in multitemporal image super-resolution. IEEE Transactions on Geoscience and Remote Sensing 60 (2022), 1–12. https://doi.org/10.1109/TGRS.2021.3130673Google ScholarCross Ref
- Junwei Wang, Kun Gao, Zhenzhou Zhang, Chong Ni, Zibo Hu, Dayu Chen, and Qiong Wu. 2021. Multisensor Remote Sensing Imagery Super-Resolution with Conditional GAN. Journal of Remote Sensing 2021, Article 9829706 (Sept. 2021), 9829706 pages. https://doi.org/10.34133/2021/9829706Google ScholarCross Ref
- Peijuan Wang and Elif Sertel. 2023. Multi-frame super-resolution of remote sensing images using attention-based GAN models. Knowledge-Based Systems 266 (2023), 110387. https://doi.org/10.1016/j.knosys.2023.110387Google ScholarDigital Library
- Yi Xiao, Xin Su, Qiangqiang Yuan, Denghong Liu, Huanfeng Shen, and Liangpei Zhang. 2022. Satellite Video Super-Resolution via Multiscale Deformable Convolution Alignment and Temporal Grouping Projection. IEEE Transactions on Geoscience and Remote Sensing 60 (2022), 1–19. https://doi.org/10.1109/TGRS.2021.3107352Google ScholarCross Ref
- Yi Xiao, Qiangqiang Yuan, Jiang He, Qiang Zhang, Jing Sun, Xin Su, Jialian Wu, and Liangpei Zhang. 2022. Space-time super-resolution for satellite video: A joint framework based on multi-scale spatial-temporal transformer. International Journal of Applied Earth Observation and Geoinformation 108 (2022), 102731. https://doi.org/10.1016/j.jag.2022.102731Google ScholarCross Ref
- Li Yan and Kun Chang. 2021. A New Super Resolution Framework Based on Multi-Task Learning for Remote Sensing Images. Sensors 21, 5 (2021). https://doi.org/10.3390/s21051743Google ScholarCross Ref
Index Terms
- MCDNet: Multi Context Dense Network for multi-frame super resolution of satellite images
Recommendations
Super Resolution Reconstruction via Multiple Frames Joint Learning
CMSP '11: Proceedings of the 2011 International Conference on Multimedia and Signal Processing - Volume 01This paper presents a novel multi-frame joint learning approach for image super resolution via sparse representation. Based on the assumption that several low-resolution patches degraded from a same high-resolution patch under subpixel translation can ...
VarSR: Variational Super-Resolution Network for Very Low Resolution Images
Computer Vision – ECCV 2020AbstractAs is well known, single image super-resolution (SR) is an ill-posed problem where multiple high resolution (HR) images can be matched to one low resolution (LR) image due to the difference in their representation capabilities. Such many-to-one ...
An Improved Regularization-Based Super-Resolution Reconstruction for Multi-frame Passive Millimeter-Wave Images
CIT '12: Proceedings of the 2012 IEEE 12th International Conference on Computer and Information TechnologyIn passive millimeter wave imaging, the image gets low resolution for the limitation of the antenna aperture. It is necessary to improve the quality of the image in the way of signal processing. This paper puts forward an improved regularization-based ...
Comments