Yabo Duan
ABSTRACT
High-resolution images of the lunar surface are generally used to study the lunar soil and terrain. Nonetheless, acquiring higher-resolution images involves greater memory and calculation power, which is a challenge for lunar landers or rovers. In this study, a deep convolution neural network single image super-resolution reconstruction method based on channel-space attention is proposed to achieve the mapping of low-resolution lunar surface images to high-resolution lunar surface images. An enhanced block named feature fusion block is used for feature extraction. Furthermore, a channel-spatial attention module including efficient channel attention module and enhanced spatial attention module can extract more discriminative channel features and critical spatial features. Last, the model utilizes local implicit image function to predict RGB values of the images. The images of lander terrain camera and rover panoramic camera carried by Chang'e 3 and Chang'e 4 are used to training and validating models, and the Apollo rover images are used to test. The experiment demonstrates the superiority of the proposed novel model over the comparative method by using images of the Apollo project as the test dataset. Compared with the traditional methods, the PSNR value of the proposed method is improved by about 0.26dB in the 4x super-resolution reconstruction experiment.
- Maki, J. N., Gruel, D., McKinney, C., Ravine, M. A., Morales, M., Lee, D., Willson, R., Copley-Woods, D., Valvo, M., Goodsall, T., McGuire, J., Sellar, R. G., Schaffner, J. A., Caplinger, M. A., Shamah, J. M., Johnson, A. E., Ansari, H., Singh, K., Litwin, T., Deen, R., Culver, A., Ruoff, N., Petrizzo, D., Kessler, D., Basset, C., Estlin, T., Alibay, F., Nelessen, A. and Algermissen, S. The Mars 2020 Engineering Cameras and Microphone on the Perseverance Rover: A Next-Generation Imaging System for Mars Exploration. Space Sci Rev, 216, 8 (Dec 2020).Google Scholar
- Jia, Y. Z., Dai, S. W., Zou, Y. L. and Chen, X. W. An Overview of the Scientific Payloads in Chinese Chang'e-3 Lunar Mission. City, 2016.Google Scholar
- Jia, Y. Z., Zou, Y. L., Ping, J. S., Xue, C. B., Yan, J. and Ning, Y. M. The scientific objectives and payloads of Chang'E-4 mission. Planet Space Sci, 162 (Nov 1 2018), 207-215.Google ScholarCross Ref
- Wang, Z. H., Chen, J. and Hoi, S. C. H. Deep Learning for Image Super-Resolution: A Survey. Ieee T Pattern Anal, 43, 10 (Oct 1 2021), 3365-3387.Google Scholar
- Yang, J., Wright, J., Huang, T. S. and Ma, Y. Image super-resolution via sparse representation. IEEE Trans Image Process, 19, 11 (Nov 2010), 2861-2873.Google Scholar
- Chang, H., Yeung, D. Y. and Xiong, Y. Super-resolution through neighbor embedding. Proc Cvpr Ieee (2004), 275-282.Google ScholarCross Ref
- Kim, K. I. and Kwon, Y. Single-Image Super-Resolution Using Sparse Regression and Natural Image Prior. Ieee T Pattern Anal, 32, 6 (Jun 2010), 1127-1133.Google Scholar
- Timofte, R., De Smet, V. and Van Gool, L. A plus : Adjusted Anchored Neighborhood Regression for Fast Super-Resolution. Lect Notes Comput Sc, 9006 (2015), 111-126.Google ScholarCross Ref
- Mahapatra, D., Bozorgtabar, B. and Garnavi, R. Image super-resolution using progressive generative adversarial networks for medical image analysis. Comput Med Imag Grap, 71 (Jan 2019), 30-39.Google ScholarCross Ref
- Rasti, P., Uiboupin, T., Escalera, S. and Anbarjafari, G. Convolutional Neural Network Super Resolution for Face Recognition in Surveillance Monitoring. Articulated Motion and Deformable Objects, 9756 (2016), 175-184.Google ScholarCross Ref
- Tao, Y. and Muller, J. P. A novel method for surface exploration: Super-resolution restoration of Mars repeat-pass orbital imagery. Planet Space Sci, 121 (Feb 2016), 103-114.Google ScholarCross Ref
- Dong, C., Loy, C. C., He, K. M. and Tang, X. O. Image Super-Resolution Using Deep Convolutional Networks. Ieee T Pattern Anal, 38, 2 (Feb 2016), 295-307.Google Scholar
- Kim, J., Lee, J. K. and Lee, K. M. Accurate Image Super-Resolution Using Very Deep Convolutional Networks. 2016 Ieee Conference on Computer Vision and Pattern Recognition (Cvpr) (2016), 1646-1654.Google Scholar
- Tai, Y., Yang, J., Liu, X. M. and Xu, C. Y. MemNet: A Persistent Memory Network for Image Restoration. Ieee I Conf Comp Vis (2017), 4549-4557.Google Scholar
- Tai, Y., Yang, J. and Liu, X. M. Image Super-Resolution via Deep Recursive Residual Network. 30th Ieee Conference on Computer Vision and Pattern Recognition (Cvpr 2017) (2017), 2790-2798.Google ScholarCross Ref
- Kim, J., Lee, J. K. and Lee, K. M. Deeply-Recursive Convolutional Network for Image Super-Resolution. 2016 Ieee Conference on Computer Vision and Pattern Recognition (Cvpr) (2016), 1637-1645.Google Scholar
- Dong, C., Loy, C. C. and Tang, X. O. Accelerating the Super-Resolution Convolutional Neural Network. Computer Vision - Eccv 2016, Pt Ii, 9906 (2016), 391-407.Google Scholar
- Shi, W. Z., Caballero, J., Huszar, F., Totz, J., Aitken, A. P., Bishop, R., Rueckert, D. and Wang, Z. H. Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network. 2016 Ieee Conference on Computer Vision and Pattern Recognition (Cvpr) (2016), 1874-1883.Google Scholar
- Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z. H. and Shi, W. Z. Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network. 30th Ieee Conference on Computer Vision and Pattern Recognition (Cvpr 2017) (2017), 105-114.Google ScholarCross Ref
- Lim, B., Son, S., Kim, H., Nah, S. and Lee, K. M. Enhanced Deep Residual Networks for Single Image Super-Resolution. Ieee Comput Soc Conf (2017), 1132-1140.Google Scholar
- Zhang, Y. L., Tian, Y. P., Kong, Y., Zhong, B. N. and Fu, Y. Residual Dense Network for Image Super-Resolution. 2018 Ieee/Cvf Conference on Computer Vision and Pattern Recognition (Cvpr) (2018), 2472-2481.Google Scholar
- Dai, T., Cai, J. R., Zhang, Y. B., Xia, S. T. and Zhang, L. Second-order Attention Network for Single Image Super-Resolution. 2019 Ieee/Cvf Conference on Computer Vision and Pattern Recognition (Cvpr 2019) (2019), 11057-11066.Google Scholar
- Lai, W. S., Huang, J. B., Ahuja, N. and Yang, M. H. Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution. 30th Ieee Conference on Computer Vision and Pattern Recognition (Cvpr 2017) (2017), 5835-5843.Google Scholar
- Haris, M., Shakhnarovich, G. and Ukita, N. Deep Back-Projection Networks For Super-Resolution. 2018 Ieee/Cvf Conference on Computer Vision and Pattern Recognition (Cvpr) (2018), 1664-1673.Google Scholar
- Ayhan, B. and Kwan, C. Mastcam Image Resolution Enhancement with Application to Disparity Map Generation for Stereo Images with Different Resolutions. Sensors-Basel, 19, 16 (Aug 20 2019).Google ScholarCross Ref
- Tao, Y., Conway, S. J., Muller, J. P., Putri, A. R. D., Thomas, N. and Cremonese, G. Single Image Super-Resolution Restoration of TGO CaSSIS Colour Images: Demonstration with Perseverance Rover Landing Site and Mars Science Targets. Remote Sens-Basel, 13, 9 (May 2021).Google Scholar
- Muqeet, A., Hwang, J., Yang, S., Kang, J. H., Kim, Y. and Bae, S.-H. Multi-attention Based Ultra Lightweight Image Super-Resolution. City, 2020.Google Scholar
- Wang, Q., Wu, B., Zhu, P., Li, P., Zuo, W. and Hu, Q. ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020), 11531-11539.Google Scholar
- Liu, J., Zhang, W. J., Tang, Y. T., Tang, J. and Wu, G. S. Residual Feature Aggregation Network for Image Super-Resolution. 2020 Ieee/Cvf Conference on Computer Vision and Pattern Recognition (Cvpr) (2020), 2356-2365.Google Scholar
- Woo, S. H., Park, J., Lee, J. Y. and Kweon, I. S. CBAM: Convolutional Block Attention Module. Computer Vision - Eccv 2018, Pt Vii, 11211 (2018), 3-19.Google Scholar
- Chen, Y., Liu, S. and Wang, X. Learning Continuous Image Representation with Local Implicit Image Function. City, 2021.Google Scholar
- Heo, B., Chun, S., Oh, S. J., Han, D., Yun, S., Kim, G., Uh, Y. and Ha, J.-W. AdamP: Slowing Down the Slowdown for Momentum Optimizers on Scale-invariant Weights. City, 2020.Google Scholar
Recommendations
Deep Residual Attention Network for Spectral Image Super-Resolution
Computer Vision – ECCV 2018 WorkshopsAbstractSpectral imaging sensors often suffer from low spatial resolution, as there exists an essential tradeoff between the spectral and spatial resolutions that can be simultaneously achieved, especially when the temporal resolution needs to be ...
Spatial-Spectral Deep Residual Network for Hyperspectral Image Super-Resolution
AbstractRecently, single hyperspectral image super-resolution (SR) methods based on deep learning have been extensively studied. However, there has been limited technical development focusing on single hyperspectral image super-resolution due to the high-...
Edge-directed single image super-resolution via cross-resolution sharpening function learning
Edge-directed single image super-resolution methods have been paid more attentions due to their sharp edge preserving in the recovered high-resolution image. Their core is the high-resolution gradient estimation. In this paper, we propose a novel cross-...
Comments