Skip to main content
SpringerLink
Log in

Remote Sensing Image Fusion Method Based on Retinex Model and Hybrid Attention Mechanism

  • Conference paper
  • First Online:
Space Information Networks (SINC 2023)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 2057))

Included in the following conference series:

  • 38 Accesses

Abstract

Pansharpening is a technique that fuses a low-resolution multispectral image (LRMS) and a panchromatic image (PAN) to obtain a high-resolution multispectral image (HRMS). Based on the observation that PAN and LRMS respectively have the characteristics of illumination component and reflection component of HRMS after Retinex decomposition, this paper proposes an inverse Retinex model guided pansharpening network, termed as AIRNet. Specifically, a Spatial Attention based Illuminance Module (SAIM) is proposed to convert the PAN to the illuminance component of HRMS. And a Hybrid Attention-based Reflectance Module (HARM) is used to convert the LRMS to the reflection component of the HRMS. Finally, based on the inverse Retinex model, the corresponding illuminance component and reflection component of the obtained HRMS are fused to obtain HRMS. Qualitative and quantitative comparison experiments with state-of-the-art pansharpening methods on multiple remote sensing image datasets show that AIRNet has significantly outstanding performance. In addition, multiple ablation experiments also show that the proposed SAIM and HARM are effective modules of AIRNet for pansharpening.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 49.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 64.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Aiazzi, B., Alparone, L., Baronti, S., et al.: MTF-tailored multiscale fusion of high-resolution MS and Pan imagery. Photogramm. Eng. Remote. Sens. 72(5), 591–596 (2006)

    Article  Google Scholar 

  2. Aiazzi, B., Baronti, S., Selva, M.: Improving component substitution pansharpening through multivariate regression of MS + Pan data. IEEE Trans. Geosci. Remote Sens. 45(10), 3230–3239 (2007)

    Article  Google Scholar 

  3. Alparone, L., Aiazzi, B., Baronti, S., et al.: Multispectral and panchromatic data fusion assessment without reference. Photogramm. Eng. Remote. Sens. 74(2), 193–200 (2008)

    Article  Google Scholar 

  4. Alparone, L., Wald, L., Chanussot, J., et al.: Comparison of pansharpening algorithms: outcome of the 2006 GRS-S data-fusion contest. IEEE Trans. Geosci. Remote Sens. 45(10), 3012–3021 (2007)

    Article  Google Scholar 

  5. Xiao, L., Liu, P.F., Li, H.: Progress and challenges in the fusion of multisource spatial-spectral remote sensing images. J. Image Graph. 25(5), 851–863 (2020)

    Google Scholar 

  6. Cai, J., Huang, B.: Super-resolution-guided progressive pansharpening based on a deep convolutional neural network. IEEE Trans. Geosci. Remote Sens. 59(6), 5206–5220 (2020)

    Article  Google Scholar 

  7. Cao, X., Yao, J., Xu, Z., et al.: Hyperspectral image classification with convolutional neural network and active learning. IEEE Trans. Geosci. Remote Sens. 58(7), 4604–4616 (2020)

    Article  Google Scholar 

  8. Dong, W., Xiao, S., Li, Y., et al.: Hyperspectral pansharpening based on intrinsic image decomposition and weighted least squares filter. Remote Sens. 10(3), 445 (2018)

    Article  MathSciNet  Google Scholar 

  9. Du, P., Liu, S., Xia, J., et al.: Information fusion techniques for change detection from multi-temporal remote sensing images. Inf. Fusion 14(1), 19–27 (2013)

    Article  Google Scholar 

  10. Fang, F., Li, F., Shen, C., et al.: A variational approach for pan-sharpening. IEEE Trans. Image Process. 22(7), 2822–2834 (2013)

    Article  Google Scholar 

  11. Hu, J., Hu, P., Kang, X., et al.: Pan-sharpening via multiscale dynamic convolutional neural network. IEEE Trans. Geosci. Remote Sens. 59(3), 2231–2244 (2020)

    Article  Google Scholar 

  12. Yong, Y.: Deep-learning approaches for pixel-level pansharpening. Natl. Remote Sens. Bull. 26(12), 2411–2432 (2023)

    Article  Google Scholar 

  13. Jin, X., Gu, Y., Liu, T.: Intrinsic image recovery from remote sensing hyperspectral images. IEEE Trans. Geosci. Remote Sens. 57(1), 224–238 (2018)

    Article  Google Scholar 

  14. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, vol. 25 (2012)

    Google Scholar 

  15. Land, E.H., McCann, J.J.: Lightness and retinex theory. Josa 61(1), 1–11 (1971)

    Article  Google Scholar 

  16. Larochelle, H., Hinton, G.E.: Learning to combine foveal glimpses with a third-order Boltzmann machine. In: Advances in Neural Information Processing Systems, vol. 23 (2010)

    Google Scholar 

  17. Ma, J., Yu, W., Chen, C., et al.: Pan-GAN: an unsupervised pan-sharpening method for remote sensing image fusion. Inf. Fusion 62, 110–120 (2020)

    Article  Google Scholar 

  18. Masi, G., Cozzolino, D., Verdoliva, L., et al.: Pansharpening by convolutional neural networks. Remote Sens. 8(7), 594 (2016)

    Article  Google Scholar 

  19. Otazu, X., González-Audícana, M., Fors, O., et al.: Introduction of sensor spectral response into image fusion methods. Application to wavelet-based methods. IEEE Trans. Geosci. Remote Sens. 43(10), 2376–2385 (2005)

    Article  Google Scholar 

  20. Schowengerdt, R.A.: Remote Sensing: Models and Methods for Image Processing. Elsevier, Amsterdam (2006)

    Google Scholar 

  21. Shah, V.P., Younan, N.H., King, R.L.: An efficient pan-sharpening method via a combined adaptive PCA approach and contourlets. IEEE Trans. Geosci. Remote Sens. 46(5), 1323–1335 (2008)

    Article  Google Scholar 

  22. Song, X., Huang, J., Cao, J., et al.: Multi-scale joint network based on Retinex theory for low-light enhancement. Signal Image Video Process. 15, 1–8 (2021)

    Google Scholar 

  23. Vivone, G., Alparone, L., Chanussot, J., et al.: A critical comparison among pansharpening algorithms. IEEE Trans. Geosci. Remote Sens. 53(5), 2565–2586 (2014)

    Article  Google Scholar 

  24. Vivone, G., Restaino, R., Dalla Mura, M., et al.: Contrast and error-based fusion schemes for multispectral image pansharpening. IEEE Geosci. Remote Sens. Lett. 11(5), 930–934 (2013)

    Article  Google Scholar 

  25. Wald, L., Ranchin, T., Mangolini, M.: Fusion of satellite images of different spatial resolutions: assessing the quality of resulting images. Photogramm. Eng. Remote. Sens. 63(6), 691–699 (1997)

    Google Scholar 

  26. Wang, F., Jiang, M., Qian, C., et al.: Residual attention network for image classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3156–3164 (2017)

    Google Scholar 

  27. Wang, T., Fang, F., Li, F., et al.: High-quality Bayesian pansharpening. IEEE Trans. Image Process. 28(1), 227–239 (2018)

    Article  MathSciNet  Google Scholar 

  28. Wang, Y., Deng, L.J., Zhang, T.J., et al.: SSconv: explicit spectral-to-spatial convolution for pansharpening. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 4472–4480 (2021)

    Google Scholar 

  29. Woo, S., Park, J., Lee, J.-Y., Kweon, I.S.: Cbam: Convolutional block attention module. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 3–19. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_1

    Chapter  Google Scholar 

  30. Xu, J., Hou, Y., Ren, D., et al.: Star: a structure and texture aware retinex model. IEEE Trans. Image Process. 29, 5022–5037 (2020)

    Article  Google Scholar 

  31. Xu, S., Zhang, J., Zhao, Z., et al.: Deep gradient projection networks for pan-sharpening. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1366–1375 (2021)

    Google Scholar 

  32. Yuhas, R.H., Goetz, A.F.H., Boardman, J.W.: Discrimination among semi-arid landscape endmembers using the spectral angle mapper (SAM) algorithm. In: JPL, Summaries of the Third Annual JPL Airborne Geoscience Workshop. Volume 1: AVIRIS Workshop (1992)

    Google Scholar 

  33. Zhou, J., Civco, D.L., Silander, J.A.: A wavelet transform method to merge Landsat TM and SPOT panchromatic data. Int. J. Remote Sens. 19(4), 743–757 (1998)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, East China Normal University, Shanghai, 200062, China

    Yongxu Ye, Tingting Wang, Faming Fang & Guixu Zhang

Authors
  1. Yongxu Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tingting Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Faming Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Guixu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Faming Fang .

Editor information

Editors and Affiliations

  1. Institute of China Electronic Equipment System Engineering Corporation, Beijing, China

    Quan Yu

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ye, Y., Wang, T., Fang, F., Zhang, G. (2024). Remote Sensing Image Fusion Method Based on Retinex Model and Hybrid Attention Mechanism. In: Yu, Q. (eds) Space Information Networks. SINC 2023. Communications in Computer and Information Science, vol 2057. Springer, Singapore. https://doi.org/10.1007/978-981-97-1568-8_7

Download citation

  • DOI: https://doi.org/10.1007/978-981-97-1568-8_7

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-97-1567-1

  • Online ISBN: 978-981-97-1568-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation