Skip to main content
SpringerLink
Log in

A novel detail injection framework using latent low-rank decomposition for multispectral pan-sharpening

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

A novel framework for pansharpening based on Latent Low-Rank Representation theory, called Detail injection using Latent Low-Rank decomposition based Pansharpening approach (DiLRP), is proposed. Our proposal comprises two fusion stages. In the first step, a primary joint fusion scheme is defined as a combination of low-rank and saliency images, which is used further for extracting spatial details. In this model, the histogram-matched PAN and up-sampled images are decomposed into low-rank and saliency components, and the corresponding fusion strategies are designed according to their characteristics. Indeed, to preserve more global structural information, low-rank components are combined by an optimized weighted average fusion strategy to generate a low-rank image. Furthermore, the saliency image is obtained by a simple average fusion rule in order get high-frequencies details (i.e., edges) that are highly relevant to MS image. The second stage consists in injecting (globally or locally) the high-frequency details, extracted from the reconstructed primary joint fused image using a multi-scale approach, into the up-sampled MS image. The performances of the proposed method have been studied both at reduced resolution and at full resolution. Three different datasets, acquired by the QuickBird, Pléaides-1A and WorldView-2 sensors, are used for validation. Compared with several well-known algorithms, experimental results reveal the validity and the advantages of the proposed DiLRP method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Aiazzi B, Alparone L, Baronti S, Garzelli A (2002) Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis. IEEE Trans Geosci Remote Sens 40(10):2300–2312

    Article  Google Scholar 

  2. Aiazzi B, Alparone L, Baronti S, Garzelli A, Selva M (2006) MTF-Tailored multiscale fusion of high-resolution ms and pan imagery. Photogrammetric Engineering and Remote Sensing 72(5):591–596

    Article  Google Scholar 

  3. Aiazzi B, Baronti S, Selva M (2007) Improving component substitution pansharpening through multivariate regression of MS +Pan data. IEEE Trans Geosci Remote Sens 45(10):3230–3239

    Article  Google Scholar 

  4. Alparone L, Aiazzi B, Baronti S, Garzelli A, Nencini F, Selva M (2008) Multispectral and panchromatic data fusion assessment without reference. Photogrammetric Engineering and Remote Sensing 74(2):193–200

    Article  Google Scholar 

  5. Alparone L, Baronti S, Aiazzi B, Garzelli A (2016) Spatial methods for multispectral pansharpening: Multiresolution analysis demystified. IEEE Trans Geosci Remote Sens 54(5):2563–2576

    Article  Google Scholar 

  6. Alparone L, Baronti S, Garzelli A, Nencini F (2004) A global quality measurement of pan-sharpened multispectral imagery. IEEE Geosci Remote Sens Lett 1:313–317

    Article  Google Scholar 

  7. Amro I, Mateos J, Vega M, Molina R, Katsaggelos AK (2011) A survey of classical methods and new trends in pansharpening of multispectral images. EURASIP Journal on Advances in Signal Processing 2011:79

    Article  Google Scholar 

  8. Azarang A, Manoochehri HE, Kehtarnavaz N (2019) Convolutional autoencoder-based multispectral image fusion. IEEE Access 7:35673–35683

    Article  Google Scholar 

  9. Ballester C, Caselles V, Igual L, Verdera J, Rougé B (2006) A variational model for p+xs image fusion. Int J Comput Vis 69(1):43–58

    Article  Google Scholar 

  10. Carper W, Lillesand T, Kiefer P (1990) The use of intensity-hue-saturation transformations for merging spot panchromatic and multispectral image data. Photogrammetric Engineering and Remote Sensing 56 (4):459–467

    Google Scholar 

  11. Chavez P, Sides S, Anderson J (1991) Comparison of three different methods to merge multiresolution and multispectral data: Landsat tm and spot panchromatic. Photogrammetric Engineering and Remote Sensing 57(3):295–303

    Google Scholar 

  12. Cheng M, Wang C, Li J (2014) Sparse representation based pansharpening using trained dictionary. IEEE Geosci Remote Sens Lett 11(1):293–297

    Article  Google Scholar 

  13. Choi J, Yu K, Kim Y (2011) A new adaptive component-substitution-based satellite image fusion by using partial replacement. IEEE Trans Geosci Remote Sens 49(1):295–309

    Article  Google Scholar 

  14. Demirel H, Anbarjafari G (2011) Discrete wavelet transform-based satellite image resolution enhancement. IEEE Trans Geosci Remote Sens 49(6):1997–2004

    Article  MATH  Google Scholar 

  15. Fan DP, Cheng MM, Liu JJ, Gao SH, Hou Q, Borji A (2018) Salient objects in clutter: Bringing salient object detection to the foreground. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y (eds) Computer vision – ECCV 2018. Springer International Publishing, Cham, pp 196–212

  16. Fei R, Zhang J, Liu J, Du F, Chang P, Hu J (2019) Convolutional sparse representation of injected details for pansharpening. IEEE Geosci Remote Sens Lett 16(10):1595–1599

    Article  Google Scholar 

  17. Gao Y, Song C, Yang C, Wang M, Yang W (2019) Pansharpening with joint local low rank decomposition and hierarchical geometric filtering. IEEE Access 99:1–13

    Google Scholar 

  18. Garzelli A, Nencini F (2009) Hypercomplex quality assessment of multi/hyperspectral images. IEEE Geosci Remote Sens Lett 6(4):662–665

    Article  Google Scholar 

  19. Garzelli A, Nencini F, Capobianco L (2008) Optimal mmse pan sharpening of very high resolution multispectral images. IEEE Trans Geosci Remote Sens 46(1):228–236

    Article  Google Scholar 

  20. Hallabia H, Hamam H, Hamida AB (2021) An optimal use of SCE-UA method cooperated with superpixel segmentation for pansharpening. IEEE Geosci Remote Sens Lett 18(9):1620–1624

    Article  Google Scholar 

  21. Hallabia H, Kallel A, Hamida AB (2018) Multiresolution filter banks for pansharpening application. In: Dolecek GJ (ed) Advances in multirate systems. Springer International Publishing AG 2018, pp 271–79

  22. Hallabia H, Kallel A, Hamida AB, Hégarat-Mascle SL (2016) High spectral quality pansharpening approach based on mtf-matched filter banks. Multidim Syst Sign Process 27(4):831–861

    Article  MATH  Google Scholar 

  23. He X, Condat L, Bioucas-Dias J, Chanussot J, Xia J (2014) A new pansharpening method based on spatial and spectral sparsity priors. IEEE Trans Image Process 23(9):4160–4174

    Article  MATH  Google Scholar 

  24. Imani M (2018) Band dependent spatial details injection based on collaborative representation for pansharpening. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, pp 1–11

  25. Kallel A (2015) MTF-Adjusted pansharpening approach based on coupled multiresolution decompositions. IEEE Trans Geosci Remote Sens 53 (6):3124–3145

    Article  Google Scholar 

  26. Khademi G, Ghassemian H (2018) Incorporating an adaptive image prior model into bayesian fusion of multispectral and panchromatic images. IEEE Geosci Remote Sens Lett 15(6):917–921

    Article  Google Scholar 

  27. Kim Y, Kim M, Choi J, Kim Y (2017) Image fusion of spectrally nonoverlapping imagery using spca and mtf-based filters. IEEE Geosci Remote Sens Lett 14(12):2295–2299

    Article  Google Scholar 

  28. Laben C, Brower B (2000) Process for enhancing the spatial resolution of multispectral imagery using pan-sharpening. US Patent 6,011,875

  29. Li H, Jing L, Tang Y, Wang L (2018) An image fusion method based on image segmentation for high-resolution remotely-sensed imagery. Remote Sens 10(5):790

    Article  Google Scholar 

  30. Li H, Wu X (2018) Infrared and visible image fusion using latent low-rank representation. CoRR arXiv:abs/1804.08992

  31. Li Z, Leung H (2009) Fusion of multispectral and panchromatic images using a restoration-based method. IEEE Trans Geosci Remote Sens 47 (5):1482–1491

    Article  Google Scholar 

  32. Liu G, Lin Z, Yu Y (2010) Robust subspace segmentation by low-rank representation. In: Proceedings of the 27th international conference on international conference on machine learning, ICML’10. Omnipress, USA, pp 663–670

  33. Liu G, Yan S (2011) Latent low-rank representation for subspace segmentation and feature extraction. In: 2011 International conference on computer vision, pp 1615–1622

  34. Liu JG (2000) Smoothing filter-based intensity modulation: A spectral preserve image fusion technique for improving spatial details. Int J Remote Sens 21(18):3461–3472

    Article  Google Scholar 

  35. Liu P, Xiao L, Li T (2018) A variational pan-sharpening method based on spatial fractional-order geometry and spectral–spatial low-rank priors. IEEE Trans Geosci Remote Sens 56(3):1788–1802

    Article  Google Scholar 

  36. Liu X, Zhao G, Yao J, Qi C (2015) Background subtraction based on low-rank and structured sparse decomposition. IEEE Trans Image Process 24(8):2502–2514

    Article  MATH  Google Scholar 

  37. Liu Y, Chen X, Wang Z, Wang ZJ, Ward RK, Wang X (2018) Deep learning for pixel-level image fusion: Recent advances and future prospects. Information Fusion 42:158–173

    Article  Google Scholar 

  38. Mahyari AG, Yazdi M (2011) Panchromatic and multispectral image fusion based on maximization of both spectral and spatial similarities. IEEE Trans Geosci Remote Sens 49(6):1976–1985

    Article  Google Scholar 

  39. Masi G, Cozzolino D, Verdoliva L, Scarpa G (2017) Cnn-based pansharpening of multi-resolution remote-sensing images. In: 2017 Joint urban remote sensing event (JURSE), pp 1–4

  40. Nunez J, Otazu X, Fors O, Prades A, Pala V, Arbiol R (1999) Multiresolution-based image fusion with additive wavelet decomposition. IEEE Trans Geosci Remote Sens 37(3):1204–1211

    Article  Google Scholar 

  41. Otazu X, González-Audícana M, Fors O, Nunez J (2005) Introduction of sensor spectral response into image fusion methods. application to wavelet-based methods. IEEE Trans Geosci Remote Sens 43(10):2376–2385

    Article  Google Scholar 

  42. Palsson F, Sveinsson JR, Ulfarsson MO, Benediktsson JA (2016) Mtf-based deblurring using a wiener filter for cs and mra pansharpening methods. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 9 (6):2255–2269

    Article  Google Scholar 

  43. Restaino R, Vivone G, Mura MD, Chanussot J (2016) Fusion of multispectral and panchromatic images based on morphological operators. IEEE Trans Image Process 25(6):2882–2895

    Article  MATH  Google Scholar 

  44. Rong K, Jiao L, Wang S, Liu F (2014) Pansharpening based on low-rank and sparse decomposition. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 7(12):4793–4805

    Article  Google Scholar 

  45. Scarpa G, Vitale S, Cozzolino D (2018) Target-adaptive cnn-based pansharpening. IEEE Trans Geosci Remote Sens 56(9):5443–5457

    Article  Google Scholar 

  46. Thomas C, Ranchin T, Wald L, Chanussot J (2008) Synthesis of multispectral images to high spatial resolution: A critical review of fusion methods based on remote sensing physics. IEEE Trans Geosci Remote Sens 46(5):1301–1312

    Article  Google Scholar 

  47. Tu TM, Huang PS, Hung CL, Chang CP (2004) A fast intensity-hue-saturation fusion technique with spectral adjustment for ikonos imagery. IEEE Geosci Remote Sens Lett 1(4):309–312

    Article  Google Scholar 

  48. Ulfarsson MO, Mura MD (2018) A low-rank method for sentinel-2 sharpening using cyclic descent. In: IGARSS 2018 - 2018 IEEE International geoscience and remote sensing symposium, pp 8857–8860

  49. Vicinanza MR, Restaino R, Vivone G, Mura MD, Chanussot J (2015) A pansharpening method based on the sparse representation of injected details. IEEE Geosci Remote Sens Lett 12(1):180–184

    Article  Google Scholar 

  50. Vivone G, Alparone L, Chanussot J, Dalla Mura M, Garzelli A, Licciardi G, Restaino R, Wald L (2015) A critical comparison among pansharpening algorithms. IEEE Trans Geosci Remote Sens 53(5):2565–2586

    Article  Google Scholar 

  51. Vivone G, Restaino R, Chanussot J (2018) Full scale regression-based injection coefficients for panchromatic sharpening. IEEE Trans Image Process 27 (7):3418–3431

    Article  MATH  Google Scholar 

  52. Vivone G, Restaino R, Chanussot J (2018) A regression-based high-pass modulation pansharpening approach. IEEE Trans Geosci Remote Sens 56 (2):984–996

    Article  MATH  Google Scholar 

  53. Wald L, Thierry R, Mangolini M (1997) Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images. Photogrammetric Engineering and Remote Sensing 63(6):691–699

    Google Scholar 

  54. Wang J, Shao Z, Huang X, Lu T, Zhang R (2022) A dual-path fusion network for pan-sharpening. IEEE Trans Geosci Remote Sens 60:1–14

    Google Scholar 

  55. Wang W, Liu H, Liang L, Liu Q, Xie G (2019) A regularised model-based pan-sharpening method for remote sensing images with local dissimilarities. Int J Remote Sens 40(8):3029–3054

    Article  Google Scholar 

  56. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: From error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  57. Ward CM, Harguess J, Crabb B, Parameswaran S (2019) Image quality assessment for determining efficacy and limitations of super-resolution convolutional neural network (SRCNN). CoRR arXiv:abs/1905.05373

  58. Wei Y, Yuan Q, Shen H, Zhang L (2017) Boosting the accuracy of multispectral image pansharpening by learning a deep residual network. IEEE Geosci Remote Sens Lett 14(10):1795–1799

    Article  Google Scholar 

  59. Wright J, Arvind G, Shankar R, Yigang P, Ma Y (2009) Robust principal component analysis: Exact recovery of corrupted low-rank matrices via convex optimization. In: Bengio Y, Schuurmans D, Lafferty JD, Williams CKI, Culotta A (eds) Advances in neural information processing systems 22. Curran Associates, Inc, pp 2080–2088

  60. Xu H, Ma J, Shao Z, Zhang H, Jiang J, Guo X (2021) SDPNEt: A deep network for pan-sharpening with enhanced information representation. IEEE Trans Geosci Remote Sens 59(5):4120–4134

    Article  Google Scholar 

  61. Yang S, Zhang K, Wang M (2018) Learning low-rank decomposition for pan-sharpening with spatial-spectral offsets. IEEE Transactions on Neural Networks and Learning Systems 29(8):3647–3657

    Article  Google Scholar 

  62. Yang Y, Wu L, Huang S, Sun J, Wan W, Wu J (2018) Compensation details-based injection model for remote sensing image fusion. IEEE Geosci Remote Sens Lett 15(5):734–738

    Article  Google Scholar 

  63. Yang Y, Wu L, Huang S, Wan W, Que Y (2018) Remote sensing image fusion based on adaptively weighted joint detail injection. IEEE Access 6:6849–6864

    Article  Google Scholar 

  64. Yin H (2015) Sparse representation based pansharpening with details injection model. Signal Process 113(5):218–227

    Article  Google Scholar 

  65. Yin H (2017) A joint sparse and low-rank decomposition for pansharpening of multispectral images. IEEE Trans Geosci Remote Sens 55(6):3545–3557

    Article  Google Scholar 

  66. Yuan X, Yang J (2009) Sparse and low-rank matrix decomposition via alternating direction methods. Tech rep

  67. Yuhas RH, Goetz AF, Boardman JW (1992) Discrimination among semi-arid landscape endmembers using the spectral angle mapper (sam) algorithm. In: JPL, summaries of the Third annual JPL airborne geoscience workshop, USA, pp 147–149

  68. Zhang G, Fang F, Zhou A, Li F (2015) Pan-sharpening of multi-spectral images using a new variational model. Int J Remote Sens 36(5):1484–1508

    Article  Google Scholar 

  69. Zhang L, Li A, Zhang Z, Yang K (2016) Global and local saliency analysis for the extraction of residential areas in high-spatial-resolution remote sensing image. IEEE Trans Geosci Remote Sens 54(7):3750–3763

    Article  Google Scholar 

  70. Zhang L, Zhang J (2017) A new saliency-driven fusion method based on complex wavelet transform for remote sensing images. IEEE Geosci Remote Sens Lett 14(12):2433–2437

    Article  Google Scholar 

  71. Zhang Y, Li H, Xiao L (2018) Multivariate regression-based pan-sharpening with low rank regularization. In: IGARSS 2018 - 2018 IEEE International geoscience and remote sensing symposium, pp 7188–7191

  72. Zhao J, Liu J, Fan D, Cao Y, Yang J, Cheng M (2019) Egnet: Edge guidance network for salient object detection. CoRR arXiv:abs/1908.08297

  73. Zhou J, Civco DL, Silander JA (1998) A wavelet transform method to merge landsat tm and spot panchromatic data. Int J Remote Sens 19(4):743–757

    Article  Google Scholar 

  74. Zhu XX, Bamler R (2013) A sparse image fusion algorithm with application to pan-sharpening. IEEE Trans Geosci Remote Sens 51(5):2827–2836

    Article  Google Scholar 

Download references

Acknowledgements

The authors express their sincere gratitude to Dr. G. Vivone for providing the CS-D code.

Author information

Authors and Affiliations

  1. Laboratoire d’Iformatique et Systèmes, Équipe de modélisation géométrique (G-Mod), Aix-Marseille Université, Arles, France

    Hind Hallabia

  2. Faculty of Engineering, University of Moncton, Moncton, NB, Canada

    Habib Hamam

  3. ATMS, University of Sfax, Sfax, Tunisia

    Ahmed Ben Hamida

Authors
  1. Hind Hallabia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Habib Hamam
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ahmed Ben Hamida
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hind Hallabia.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hallabia, H., Hamam, H. & Hamida, A.B. A novel detail injection framework using latent low-rank decomposition for multispectral pan-sharpening. Multimed Tools Appl 82, 5987–6012 (2023). https://doi.org/10.1007/s11042-022-12770-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12770-x

Keywords

Search

Navigation