Skip to main content
SpringerLink
Log in

Superpixel guided structure sparsity for multispectral and hyperspectral image fusion over couple dictionary

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper proposed a hyperspectral (HS) and multispectral (MS) image fusion method based on superpixel guided structure sparsity and couple dictionary (SGSSCD). It is assumed that the pixels in a homogeneous area of MS image are similar due to the consistent spatial consistency. Superpixel technique is used to find the similar pixels in MS image by considering the region homogeneity. Then these pixels naturally share the same atoms in low spectral resolution dictionary. In order to capture the similarity prior, structural sparsity is employed to find more efficient coding of the similar pixels in MS image over low spectral resolution dictionary. Finally, high spatial resolution HS image can be produced by combining the codes of MS image with high spectral resolution dictionary. Besides, the couple dictionary is learned from HS and low spatial resolution MS images to ensure the spectral correspondence, which can further improve the quality of fusion results. The experimental results on different datasets demonstrate the effectiveness of the proposed method when compared with some existing methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Achanta R, Shaji A, Smith K et al (2012) SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 11:2274–2282

    Article  Google Scholar 

  2. Aharon M, Elad M, Bruckstein A (2006) K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans Signal Process 11:4311–4322

    Article  Google Scholar 

  3. Aiazzi B, Baronti S, Lotti F et al (2009) A comparison between global and context-adaptive pansharpening of multispectral images. IEEE Geosci Remote Sens Lett 2:302–306

    Article  Google Scholar 

  4. Akhtar N, Shafait F, Mian A (2014) Sparse spatio-spectral representation for hyperspectral image super-resolution. Proc ECCV, pp 63–78

  5. Akhtar N, Shafait F, Mian A (2015) Bayesian sparse representation for hyperspectral image super resolution. Proc IEEE CVPR, pp 3631–3640

  6. Amro I, Mateos J, Vega M et al (2011) A survey of classical methods and new trends in pansharpening of multispectral images. EURASIP J Advan Signal Process 2011(1):79

    Article  Google Scholar 

  7. Ankarao V, Sowmya V, Soman KP (2018) Multi-sensor data fusion using NIHS transform and decomposition algorithms. Multimed Tools Appl 23:30381–30402

    Article  Google Scholar 

  8. Bioucasdias J, Plaza A, Campsvalls G et al (2013) Hyperspectral remote sensing data analysis and future challenges. IEEE Geosci Remote Sens Mag 1(2):6–36

    Article  Google Scholar 

  9. Campsvalls G, Bruzzone L (2005) Kernel-basted methods for hyperspectral image classification. IEEE Trans Geosci Remote Sens 6:1351–1362

    Article  Google Scholar 

  10. Candès EJ, Li X, Ma Y, Wright J (2009) Robust principal component analysis? J ACM 1:1–37

    MATH  Google Scholar 

  11. Chang X, Ma Z, Lin M et al (2017) Feature interaction augmented sparse learning for fast kinect motion detection. IEEE Trans Image Process 8:3911–3920

    Article  MathSciNet  Google Scholar 

  12. Chavez PS, Sides SC, Anderson JA (1991) Comparison of three different methods to merge multiresolution and multispectral data: Landsat TM and SPOT panchromatic. Photogramm Eng Remote Sens 3:265–303

    Google Scholar 

  13. Chen HY, Leou JJ (2012) Multispectral and multiresolution image fusion using particle swarm optimization. Multimed Tools Appl 3:495–518

    Article  Google Scholar 

  14. Chen Z, Pu HY, Wang B et al (2014) Fusion of hyperspectral and multispectral images: a novel framework based on generalization of pan-sharpening methods. IEEE Geosci Remote Sens Lett 8:1418–1422

    Article  Google Scholar 

  15. Conn AR, Gould NIM, Toint PL (1991) A globally convergent augmented Lagrangian algorithm for optimization with general constraints and simple bounds. Siam J Num Anal 2:545–572

    Article  MathSciNet  Google Scholar 

  16. Hancock GR, Freeman MJ (2001) Power and sample size for the root mean square error of approximation test of not close fit in structural equation modeling. Educ Psychol Meas 5:741–758

    Article  MathSciNet  Google Scholar 

  17. Kanmani M, Narasimhan V (2016) An optimal weighted averaging fusion strategy for thermal and visible images using dual tree discrete wavelet transform and self tunning particle swarm optimization. Multimed Tools Appl 20:1–22

    Google Scholar 

  18. Kawakami R, Wright J, Tai YW (2011) High-resolution hyperspectral imaging via matrix factorization. Proc IEEE CVPR, pp 2329–2336

  19. Kunkel B, Blechinger F, Viehmann D et al (1991) ROSIS imaging spectrometer and its potential for ocean parameter measurements (airborne and space-borne). Int J Remote Sens 4:753–761

    Article  Google Scholar 

  20. Li Z, Nie F, Chang X et al (2017) Beyond trace ratio: weighted harmonic mean of trace ratios for multiclass discriminant analysis. IEEE Trans Knowl Data Eng 10:2100–2110

    Article  Google Scholar 

  21. Lin B, Tao X, Duan Y et al (2018) Hyperspectral and multispectral image fusion based on low rank constrained gaussian mixture model. IEEE Access 6:16901–16910

    Article  Google Scholar 

  22. Liu M Y, Tuzel O, Ramalingam S et al (2011) Entropy rate superpixel segmentation. Proc IEEE CVPR, pp 2097–2104

  23. Mangalraj P, Agrawal A (2017) Novel approach on fusion of multisensor images based on neutrosophic domain in consideration of regional relevance. Multimed Tools Appl 1:1–18

    Google Scholar 

  24. Nasrabadi NM (2013) Hyperspectral target detection: an overview of current and future challenges. IEEE Signal Process Mag 1:34–44

    Google Scholar 

  25. Nezhad ZH, Karami A, Heylen R et al (2016) Fusion of hyperspectral and multispectral images using spectral unmixing and sparse coding. IEEE J Sel Topics Appl Earth Obser Remote Sens 6:2377–2389

    Article  Google Scholar 

  26. Paramanandham N, Rajendiran K (2017) Multi sensor image fusion for surveillance applications using hybrid image fusion algorithm. Multimed Tools Appl 7:1–32

    Google Scholar 

  27. Schweizer SM, Moura JMF (2001) Efficient detection in hyperspectral imagery. IEEE Trans Image Process 4:584–597

    Article  Google Scholar 

  28. Selva M, Aiazzi B, Butera F et al (2017) Hyper-sharpening: a first approach on SIM-GA data. IEEE J Sel Topics Appl Earth Obser Remote Sens 6:3008–3024

    Google Scholar 

  29. Shen J, Du Y, Wang W et al (2014) Lazy random walks for superpixel segmentation. IEEE Trans Image Process 4:1451–1462

    Article  MathSciNet  Google Scholar 

  30. Stein DWJ, Beaven SG, Hoff LE et al (2002) Anomaly detection from hyperspectral imagery. IEEE Signal Process Mag 1:58–69

    Article  Google Scholar 

  31. Veganzones MA, Simoes M, Licciardi G et al (2016) Hyperspectral super-resolution of locally low rank images from complementary multisource data. IEEE Trans Image Process 1:274–288

    Article  MathSciNet  Google Scholar 

  32. Vivone G, Alparone L, Chanussot J et al (2015) A critical comparison among pansharpening algorithms. IEEE Trans Geosci Remote Sens 5:2565–2586

    Article  Google Scholar 

  33. Wald L (2000) Quality of high resolution synthesised images: is there a simple criterion? Proc. Int. Conf. Fusion Earth Data, Nice, France, pp 99–103

  34. Wald L, Ranchin T, Mangolini M (1997) Fusion of satellite images of different spatial resolutions: assessing the quality of resulting images. Photogramm Eng Remote Sens 63:691–699

    Google Scholar 

  35. Wang Z, Bovik AC (2002) A universal image quality index. IEEE Signal Process Lett 3:81–84

    Article  Google Scholar 

  36. Wei J, Huang Y (2017) NMPE: a normalized metric for measuring generalized spatial distortion of multispectral panshapening fusion. Multimed Tools Appl 4:1–18

    Google Scholar 

  37. Wei Q, BioucasDias José M, Dobigeon N et al (2015) Hyperspectral and multispectral image fusion based on a sparse representation. IEEE Trans Geosci Remote Sens 7:3658–3668

    Article  Google Scholar 

  38. Wei Q, Dobigeon N, Tourneret JY (2015) Fast fusion of multi-band images based on solving a Sylvester equation. IEEE Trans Image Process 11:1632–1636

    MathSciNet  MATH  Google Scholar 

  39. Yokoya N, Yairi T, Iwasaki A (2012) Coupled nonnegative matrix factorization unmixing for hyperspectral and multispectral data fusion. IEEE Trans Geosci Remote Sens 2:528–537

    Article  Google Scholar 

  40. Yuhas RH, Goetz AFH, Boardman JW (1992) Discrimination among semi-arid landscape endmembers using the Spectral Angle Mapper (SAM) algorithm. Proc. Summaries 4th JPL Airborne Earth Sci. Workshop, pp 147–149

  41. Zhang K, Wang M, Yang S (2017) Multispectral and hyperspectral image fusion based on group spectral embedding and low-rank factorization. IEEE Trans Geosci Remote Sens 3:1363–1371

    Article  Google Scholar 

  42. Zhang K, Wang M, Yang S et al (2018) Spatial–spectral-graph-regularized low-rank tensor decomposition for multispectral and hyperspectral image fusion. IEEE J Sel Topics Appl Earth Obser Remote Sens 4:1030–1040

    Article  Google Scholar 

  43. Zhu XX, Bamler R (2013) A sparse image fusion algorithm with application to pan-sharpening. IEEE Trans Geosci Remote Sens 5:2827–2836

    Article  Google Scholar 

  44. Zhu XX, Claas G, Bamler R (2016) Exploiting joint sparsity for pansharpening: the J-SparseFI algorithm. IEEE Trans Geosci Remote Sens 5:2664–2681

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Shandong Normal University, Jinan, 250014, Shandong Province, China

    Feng Zhang & Kai Zhang

  2. China Mobile Online Services Company Limited, Zhengzhou, 450000, Henan, China

    Feng Zhang

Authors
  1. Feng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kai Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kai Zhang.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, F., Zhang, K. Superpixel guided structure sparsity for multispectral and hyperspectral image fusion over couple dictionary. Multimed Tools Appl 79, 4949–4964 (2020). https://doi.org/10.1007/s11042-019-7188-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-7188-1

Keywords

Search

Navigation