Skip to main content
SpringerLink
Log in

Pansharpening Image Fusion Using Cross-Channel Correlation: A Framelet-Based Approach

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

This paper aims at developing a variational model for pansharpening image fusion. To resolve the ill-posedness of image fusion, we propose a new regularization technique that explores the cross-channel correlation of different spectral channels in wavelet tight frame (or framelet) domain. Besides using a regular cross-channel sparsity prior (Inverse Probl Imaging 7(3):777–794, 2013), the proposed model also makes efficient use of the panchromatic image as a guidance for image feature alignment. An ADMM-based iterative scheme is derived for solving the proposed model, and its performance is tested on several datasets including natural images, aerial images, and real multispectral satellite images. Numerical results suggest that the proposed approach works well on the testing datasets and outperforms some state-of-the-art algorithms in comparison.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. In many papers and textbooks, tight frame only implies \(A=B\).

  2. http://www.ipol.im.

  3. http://sipi.usc.edu/database/.

  4. http://www.satimagingcorp.com/gallery/quickbird/.

References

  1. Amro, I., Mateos, J., Vega, M., Molina, R., Katsaggelos, A.K.: A survey of classical methods and new trends in pansharpening of multispectral images. EURASIP J. Adv. Signal Process. 2011, 79 (2011)

    Article  Google Scholar 

  2. Ballester, C., Caselles, V., Igual, L., Verdera, J.: A variational model for P+XS image fusion. Int. J. Comput. Vis. 69, 43–58 (2006)

    Article  Google Scholar 

  3. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–124 (2011)

    Article  MATH  Google Scholar 

  4. Cai, J., Chan, R., Shen, Z.: A framelet-based inpainting algorithm. Appl. Comput. Harmon. Anal. 24, 131–149 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  5. Cai, J., Osher, S., Shen, Z.: Split Bregman methods and frame based image restoration. Multiscale Model. Simul. 8(2), 337–369 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  6. Cai, J., Ji, H., Liu, C., Shen, Z.: Framelet based blind image deblurring from a single image. IEEE Trans. Image Process. 21(2), 562–572 (2012)

    Article  MathSciNet  Google Scholar 

  7. Chan, R.H., Chan, T.F., Shen, L., Shen, Z.: Wavelet algorithms for high-resolution image reconstrction. SIAM J. Sci. Comput. 24(4), 1408–1432 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  8. Daubechies, I., Han, B., Ron, A., Shen, Z.: Framelets: MRA-based constructions of wavelet frames. Appl. Comput. Harmon. Anal. 14, 1–46 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  9. DigitalGlobe. 2002. Sundarbans Reserved Forest. ftp://ftp.glcf.umd.edu/glcf/QuickBird/02DEC04043843-X2AS_R2C1-000000185940_01_P002-Bangladesh-The_Sundarbans. Accessed 25 June 2015

  10. Dong, B., Shen, Z.: MRA-based Wavelet Frames and Applications. IAS/Park City Mathematics Series, vol. 19. American Mathematical Society, Providence (2010)

    Google Scholar 

  11. Duran, J., Buades, A., Coll, B., Sbert, C.: Implementation of nonlocal pansharpening image fusion. Image Process. Line (IPOL) 4, 1–15 (2014a)

    Article  MATH  Google Scholar 

  12. Duran, J., Buades, A., Coll, B., Sbert, C.: A nonlocal variational model for pansharpening image fusion. SIAM J. Imaging Sci. 7(2), 761–796 (2014b)

    Article  MathSciNet  MATH  Google Scholar 

  13. Goldstein, T., Osher, S.: The split Bregman iteration for \(L\)1-regularized problems. SIAM J. Imaging Sci. 2(2), 323–343 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  14. Gong, Z., Shen, Z., Toh, K.C.: Image restoration with mixed or unknown noises. Multiscale Model. Simul. 12(2), 458–487 (2014)

  15. Haydan, R., Dalke, G.W., Henkel, J., Bare, J.E.: Applications of the IHS color transform to the processing of multisensor data and image enhancement. In: Proceedings of the 1st international symposium on remote sensing environment, Cairo, Egypt, pp. 599–616 (1982)

  16. Hestenes, M.R.: Multiplier and gradient methods. J. Optim. Theory Appl. 4, 303–320 (1969)

    Article  MathSciNet  MATH  Google Scholar 

  17. Hamilton Jr, J., Adams Jr J.: Adaptive color plan interpolation in single sensor color electronic camera. U.S. Patent 5, 629–734 (1997)

  18. Kakarala, R., Baharav, Z.: Adaptive demosaicing with the principal vector method. IEEE Trans. Consum. Electron. 48(4), 932–937 (2002)

    Article  Google Scholar 

  19. Liang, J., Li, J., Shen, Z., Zhang, X.: Wavelet frame based color image demosaicing. Inverse Probl Imaging 7(3), 777–794 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  20. Mallat, S.: A Wavelet Tour of Signal Processing, 2nd edn. Academic Press Incorporation, San Diego (1998)

    MATH  Google Scholar 

  21. Möller, M., Wittman, T., Bertozzi, A.L., Burger, M.: A variational approach for sharpening high dimensional images. SIAM J. Imaging Sci. 5(1), 150–178 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  22. Pohl, Christine, van Genderen, John: Structuring contemporary remote sensing image fusion. Int. J. Image Data Fusion 6(1), 3–21 (2015)

    Article  Google Scholar 

  23. Powell, M.J.D.: A method for nonlinear constraints in minimization problems. In: Fletcher, R. (ed.) Optimization, pp. 283–298. Academic Press, New York (1969)

    Google Scholar 

  24. Ron, A., Shen, Z.: Frames and stable bases for shift-invariant subspaces of \(L_2(R^d)\). Can. J. Math. 47, 1051–1094 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  25. Ron, A., Shen, Z.: Affine systems in \(L_2(R^d)\): the analysis of the analysis operator. J. Funct. Anal. 148, 408–447 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  26. SpaceImaging.: IKONOS scene (2000) ftp://ftp.glcf.umd.edu/glcf/China_earthquake_May_2008/IKONOS/po_58205_0000000.20001003.China-Sichuan. Accessed 1 July 2015

  27. Tai, C., Zhang, X., Shen, Z.: Wavelet frame based multi-phase image segmentation. SIAM J. Imaging Sci. 6, 2521–2546 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  28. Thomas, C., Ranchin, T., Wald, L., Chanussot, J.: Synthesis of multispectral images to high spatial resolution: a critical review of fusion methods based on remote sensing physics. IEEE Trans. Geosci. Remote Sens. 46(5), 1301–1312 (2008)

    Article  Google Scholar 

  29. Tsai, C.Y., TaiSong, K.: Heterogeneity-projection hard-decision color interpolation using spectral-spatial correlation. IEEE Trans. Image Process. 16(1), 78–97 (2007)

    Article  MathSciNet  Google Scholar 

  30. Wu, C., Tai, X.C.: Augmented lagrangian method, dual methods, and split bregman iteration for rof, vectorial tv, and high order models. SIAM J. Imaging Sci. 3(3), 300–339 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  31. Zhang, L., Wu, X.: Color demosaicking via directional linear minimum mean square-error estimation. IEEE Trans. Image Process. 14(12), 2167–2178 (2005)

    Article  Google Scholar 

  32. Zhang, L., Wu, X., Buades, A., Li, X.: Color demosaicking by local directional interpolation and non-local adaptive thresholding. J. Electron. Imaging 20(2), 023016 (2011)

    Article  Google Scholar 

Download references

Acknowledgments

L. Hou and X. Zhang would like to thank the anonymous reviewers for their helpful comments in improving the presentation of this paper. They also would like to thank the authors of [11, 12] for making their resources available online for free academic use. This work is partially supported by NSFC91330102 and NSFC (GZ1025), China Postdoc Science Foundation (# 2014M551392), 973 Program (# 2015CB856000).

Author information

Authors and Affiliations

  1. Department of Mathematics & Institute of Natural Sciences, Shanghai Jiao Tong University, Shanghai, 200240, China

    Likun Hou & Xiaoqun Zhang

Authors
  1. Likun Hou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoqun Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaoqun Zhang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hou, L., Zhang, X. Pansharpening Image Fusion Using Cross-Channel Correlation: A Framelet-Based Approach. J Math Imaging Vis 55, 36–49 (2016). https://doi.org/10.1007/s10851-015-0612-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-015-0612-x

Keywords

Search

Navigation