Skip to main content

Advertisement

SpringerLink
Log in

Medical image fusion algorithm based on L0 gradient minimization for CT and MRI

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In this paper, a novel medical image fusion method based on L0 Gradient Minimization for CT and MRI is proposed. Compared with traditional algorithms, the proposed method performs well in preserving bones structures from CT and sustaining the soft tissue detail from MRI. It’s worth mentioning that both the proposed low- and high-frequency fusion rules have the capability of generating appropriate weight maps according to the characteristics of CT and MRI images. The fusion algorithm using L0 Gradient Minimization mainly comprises of four steps: First, source images are decomposed into multi-scale representations via L0 Gradient Minimization. Second, we propose a low-frequency fusion rule based on local energy and Gaussian filters, which can generate the fused base layer in accord with the basic principle of human beings’ visual system. Third, high-frequency sub-bands are fused by utilizing saliency detection rule based on texture extraction, which generates the satisfying maps according to the degree of significance. Finally, we get the fused result according to the image reconstruction. The proposed algorithm is compared with nine advanced fusion methods and shows superior performance in whether subjective or objective evaluations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

References

  1. Blum R, Liu Z (2005) Multi-sensor image fusion and its applications

  2. Choi M (2006) A new intensity-hue-saturation fusion approach to image fusion with a tradeoff parameter. IEEE Trans Geosci Remote Sens 44(6):1672–1682

    Article  Google Scholar 

  3. DOGRA A (2017) From Multi-Scale decomposition to Non-Multi-Scale decomposition methods: a comprehensive survey of image fusion techniques and its applications. IEEE Access 5:16040–16067

    Article  Google Scholar 

  4. Dorgham O, Al-Rahamneh B, Almomani A, Al-Hadid M, Knatatneh K (2018) Enhancing the Security of Exchanging and Storing DICOM Medical Images on the Cloud. 8(1):154–172

  5. Farbman Z, Fattal R, Lischinski D, Szeliski R (2008) Edge-preserving decompositions for multi-scale tone and detail manipulation. ACM Trans Graph 27(3):67

    Article  Google Scholar 

  6. Ganasala P, Kumar V (2014) CT And MR image fusion scheme in nonsubsampled contourlet transform domain. J Digit Imaging 27(3):407–418

    Article  Google Scholar 

  7. Gastal E, Oliveira M (2011) Domain transform for Edge-Aware image and video processing. ACM Trans Graph 30(4):69

    Article  Google Scholar 

  8. Ghoneim A, Muhammad G, Amin U, Gupta B (2018) Medical image forgery detection for smart healthcare. IEEE Commun Mag 56(4):33–37

    Article  Google Scholar 

  9. Guo P, Evans A, Bhattacharya P (2018) Nuclei segmentation for quantification of brain tumors in digital pathology images. Int J Softw Ence Comput Intell 10(2):36–49

    Article  Google Scholar 

  10. Gutman I, Zhou B (2006) Laplacian energy of a graph. Linear Algebra Appl 414(1):29–37

    Article  MathSciNet  MATH  Google Scholar 

  11. Han Y, Cai Y, Xu X (2013) A new image fusion performance metric based on visual information fidelity. Inform Fusion 14(2):127–135

    Article  Google Scholar 

  12. Harish K, Singh D (2010) Quality assessment off used image of MODIS and PALSAR. Progress Electromagnet Res B 24(24):191–221

    Article  Google Scholar 

  13. He K, Sun J, Tang X (2010) Guided image Filtering. In: Computer vision-ECCV 6311

  14. Kang X (2013) Image fusion with guided filtering. IEEE Trans Image Process 22(7):2864–2875

    Article  Google Scholar 

  15. Kumar B (2015) Image fusion based on pixel significance using cross bilateral filter. Signal Image Video Process 9(5):1193–1204

    Article  Google Scholar 

  16. Kumar M, Dass S (2009) A total variation-based algorithm for pixel- level image fusion. IEEE Trans Image Process 18(9):2137–2143

    Article  MathSciNet  MATH  Google Scholar 

  17. Li X, Guo X, Han P, Wang X, Li H, Luo T (2020) Laplacian redecomposition for multimodal medical image fusion. IEEE Trans Instrum Meas 69(9):6880–6890

    Article  Google Scholar 

  18. Li S, Kwok J, Wang Y (2001) Combination of images with diverse focuses using the spatial frequency. Inform Fusion 2(3):169–176

    Article  Google Scholar 

  19. Li B, Peng H, Wang J (2021) A novel fusion method based on dynamic threshold neural P systems and nonsubsampled contourlet transform for multi-modality medical images. Signal Process 178:107793

    Article  Google Scholar 

  20. Li H, Qi X, Xie W (2020) Fast infrared and visible image fusion with structural decomposition. Knowl-Based Syst 204:106182

    Article  Google Scholar 

  21. Li T, Wang Y (2011) Biological image fusion using a NSCT based variable-weight method. Informat Fusion 12(2):85–92

    Article  Google Scholar 

  22. Li T, Wang Y (2011) Biological image fusion using a NSCT based variable- weight method. Inform Fusion 12(2):85–92

    Article  Google Scholar 

  23. Li S, Yang B (2008) Multifocus image fusion using region segmentation and spatial frequency. Image Vis Comput 26(7):971–979

    Article  Google Scholar 

  24. Liu Y, Chen X (2019) Medical image fusion via convolutional sparsity based morphological component analysis. IEEE Signal Process Lett 26(3):485–489

    Article  Google Scholar 

  25. Liu Y, Chen X, Chen J, Peng H (2017) A medical image fusion method based on convolutional neural networks. In: 20th international conference on information fusion (fusion)

  26. Liu H, Guo Q, Wang G, Gupta B, Zhang C (2019) Medical image resolution enhancement for healthcare using nonlocal self-similarity and low-rank prior. Multimed Tools Appl 78(7):9033–9050

    Article  Google Scholar 

  27. Liu Y, Liu S, Wang Z (2015) Multi-focus image fusion with dense SIFT. Inform Fusion 23:139–155

    Article  Google Scholar 

  28. Melkemi K, Golea N (2017) ROI-Based fragile watermarking for medical image tamper detection. Int J High Perform Comput Netw 1(1):1

    Article  Google Scholar 

  29. Nayar S, Nakagawa Y (1994) Shape from focus. IEEE Trans Pattern Anal Machine Intell 16(8):824–831

    Article  Google Scholar 

  30. Petrovic V, Xydeas C (2005) Objective evaluation of signal-level image fusion performance. Opt Eng 44(8):087003

    Article  Google Scholar 

  31. Piella G (2003) A general framework for multiresolution image fusion: From pixels to regions. Information Fusion 4(4):259–280

    Article  Google Scholar 

  32. Portilla J, Strela V, Wainwright M, Simoncelli E (2001) Adaptive Wiener denoising using a Gaussian scale mixture model in the wavelet domain. In: International conference on image processing IEEE, vol 2, pp 37–40

  33. Pradhan P, King R, Younan N, Holcomb D (2006) Estimation of the number of decomposition levels for a wavelet-based multiresolution multi-sensor image fusion. IEEE Trans Geosci Remote Sens 44(12):3674–3686

    Article  Google Scholar 

  34. Qu X, Yan J, Yang G (2009) Multifocus image fusion method of sharp frequency localized Contourlet transform domain based on sum-modified-Laplacian. Opt Precision Eng 17(5):1203–1212

    Google Scholar 

  35. Qu G, Zhang D, Yan P (2002) Information measure for performance of image fusion. Electron Lett 38(7):313–315

    Article  Google Scholar 

  36. Rockinger O (1997) Image sequence fusion using a shift-invariant wavelet transform. In: International conference on image processing (ICIP), pp 288–291

  37. Rokni K, Ahmad A, Solaimani K, Hazini S (2015) A new approach for surface water change detection: Integration of pixel level image fusion and image classification techniques. Int J Appl Earth Observ Geoinformat 34:226–234

    Article  Google Scholar 

  38. Shen R, Cheng I, Basu A (2013) Cross-scale coefficient selection for volumetric medical image fusion. IEEE Trans Biomed Eng 60(4):1069–1079

    Article  Google Scholar 

  39. Subbarao M (1993) Focusing techniques. Opt Eng 32(11):2824–2836

    Article  Google Scholar 

  40. Subr K, Soler C, Durand F (2009) Edge-preserving multiscale image decomposition based on local extrema. ACM Trans Graph 28(5):147

    Article  Google Scholar 

  41. Toet A, Walraven J (1996) New false color mapping for image fusion. Opt Eng 35(3):650–658

    Article  Google Scholar 

  42. Tomasi C, Manduchi R (1998) Bilateral filtering for gray and color images. In: International conference on computer vision. IEEE

  43. Wald L (1999) Some terms of reference in data fusion. IEEE Trans Geosci Remote Sens 37(3):1190–1193

    Article  Google Scholar 

  44. Wang Z, Li X, Zhang X (2019) Multifocus image fusion using convolutional neural networks in the discrete wavelet transform domain. Multimed Tools Appl 78(24):34483–34512

    Article  Google Scholar 

  45. Wang M, Shang X (2020) A fast image fusion with discrete cosine transform. IEEE Signal Process Lett 27:990–994

    Article  Google Scholar 

  46. Xu Z (2014) Medical image fusion using multi-level local extrema. Inform Fusion 19:38–48

    Article  Google Scholar 

  47. Xu L, Lu C, Xu Y, Jia J (2011) Image smoothing via L0 gradient minimization. ACM Trans Graph 30(6):174

    Article  Google Scholar 

  48. Xu J, Yang L, Wu D (2010) Ripplet: A new transform for image processing. J Vis Commun Image Represent 21(7):627–639

    Article  Google Scholar 

  49. Xydeas C, Petrovic V (2000) Objective image fusion performance measure. Electron Lett 36(4):308–309

    Article  Google Scholar 

  50. Yang B, Jing Z, Zhao H (2010) Review of pixel-level image fusion. J Shanghai Jiaotong Univ Science) 15(01):6–12

    Article  Google Scholar 

  51. Ye F, Li X, Zhang X (2019) FusionCNN: A remote sensing image fusion algorithm based on deep convolutional neural networks. Multimed Tools Appl 78(11):14683–14703

    Article  Google Scholar 

  52. Zhang X, Li X, Liu Z, Feng Y (2014) Multi-focus image fusion using image-partition-based focus detection. Signal Process 102:64–76

    Article  Google Scholar 

  53. Zhang X, Li X, Liu Z, Feng Y (2014) Multi-focus image fusion using image-partition-based focus detection. Signal Process 102:64–76

    Article  Google Scholar 

  54. Zhang X, Li X, Liu Z, Feng Y (2017) Image fusion based on simultaneous empirical wavelet transform. Multimed Tools Appl 76(6):8175–8193

    Article  Google Scholar 

  55. Zhou W, Conrad B, Rahim S (2004) Image quality assessment: From error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

Download references

Acknowledgements

The work was supported in part by the National Science and Technology Pillar Program of China under Grant 2012BAH48F02, in part by the National Natural Science Foundation of China under Grant 61801190, in part by the Nature Science Foundation of Jilin Province under Grant 20180101055JC, in part by the Outstanding Young Talent Foundation of Jilin Province under Grant 20180520029JH, in part by the China Postdoctoral Science Foundation under Grant 2017M611323, in part by the Industrial Technology Research and Development Funds of Jilin Province under Grant 2019C054-3, in part by Graduate Innovation Fund of Jilin University, and in part by the Fundamental Research Funds for the Central Universities, JLU.

Author information

Authors and Affiliations

  1. Key Laboratory of Symbolic Computation and Knowledge Engineering, Ministry of Education, Jilin University, Changchun, 130012, China

    Siqi Zhang, Xiongfei Li, Rui Zhu, Zeyu Wang & Shuhan Zhang

  2. College of Computer Science and Technology, Jilin University, Changchun, 130012, China

    Xiongfei Li, Rui Zhu, Xiaoli Zhang, Zeyu Wang & Shuhan Zhang

Authors
  1. Siqi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiongfei Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rui Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaoli Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zeyu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Shuhan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaoli Zhang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, S., Li, X., Zhu, R. et al. Medical image fusion algorithm based on L0 gradient minimization for CT and MRI. Multimed Tools Appl 80, 21135–21164 (2021). https://doi.org/10.1007/s11042-021-10596-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-10596-7

Keywords

Search

Navigation