Skip to main content
SpringerLink
Log in

A fusion algorithm for medical structural and functional images based on adaptive image decomposition

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Multimodal medical image fusion has been widely used as a powerful tool in the clinical applications because of its ability of enriching information of medical images. In this paper, a novel fusion algorithm dedicated to medical structural and functional image fusion. In the algorithm, textures from functional images are separated from the smooth component in structural images; then we need to segment the source images into two parts function-informative and function-uninformative regions by judging whether each pixel in the functional image contains informative color (black pixel is meaningless); then get the smooth version of fused image by filling the function-informative region with the color from functional image and filling the function-uninformative region with smooth component in structural images; finally, smooth version of fused image and textures from the functional image are combined to get the final fused image. The attractive features of the algorithms include its ability of both color and texture reservation and low time consumption. Experimental results demonstrate that the proposed method can obtain state-of-the-art performance for medical structural and functional image fusion.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Ali F, El-Dokany I, Saad A, Abd El-Samie FE-S (2008) Curvelet fusion of MR and CT images. Prog Electromagn Res 3:215–224

    Article  MATH  Google Scholar 

  2. Avola D, Bernardi M, Foresti GL (2019) Fusing depth and colour information for human action recognition. Multimed Tools Appl 78(5):5919–5939

    Article  Google Scholar 

  3. Bhatnagar G, Wu Q, Liu Z (2013) Directive contrast based multimodal medical image fusion in NSCT domain. IEEE Trans Multimed 15(5):1014–1024

    Article  Google Scholar 

  4. Bhatnagar G, Wu QMJ, Liu Z (2013) Human visual system inspired multi-modal medical image fusion framework. Expert Syst Appl 40(5):1708–1720

    Article  Google Scholar 

  5. Bhatnagar G, Wu QMJ, Liu Z (2015) A new contrast based multimodal medical image fusion framework. Neurocomputing 157:143–152

    Article  Google Scholar 

  6. Calvi GG, Kisil I, Mandic DP (2018) Feature fusion via tensor network summation. In: 26th European Signal Processing Conference (EUSIPCO), Rome, Italy, pp 2623–2627

  7. Chen T, Zhang J, Zhang Y (2005) Remote sensing image fusion based on ridgelet transform. In: IEEE International Geoscience and Remote Sensing Symposium, Coex, Seoul, Korea, pp 1150–1153

  8. Choi M (2006) A new intensity-hue-saturation fusion approach to image fusion with a tradeoff parameter. IEEE Trans Geosci Remote Sens 44(6):1672–1682

    Article  Google Scholar 

  9. Daneshvar S, Ghassemian H (2010) MRI and PET image fusion by combining IHS and retina-inspired models. Inf Fusion 11(2):114–123

    Article  Google Scholar 

  10. Das S, Kundu MK (2012) NSCT-based multimodal medical image fusion using pulse-coupled neural network and modified spatial frequency. Med Biol Eng Comput 50(10):1105–1114

    Article  Google Scholar 

  11. Das S, Kundu MK (2013) A neuro-fuzzy approach for medical image fusion. IEEE Trans Biomed Eng 60(12):3347–3353

    Article  Google Scholar 

  12. Ding M, Wei L, Wang B (2015) Research on fusion method for infrared and visible images via compressive sensing. Infrared Phys Technol 57(0):56–67

    Google Scholar 

  13. Farbman Z, Fattal R, Lischinski D, Szeliski R (2008) Edge-preserving decompositions for multi-scale tone and detail manipulation. ACM Trans Graph (TOG) 27(3):15–19

    Article  Google Scholar 

  14. Girardi D, Küng J, Kleiser R, Sonnberger M, Csillag D, Trenkler J, Holzinger A (2016) Interactive knowledge discovery with the doctor-in-the-loop: a practical example of cerebral aneurysms research. Brain Informatics 3(3):133–143

    Article  Google Scholar 

  15. González-Audícana M, Saleta JL, Catalán RG, García R (2004) Fusion of multispectral and panchromatic images using improved IHS and PCA mergers based on wavelet decomposition. IEEE Trans Geosci Remote Sens 42(6):1291–1299

    Article  Google Scholar 

  16. Harikumar V, Gajjar PP, Joshi MV, Raval MS (2014) Multiresolution image fusion: use of compressive sensing and graph cuts. IEEE J Sel Top Appl Earth Observ Remote Sens 7(5):1771–1780

    Article  Google Scholar 

  17. Holzinger A (2016) Interactive machine learning for health informatics: when do we need the human-in-the-loop? Brain Informatics 3(2):119–131

    Article  Google Scholar 

  18. Holzinger A, Plass M, Holzinger K, Crişan GC, Pintea C-M, Palade V (2016) Towards interactive Machine Learning (iML): Applying Ant Colony Algorithms to Solve the Traveling Salesman Problem with the Human-in-the-Loop Approach. In: International Conference on Availability, Reliability, and Security, Salzburg, Austria, pp 81–95

  19. James AP, Dasarathy BV (2014) Medical image fusion: a survey of the state of the art. Inf Fusion 19:4–19

    Article  Google Scholar 

  20. Jinno T, Okuda M Multiple exposure fusion for high dynamic range image acquisition. IEEE Trans Image Process 21(1):358–365

  21. Li HF, Chai Y, Yin HP, Liu GQ (2012) Multifocus image fusion and denoising scheme based on homogeneity similarity. Opt Commun 285(2):91–100

    Article  Google Scholar 

  22. Li S, Kang X (2012) Fast multi-exposure image fusion with median filter and recursive filter. IEEE Trans Consum Electron 58(2):626–632

    Article  Google Scholar 

  23. Li S, Kang X, Hu J (2013) Image fusion with guided filtering. IEEE Trans Image Process 22(7):2864–2875

    Article  Google Scholar 

  24. Li H, Manjunath B, Mitra SK (1995) Multisensor image fusion using the wavelet transform. Graph Models Image Process 57(3):235–245

    Article  Google Scholar 

  25. Li ST, Yin HT, Fang LY (2012) Group-sparse representation with dictionary learning for medical image Denoising and fusion. IEEE Trans Biomed Eng 59(12):3450–3459

    Article  Google Scholar 

  26. Lischinski D, Farbman Z, Uyttendaele M, Szeliski R (2006) Interactive local adjustment of tonal values. ACM Trans Graph (TOG) 25(3):646–653

    Article  Google Scholar 

  27. Liu Y, Liu S, Wang Z (2015) A general framework for image fusion based on multi-scale transform and sparse representation. Inf Fusion 24(0):147–164

    Article  Google Scholar 

  28. Liu Y, Liu SP, Wang ZF (2015) A general framework for image fusion based on multi-scale transform and sparse representation. Inf Fusion 24:147–164

    Article  Google Scholar 

  29. Liu Z, Yin H, Chai Y, Yang SX (2014) A novel approach for multimodal medical image fusion. Expert Syst Appl 41(16):7425–7435

    Article  Google Scholar 

  30. Liu Z, Yin H, Fang B, Chai Y (2013) A novel fusion scheme for visible and infrared images based on compressive sensing. Opt Commun 335(0):168–177

    Google Scholar 

  31. Maintz JA, Viergever MA (1998) A survey of medical image registration. Med Image Anal 2(1):1–36

    Article  Google Scholar 

  32. Otsu N (1975) A threshold selection method from gray-level histograms. Automatica 11(285-296):23–27

    Google Scholar 

  33. Pajares G, De La Cruz JM (2004) A wavelet-based image fusion tutorial. Pattern Recogn 37(9):1855–1872

    Article  Google Scholar 

  34. Petrovic V (2007) Subjective tests for image fusion evaluation and objective metric validation. Inf Fusion 8(2):208–216

    Article  Google Scholar 

  35. Petrovic V, Xydeas C (2005) Objective evaluation of signal-level image fusion performance. Opt Eng 44(8):1–8

    Google Scholar 

  36. Piella G, Heijmans H (2003) A new quality metric for image fusion. In: International Conference on Image Processing, Barcelona, Spain, pp 173–176

  37. Qu GH, Zhang DL, Yan PF (2001) Medical image fusion by wavelet transform modulus maxima. Opt Express 9(4):184–190

    Article  Google Scholar 

  38. Qu GH, Zhang DL, Yan PF (2002) Information measure for performance of image fusion. Electron Lett 38(7):313–315

    Article  Google Scholar 

  39. Shutao L, Haitao Y, Leyuan F (2013) Remote sensing image fusion via sparse representations over learned dictionaries. IEEE Trans Geosci Remote Sens 51(9):4779–4789

    Article  Google Scholar 

  40. Singh S, Gupta D, Anand RS, Kumar V (2015) Nonsubsampled shearlet based CT and MR medical image fusion using biologically inspired spiking neural network. Biomed Signal Process Control 18:91–101

    Article  Google Scholar 

  41. Suh JW, Kwon OK, Scheinost D, Sinusas AJ, Cline GW, Papademetris X (2012) CT-PET weighted image fusion for separately scanned whole body rat. Med Phys 39(1):533–542

    Article  Google Scholar 

  42. Tian J, Chen L (2012) Adaptive multi-focus image fusion using a wavelet-based statistical sharpness measure. Signal Process 92(9):2137–2146

    Article  Google Scholar 

  43. Tu T-M, Cheng W-C, Chang C-P, Huang PS, Chang J-C (2007) Best tradeoff for high-resolution image fusion to preserve spatial details and minimize color distortion. Geosci Remote Sens Lett IEEE 4(2):302–306

    Article  Google Scholar 

  44. Wan T, Canagarajah N, Achim A (2008) Compressive image fusion, in International Conference on Image Processing, San Diego, CA, USA, pp 1308–1311

  45. Wang Z, Bovik AC (2002) A universal image quality index. Signal Process Lett 9(3):81–84

    Article  Google Scholar 

  46. Wang Z, Bovik AC (2006) Modern image quality assessment. Synth Lectures ImageVideo Multimed Process 2(1):1–156

    Article  Google Scholar 

  47. Wang QZ, Li S, Qin H, Hao AM (2015) Robust multi-modal medical image fusion via anisotropic heat diffusion guided low-rank structural analysis. Inf Fusion 26:103–121

    Article  Google Scholar 

  48. Wang L, Li B, Tian LF (2014) EGGDD: an explicit dependency model for multi-modal medical image fusion in shift-invariant shearlet transform domain. Inf Fusion 19:29–37

    Article  Google Scholar 

  49. Wang L, Li B, Tian LF (2014) Multi-modal medical image fusion using the inter-scale and intra-scale dependencies between image shift-invariant shearlet coefficients. Inf Fusion 19:20–28

    Article  Google Scholar 

  50. Wang ZB, Ma Y (2008) Medical image fusion using m-PCNN. Inf Fusion 9(2):176–185

    Article  Google Scholar 

  51. Wong A, Bishop W (2008) Efficient least squares fusion of MRI and CT images using a phase congruency model. Pattern Recogn Lett 29(3):173–180

    Article  Google Scholar 

  52. Xu ZP (2014) Medical image fusion using multi-level local Extrema. Inf Fusion 19:38–48

    Article  Google Scholar 

  53. Yang L, Guo BL, Ni W (2008) Multimodality medical image fusion based on multiscale geometric analysis of contourlet transform. Neurocomputing 72(1-3):203–211

    Article  Google Scholar 

  54. Zhang X, Li X, Liu Z, Feng Y (2014) Multi-focus image fusion using image-partition-based focus detection. Signal Process 102(0):64–76

    Article  Google Scholar 

Download references

Acknowledgements

The work was supported by National Natural Science Foundation of China (Grant No. 61672259, 61602203, 61876070, 61801190), Outstanding Young Talent Foundation of Jilin Province (Grant No. 20180520029JH) and China Postdoctoral Science Foundation (Grant No. 2017 M611323). We would also like to thank http://www.med.harvard.edu/aanlib/home.html for providing us the source medical images.

Author information

Authors and Affiliations

  1. Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, Changchun, 130012, China

    Jun Qin, Xuanjing Shen, Haipeng Chen, Yingda Lv & Xiaoli Zhang

  2. College of Computer Science and Technology, Jilin University, Changchun, 130012, China

    Jun Qin, Xuanjing Shen, Haipeng Chen, Yingda Lv & Xiaoli Zhang

Authors
  1. Jun Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xuanjing Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Haipeng Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yingda Lv
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaoli Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Haipeng Chen.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qin, J., Shen, X., Chen, H. et al. A fusion algorithm for medical structural and functional images based on adaptive image decomposition. Multimed Tools Appl 78, 32605–32629 (2019). https://doi.org/10.1007/s11042-019-07968-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-07968-5

Keywords

Search

Navigation