Skip to main content
SpringerLink
Log in

Fast local Laplacian filtering based enhanced medical image fusion using parameter-adaptive PCNN and local features-based fuzzy weighted matrices

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Generally, the anatomical CT/MRI modalities exhibit the brain tissue anatomy with a high spatial resolution, where PET/SPECT modalities show the metabolic features with low spatial resolution. Therefore, the integration of these two classes significantly improves several clinical applications and computer-aided diagnoses. In the proposed scheme, a fast local Laplacian filter (FLLF) is first applied to the source images to enhance the edge information and suppress the noise artifacts. Second, the RGB images are converted to YUV color space to separate the Y-component. Then to capture the spatial and spectral features of the source images, the NSST is applied to decompose the input (grayscale and/or Y-component) images into one low (LFS) and several high-frequency subbands (HFS). Third, an improved salience measure and matching factor (SMF) method by the local features-based fuzzy-weighted matrix (FW-SMF) is introduced to fuse the LFS coefficient. Due to the fast convergence with fewer iterations and robust pixels selection procedure, the PA-PCNN model is adopted to fuse the HFS coefficients. Fourth, the final fused image is obtained by applying inverse NSST and YUV format. Visual and statistical analysis performed on various experiments prove that the proposed scheme not only integrates the spatial and texture features details of the source images but also enhances the visual quality and contrast of the fused image compared to the existing state-of-arts.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Liu Y, Chen X, Wang Z, Wang Z, Ward RK, Wang X (2018) Deep learning for pixel-level image fusion: Recent advances and future prospects. Inf Fusion 42:158–173

    Article  Google Scholar 

  2. Meher B, Agrawal S, Panda R, Abraham A (2019) A survey on region based image fusion methods. Inf Fusion 48:119–132

    Article  Google Scholar 

  3. Li S, Kang X, Fang L, Hu J, Yin H (2017) Pixel-level image fusion: A survey of the state of the art. Inf Fusion 33:100–112

    Article  Google Scholar 

  4. Shahdoosti HR, Mehrabi A (2018) Mri and pet image fusion using structure tensor and dual ripplet-ii transform. Multimed Tools Appl 77(17):22 649–22 670

    Article  Google Scholar 

  5. Haddadpour M, Daneshvar S, Seyedarabi H (2017) Pet and mri image fusion based on combination of 2-d hilbert transform and ihs method. Biomed J 40(4):219–225

    Article  Google Scholar 

  6. Daneshvar S, Ghassemian H (2010) Mri and pet image fusion by combining ihs and retina-inspired models. Inf Fusion 11(2):114–123

    Article  Google Scholar 

  7. Singh S, Anand R (2019) Multimodal medical image fusion using hybrid layer decomposition with cnn-based feature mapping and structural clustering. IEEE Trans Instrum Meas 69(6):3855– 3865

    Article  Google Scholar 

  8. Benjamin JR, Jayasree T (2018) Improved medical image fusion based on cascaded pca and shift invariant wavelet transforms. Int J Comput Assist Radiol Surg 13(2):229–240

    Article  Google Scholar 

  9. Cvejic N, Bull D, Canagarajah N (2007) Region-based multimodal image fusion using ica bases. IEEE Sens J 7(5):743–751

    Article  Google Scholar 

  10. Meng L, Guo X, Li H (2019) Mri/ct fusion based on latent low rank representation and gradient transfer. Biomedical Signal Process Control 53:101536

  11. Zhao W, Xu Z, Zhao J (2016) Gradient entropy metric and p-laplace diffusion constraint-based algorithm for noisy multispectral image fusion. Inf Fusion 27:138–149

    Article  Google Scholar 

  12. Burt PJ, Adelson EH (1987) The laplacian pyramid as a compact image code. In: Readings in computer vision. Elsevier, pp 671–679

  13. Toet A (1989) A morphological pyramidal image decomposition. Pattern Recogn Lett 9(4):255–261

    Article  MATH  Google Scholar 

  14. Shensa MJ et al (1992) The discrete wavelet transform: wedding the a trous and mallat algorithms. IEEE Trans Signal Process 40(10):2464–2482

    Article  MATH  Google Scholar 

  15. Nason GP, Silverman BW (1995) The stationary wavelet transform and some statistical applications. In: Wavelets and statistics. Springer, pp 281–299

  16. Selesnick IW, Baraniuk RG, Kingsbury NC (2005) The dual-tree complex wavelet transform. IEEE Signal Process Mag 22(6):123–151

    Article  Google Scholar 

  17. Do MN, Vetterli M (2005) The contourlet transform: an efficient directional multiresolution image representation. IEEE Trans Image Process 14(12):2091–2106

    Article  Google Scholar 

  18. Da Cunha AL, Zhou J, Do MN (2006) The nonsubsampled contourlet transform: theory, design, and applications. IEEE Trans Image Process 15(10):3089–3101

    Article  Google Scholar 

  19. Easley G, Labate D, Lim W-Q (2008) Sparse directional image representations using the discrete shearlet transform. Appl Comput Harmon Anal 25(1):25–46

    Article  MathSciNet  MATH  Google Scholar 

  20. Shahdoosti HR, Mehrabi A (2018) Multimodal image fusion using sparse representation classification in tetrolet domain. Digital Signal Process 79:9–22

    Article  MathSciNet  Google Scholar 

  21. Zhu Z, Yin H, Chai Y, Li Y, Qi G (2018) A novel multi-modality image fusion method based on image decomposition and sparse representation. Inf Sci 432:516–529

    Article  MathSciNet  Google Scholar 

  22. Du J, Li W, Xiao B (2017) Anatomical-functional image fusion by information of interest in local laplacian filtering domain. IEEE Trans Image Process 26(12):5855–5866

    Article  MathSciNet  Google Scholar 

  23. Yin M, Liu X, Liu Y, Chen X (2018) Medical image fusion with parameter-adaptive pulse coupled neural network in nonsubsampled shearlet transform domain. IEEE Trans Instrum Meas 68(1):49–64

    Article  Google Scholar 

  24. Ullah H, Ullah B, Wu L, Abdalla FY, Ren G, Zhao Y (2020) Multi-modality medical images fusion based on local-features fuzzy sets and novel sum-modified-laplacian in non-subsampled shearlet transform domain. Biomed Signal Process Control 57:101724

  25. Balasubramaniam P, Ananthi V (2014) Image fusion using intuitionistic fuzzy sets. Inf Fusion 20:21–30

    Article  Google Scholar 

  26. Darwish SM (2013) Multi-level fuzzy contourlet-based image fusion for medical applications. IET Image Process 7(7):694–700

    Article  Google Scholar 

  27. Nirmala DE, Vignesh R, Vaidehi V (2013) Fusion of multisensor images using nonsubsampled contourlet transform and fuzzy logic. In: 2013 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE). IEEE, pp 1–8

  28. Yang Y, Que Y, Huang S, Lin P (2016) Multimodal sensor medical image fusion based on type-2 fuzzy logic in nsct domain. IEEE Sens J 16(10):3735–3745

    Article  Google Scholar 

  29. Manchanda M, Sharma R (2018) An improved multimodal medical image fusion algorithm based on fuzzy transform. J Vis Commun Image Represent 51:76–94

    Article  Google Scholar 

  30. Xiao-Bo Q, Jing-Wen Y, Hong-Zhi X, Zi-Qian Z (2008) Image fusion algorithm based on spatial frequency-motivated pulse coupled neural networks in nonsubsampled contourlet transform domain. Acta Autom Sin 34(12):1508–1514

    Google Scholar 

  31. Wang Z, Wang S, Zhu Y, Ma Y (2016) Review of image fusion based on pulse-coupled neural network. Arch Comput Methods Eng 23(4):659–671

    Article  MathSciNet  MATH  Google Scholar 

  32. Ouerghi H, Mourali O, Zagrouba E (2018) Non-subsampled shearlet transform based mri and pet brain image fusion using simplified pulse coupled neural network and weight local features in yiq colour space. IET Image Process 12(10):1873–1880

    Article  Google Scholar 

  33. El-Hoseny HM, Abd El-Rahman W, El-Rabaie E-SM, Abd El-Samie FE, Faragallah OS (2018) An efficient dt-cwt medical image fusion system based on modified central force optimization and histogram matching. Infrared Phys Technol 94:223–231

    Article  Google Scholar 

  34. Xu L, Si Y, Jiang S, Sun Y, Ebrahimian H (2020) Medical image fusion using a modified shark smell optimization algorithm and hybrid wavelet-homomorphic filter. Biomed Signal Process Control 59:101885

  35. Dinh P-H (2021) Multi-modal medical image fusion based on equilibrium optimizer algorithm and local energy functions. Appl Intell:1–16

  36. Padmavathi K, Asha C, Maya VK (2020) A novel medical image fusion by combining tv-l1 decomposed textures based on adaptive weighting scheme. Eng Sci Technol Int J 23(1):225–239

    Google Scholar 

  37. Dinh P-H (2021) A novel approach based on grasshopper optimization algorithm for medical image fusion. Expert Syst Appl 171:114576

  38. Dinh P-H (2021) Combining gabor energy with equilibrium optimizer algorithm for multi-modality medical image fusion. Biomed Signal Process Control 68:102696

  39. Dinh P-H (2021) A novel approach based on three-scale image decomposition and marine predators algorithm for multi-modal medical image fusion. Biomed Signal Process Control 67:102536

  40. Hou R, Zhou D, Nie R, Liu D, Ruan X (2019) Brain ct and mri medical image fusion using convolutional neural networks and a dual-channel spiking cortical model. Med Biol Eng Comput 57 (4):887–900

    Article  Google Scholar 

  41. Hermessi H, Mourali O, Zagrouba E (2018) Convolutional neural network-based multimodal image fusion via similarity learning in the shearlet domain. Neural Comput Appl 30(7):2029–2045

    Article  Google Scholar 

  42. Liu Y, Chen X, Peng H, Wang Z (2017) Multi-focus image fusion with a deep convolutional neural network. Inf Fusion 36:191–207

    Article  Google Scholar 

  43. Liu Y, Chen X, Cheng J, Peng H (2017) A medical image fusion method based on convolutional neural networks. In: 2017 20th International Conference on Information Fusion (Fusion). IEEE, pp 1–7

  44. Burt PJ, Kolczynski RJ (1993) Enhanced image capture through fusion. In: 1993 (4th) International Conference on Computer Vision. IEEE, pp 173–182

  45. Aubry M, Paris S, Hasinoff SW, Kautz J, Durand F (2014) Fast local laplacian filters: Theory and applications. ACM Trans Graph (TOG) 33(5):1–14

    Article  Google Scholar 

  46. Paris S, Hasinoff SW, Kautz J (2011) Local laplacian filters: Edge-aware image processing with a laplacian pyramid. ACM Trans Graph 30(4):68

    Article  Google Scholar 

  47. Soltanian-Zadeh H, Windham JP, Yagle AE (1995) A multidimensional nonlinear edge-preserving filter for magnetic resonance image restoration. IEEE Trans Image Process 4(2):147–161

    Article  Google Scholar 

  48. Langner J, Hofheinz F, Beuthien-Baumann B, van den Hoff J (2011) Locally adaptive filtering for edge preserving noise reduction on images with low snr in pet. In: 2011 IEEE Nuclear Science Symposium Conference Record. IEEE, pp 3102–3104

  49. Li W, Zhao Z, Du J, Wang Y (2017) Edge-preserve filter image enhancement with application to medical image fusion. J Med Imaging Health Inf 7(1):16–24

    Article  Google Scholar 

  50. Qiang Z, He L, Chen Y, Chen X, Xu D (2019) Adaptive fast local laplacian filters and its edge-aware application. Multimed Tools Appl 78(1):619–639

    Article  Google Scholar 

  51. Johnson JL, Padgett ML (1999) Pcnn models and applications. IEEE Trans Neural Netw 10 (3):480–498

    Article  Google Scholar 

  52. Eckhorn R, Reitboeck HJ, Arndt M, Dicke P (1990) Feature linking via synchronization among distributed assemblies: Simulations of results from cat visual cortex. Neural Comput 2(3):293–307

    Article  Google Scholar 

  53. Nie R, He M, Cao J, Zhou D, Liang Z (2019) Pulse coupled neural network based mri image enhancement using classical visual receptive field for smarter mobile healthcare. J Ambient Intell Human Comput 10(10):4059–4070

    Article  Google Scholar 

  54. Xu X, Liang T, Wang G, Wang M, Wang X (2017) Self-adaptive pcnn based on the aco algorithm and its application on medical image segmentation. Intell Autom Soft Comput 23(2):303–310

    Article  Google Scholar 

  55. Chen Y, Park S-K, Ma Y, Ala R (2011) A new automatic parameter setting method of a simplified pcnn for image segmentation. IEEE Trans Neural Netw 22(6):880–892

    Article  Google Scholar 

  56. Johnson JL (1994) Pulse-coupled neural nets: translation, rotation, scale, distortion, and intensity signal invariance for images. Appl Opt 33(26):6239–6253

    Article  Google Scholar 

  57. Jin X, Chen G, Hou J, Jiang Q, Zhou D, Yao S (2018) Multimodal sensor medical image fusion based on nonsubsampled shearlet transform and s-pcnns in hsv space. Signal Process 153:379–395

    Article  Google Scholar 

  58. Ullah H, Zhao Y, Wu L, Abdalla FY, Mkindu H (2019) Nsst based mri-pet/spect color image fusion using local features fuzzy rules and nsml in yiq space. In: 2019 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT). IEEE, pp 1–6

  59. Li X, Zhao J (2020) A novel multi-modal medical image fusion algorithm. J Ambient Intell Human Comput:1–8

  60. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  61. Xydeas C, Petrovic V (2000) Objective image fusion performance measure. Electron Lett 36 (4):308–309

    Article  Google Scholar 

  62. Haghighat MBA, Aghagolzadeh A, Seyedarabi H (2011) A non-reference image fusion metric based on mutual information of image features. Comput Electr Eng 37(5):744–756

    Article  MATH  Google Scholar 

  63. Piella G, Heijmans H (2003) A new quality metric for image fusion. In: Proceedings 2003 international conference on image processing (Cat. No. 03CH37429), vol 3. IEEE, pp III–173

  64. Chen Y, Blum RS (2009) A new automated quality assessment algorithm for image fusion. Image Vis Comput 27(10):1421–1432

    Article  Google Scholar 

  65. Liu Y, Chen X, Ward RK, Wang Z (2019) Medical image fusion via convolutional sparsity based morphological component analysis. IEEE Signal Process Lett 26(3):485–489

    Article  Google Scholar 

  66. Tan W, Thitøn W, Xiang P, Zhou H (2021) Multi-modal brain image fusion based on multi-level edge-preserving filtering. Biomed Signal Process Control 64:102280

  67. Zhang Y, Liu Y, Sun P, Yan H, Zhao X, Zhang L (2020) Ifcnn: a general image fusion framework based on convolutional neural network. Inf Fusion 54:99–118

    Article  Google Scholar 

  68. Gao Y, Ma S, Liu J, Liu Y, Zhang X (2021) Fusion of medical images based on salient features extraction by pso optimized fuzzy logic in nsst domain. Biomed Signal Process Control 69:102852

  69. Maqsood S, Javed U (2020) Multi-modal medical image fusion based on two-scale image decomposition and sparse representation. Biomed Signal Process Control 57:101810

  70. Li B, Xie W (2015) Adaptive fractional differential approach and its application to medical image enhancement. Comput Electr Eng 45:324–335

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by the National Natural Science Foundation of China under Grant No. 61671185 and 62071153.

Author information

Authors and Affiliations

  1. School of Electronics and Information Engineering, Harbin Institute of Technology, Harbin, 150001, China

    Hikmat Ullah, Yaqin Zhao, Fakheraldin Y. O. Abdalla & Longwen Wu

Authors
  1. Hikmat Ullah
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yaqin Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fakheraldin Y. O. Abdalla
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Longwen Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Longwen Wu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ullah, H., Zhao, Y., Abdalla, F.Y.O. et al. Fast local Laplacian filtering based enhanced medical image fusion using parameter-adaptive PCNN and local features-based fuzzy weighted matrices. Appl Intell 52, 7965–7984 (2022). https://doi.org/10.1007/s10489-021-02834-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02834-0

Keywords

Search

Navigation