Skip to main content
SpringerLink
Log in

Medical image fusion by adaptive Gaussian PCNN and improved Roberts operator

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Medical image fusion analyzes multiple images obtained by the same/different medical modalities and constructs a robust image that is more useful for physicians by merging the complementary details contained in these images. Recently, pulse coupled neural network (PCNN) models constructed efficient image fusion algorithms, but at the expense of many parameters. Here, a novel adaptive Gaussian PCNN (AGPCNN) model is proposed that constitutes few parameters, adopts adaptive linking strength, and employs a Gaussian filter to effectively combine the surrounding neurons. In this paper, a new medical image fusion algorithm is introduced in the non-subsampled Shearlet transform domain that applies the novel AGPCNN to combine the high-pass sub-bands, whereas a new improved Roberts operator-based mechanism is incorporated to merge the low-pass sub-bands. The power of the proposed method is demonstrated using the experimental results of seven latest methods with twelve objective metrics on ten diverse medical image pairs that include the image pairs of an AIDS dementia complex patient.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Availability of data and materials

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

References

  1. Azam, M.A., Khan, K.B., Salahuddin, S., Rehman, E., Khan, S.A., Khan, M.A., Kadry, S., Gandomi, A.H.: A review on multimodal medical image fusion: compendious analysis of medical modalities, multimodal databases, fusion techniques and quality metrics. Comput. Biol. Med. 144, 105253 (2022)

    Article  Google Scholar 

  2. Jiao, D., Li, W., Ke, L., Xiao, B.: An overview of multi-modal medical image fusion. Neurocomputing 215, 3–20 (2016)

    Article  Google Scholar 

  3. Dinh, P.-H.: Multi-modal medical image fusion based on equilibrium optimizer algorithm and local energy functions. Appl. Intell. 51(11), 8416–8431 (2021)

    Article  Google Scholar 

  4. Dinh, P.-H.: Medical image fusion based on enhanced three-layer image decomposition and chameleon swarm algorithm. Biomed. Signal Process. Control 84, 104740 (2023)

    Article  Google Scholar 

  5. Liu, Yu., Chen, X., Liu, A., Ward, R.K., Wang, Z.J.: Recent advances in sparse representation based medical image fusion. IEEE Instrum. Meas. Magn. 24(2), 45–53 (2021)

    Article  Google Scholar 

  6. Dinh, P.-H.: Combining spectral total variation with dynamic threshold neural P systems for medical image fusion. Biomed. Signal Process. Control 80, 104343 (2023)

    Article  Google Scholar 

  7. Panigrahy, C., Seal, A., Mahato, N.K.: Mri and spect image fusion using a weighted parameter adaptive dual channel pcnn. IEEE Signal Process. Lett. 27, 690–694 (2020)

    Article  Google Scholar 

  8. Li, Q., Wang, W., Chen, G., Zhao, D.: Medical image fusion using segment graph filter and sparse representation. Comput. Biol. Med. 131, 104239 (2021)

    Article  Google Scholar 

  9. Xia, K., Yin, H., Wang, J.: A novel improved deep convolutional neural network model for medical image fusion. Clust. Comput. 22(1), 1515–1527 (2019)

    Article  Google Scholar 

  10. Dinh, P.-H.: An improved medical image synthesis approach based on marine predators algorithm and maximum gabor energy. Neural Comput. Appl. 34(6), 4367–4385 (2022)

    Article  Google Scholar 

  11. Dinh, P.-H.: Combining gabor energy with equilibrium optimizer algorithm for multi-modality medical image fusion. Biomed. Signal Process. Control 68, 102696 (2021)

    Article  Google Scholar 

  12. Dinh, P.-H.: A novel approach based on grasshopper optimization algorithm for medical image fusion. Expert Syst. Appl. 171, 114576 (2021)

    Article  Google Scholar 

  13. Hermessi, H., Mourali, O., Zagrouba, E.: Multimodal medical image fusion review: theoretical background and recent advances. Signal Process. 183, 108036 (2021)

    Article  Google Scholar 

  14. Panigrahy, C., Seal, A., Gonzalo-Martín, C., Pathak, P., Jalal, A.S.: Parameter adaptive unit-linking pulse coupled neural network based MRI-PET/SPECT image fusion. Biomed. Signal Process. Control 83, 104659 (2023)

    Article  Google Scholar 

  15. Jiang, L., Zhang, D., Che, L.: Texture analysis-based multi-focus image fusion using a modified pulse-coupled neural network (pcnn). Signal Process. Image Commun. 91, 116068 (2021)

    Article  Google Scholar 

  16. Paramanandham, N., Rajendiran, K.: Multi sensor image fusion for surveillance applications using hybrid image fusion algorithm. Multimedia Tools Appl. 77(10), 12405–12436 (2018)

    Article  Google Scholar 

  17. Xie, X., Ge, S., Xie, M., Fengping, H., Jiang, N.: An improved industrial sub-pixel edge detection algorithm based on coarse and precise location. J. Ambient. Intell. Humaniz. Comput. 11(5), 2061–2070 (2020)

    Article  Google Scholar 

  18. Lindblad, T., Kinser, J.M., Taylor, J.G.: Image Processing Using Pulse-Coupled Neural Networks. Springer (2005)

  19. The whole brain atlas. http://www.med.harvard.edu/AANLIB/home.html. Accessed 31 Aug 2022

  20. Xiao-Bo, Q., Jing-Wen, Y., Hong-Zhi, X., Zi-Qian, Z.: Image fusion algorithm based on spatial frequency-motivated pulse coupled neural networks in nonsubsampled contourlet transform domain. Acta Automatica Sinica 34(12), 1508–1514 (2008)

    Article  Google Scholar 

  21. Zhu, Z., Zheng, M., Qi, G., Wang, D., Xiang, Y.: A phase congruency and local laplacian energy based multi-modality medical image fusion method in nsct domain. IEEE Access 7, 20811–20824 (2019)

    Article  Google Scholar 

  22. Li, X., Guo, X., Han, P., Wang, X., Li, H., Luo, T.: Laplacian redecomposition for multimodal medical image fusion. IEEE Trans. Instrum. Meas. 69(9), 6880–6890 (2020)

    Article  Google Scholar 

  23. Tan, W., Tiwari, P., Pandey, H.M., Moreira, C., Jaiswal, A.K.: Multimodal medical image fusion algorithm in the era of big data. Neural Comput. Appl. 1–21 (2020)

  24. Tan, W., Thitøn, W., Xiang, P., Zhou, H.: Multi-modal brain image fusion based on multi-level edge-preserving filtering. Biomed. Signal Process. Control 64, 102280 (2021)

    Article  Google Scholar 

  25. Li, X., Zhou, F., Tan, H., Zhang, W., Zhao, C.: Multimodal medical image fusion based on joint bilateral filter and local gradient energy. Inf. Sci. 569, 302–325 (2021)

    Article  MathSciNet  Google Scholar 

  26. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  27. Piella, G., Heijmans, H.: A new quality metric for image fusion. In: Proceedings 2003 International Conference on Image Processing (Cat. No. 03CH37429), vol. 3, pp. III–173. IEEE (2003)

  28. Wang, Z., Bovik, A.C.: A universal image quality index. IEEE Signal Process. Lett. 9(3), 81–84 (2002)

    Article  Google Scholar 

  29. Haghighat, M., Razian, M.A.: Fast-FMI: Non-reference image fusion metric. In: 2014 IEEE 8th International Conference on Application of Information and Communication Technologies (AICT), pp. 1–3. IEEE (2014)

  30. Guihong, Q., Zhang, D., Yan, P.: Information measure for performance of image fusion. Electron. Lett. 38(7), 1 (2002)

    Google Scholar 

  31. Petrovic, V., Xydeas, C.S.: Objective image fusion performance characterisation. In: Tenth IEEE International Conference on Computer Vision (ICCV’05) Volume 1, vol. 2, pp. 1866–1871. IEEE (2005)

  32. Xydeas, C.S., Petrovic, V.: Objective image fusion performance measure. Electron. Lett. 36(4), 308–309 (2000)

    Article  Google Scholar 

  33. Panigrahy, C., Seal, A., Mahato, N.K., Krejcar, O., Herrera-Viedma, E.: Multi-focus image fusion using fractal dimension. Appl. Opt. 59(19), 5642–5655 (2020)

    Article  Google Scholar 

  34. Dinh, P.-H., Giang, N.L.: A new medical image enhancement algorithm using adaptive parameters. Int. J. Imaging Syst. Technol. 32(6), 2198–2218 (2022)

    Article  Google Scholar 

  35. Agrawal, C., Yadav, S.K., Singh, S.P., Panigrahy, C.: A simplified parameter adaptive DCPCNN based medical image fusion. In: Proceedings of International Conference on Communication and Artificial Intelligence, pp. 489–501. Springer (2022)

  36. Dinh, P.-H.: A novel approach based on three-scale image decomposition and marine predators algorithm for multi-modal medical image fusion. Biomed. Signal Process. Control 67, 102536 (2021)

    Article  Google Scholar 

  37. Dinh, P-H.: A novel approach using structure tensor for medical image fusion. Multidimens. Syst. Signal Process. 1–21 (2022)

  38. Dinh, P.-H.: A novel approach based on marine predators algorithm for medical image enhancement. Sens. Imaging 24(1), 1–23 (2023)

    Article  MathSciNet  Google Scholar 

  39. Goyal, S., Singh, V., Rani, A., Yadav, N.: FPRSGF denoised non-subsampled shearlet transform-based image fusion using sparse representation. SIViP 14(4), 719–726 (2020)

    Article  Google Scholar 

  40. Li, X., Zhang, X., Ding, M.: A sum-modified-laplacian and sparse representation based multimodal medical image fusion in Laplacian pyramid domain. Med. Biol. Eng. Comput. 57(10), 2265–2275 (2019)

    Article  Google Scholar 

  41. Liu, X., Mei, W., Huiqian, D.: Multi-modality medical image fusion based on image decomposition framework and nonsubsampled shearlet transform. Biomed. Signal Process. Control 40, 343–350 (2018)

    Article  Google Scholar 

  42. Liu, X., Mei, W., Huiqian, D., Bei, J.: A novel image fusion algorithm based on nonsubsampled shearlet transform and morphological component analysis. SIViP 10(5), 959–966 (2016)

    Article  Google Scholar 

  43. Polinati, S., Dhuli, R.: Multimodal medical image fusion using empirical wavelet decomposition and local energy maxima. Optik 205, 163947 (2020)

    Article  Google Scholar 

  44. Dinh, P-H.: A novel approach using the local energy function and its variations for medical image fusion. Imaging Sci. J. 1–17 (2023)

Download references

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Author information

Author notes

  1. Pravesh Vajpayee, Chinmaya Panigrahy and Anil Kumar have contributed equally to this work.

Authors and Affiliations

  1. Electronics and Communication Engineering, PDPM Indian Institute of Information Technology, Design and Manufacturing, Jabalpur, Madhya Pradesh, 482005, India

    Pravesh Vajpayee & Anil Kumar

  2. Computer Science and Engineering, Thapar Institute of Engineering and Technology, Patiala, Punjab, 147004, India

    Chinmaya Panigrahy

Authors
  1. Pravesh Vajpayee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chinmaya Panigrahy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anil Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors made substantial contributions to the concept, design, and revision of the paper. Methodology was done by PV, CP, and AK, software development was done by PV, and project administration/supervision was done by CP and AK. All the authors read and approved the manuscript.

Corresponding author

Correspondence to Chinmaya Panigrahy.

Ethics declarations

Conflict of interest

All authors have no conflicts of interest relevant to this study to disclose.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1 (pdf 1221 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vajpayee, P., Panigrahy, C. & Kumar, A. Medical image fusion by adaptive Gaussian PCNN and improved Roberts operator. SIViP 17, 3565–3573 (2023). https://doi.org/10.1007/s11760-023-02581-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-023-02581-4

Keywords

Search

Navigation