Skip to main content
SpringerLink
Log in

A novel variational optimization model for medical CT and MR image fusion

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

In medical imaging processing, image fusion is the process of combining complementary information from different or multi-modality images to obtain a high-quality and informative fused image in order to improve clinical diagnostic accuracy. In this paper, we propose a novel variational fusion model based on contrast and gradient features, the weight images and the fused images are constrained by the total variation regularization. The salient contrast features and clear soft tissue structure information of source CT and MR images can be preserved in the fused images. The variational problem is solved by a fast split optimization algorithm. In the numerical experiments, the proposed method is compared with seven state-of-the-art methods, and the comparison metrics MI, \(Q_W\) and \(Q^{G}\) are calculated for assessment. The proposed method shows a comprehensive advantage in preserving the contrast features as well as texture structure information, not only in visual effects but also in objective assessments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. James, A.P., Dasarathy, B.V.: Medical image fusion: a survey of the state of the art. Inf. Fusion 19, 4–19 (2014)

    Article  Google Scholar 

  2. Du, J., Li, W., Lu, K., Xiao, B.: An overview of multi-modal medical image fusion. Neurocomputing 215, 3–20 (2016)

    Article  Google Scholar 

  3. Li, S., Yang, B., Hu, J.: Performance comparison of different multi-resolution transforms for image fusion. Inf. Fusion 12(2), 74–84 (2011)

    Article  Google Scholar 

  4. Liu, Y., Liu, S., Wang, Z.: A general framework for image fusion based on multi-scale transform and sparse representation. Inf. Fusion 24, 147–164 (2015)

    Article  Google Scholar 

  5. Burt, P., Adelson, E.: The laplacian pyramid as a compact image code. IEEE Trans. Commun. 31(4), 532–540 (1983)

    Article  Google Scholar 

  6. Toet, A.: Multiscale contrast enhancement with applications to image fusion. Opt. Eng. 31(5), 1026–1032 (1992)

    Article  Google Scholar 

  7. Pajares, G., De La Cruz, J.M.: A wavelet-based image fusion tutorial. Pattern Recogn. 37(9), 1855–1872 (2004)

    Article  Google Scholar 

  8. Lewis, J.J., O¡Callaghan, R.J., Nikolov, S.G., Bull, D.R., Canagarajah, N.: Pixel-and region-based image fusion with complex wavelets. Inf. Fusion 8(2), 119–130 (2007)

    Article  Google Scholar 

  9. Na, Y., Zhao, L., Yang, Y., Ren, M.: Guided filter-based images fusion algorithm for ct and mri medical images. IET Image Proc. 12(1), 138–148 (2017)

    Article  Google Scholar 

  10. Nencini, F., Garzelli, A., Baronti, S., Alparone, L.: Remote sensing image fusion using the curvelet transform. Inf. Fusion 8(2), 143–156 (2007)

    Article  Google Scholar 

  11. Bhatnagar, G., Wu, Q.J., Liu, Z.: A new contrast based multimodal medical image fusion framework. Neurocomputing 157, 143–152 (2015)

    Article  Google Scholar 

  12. Kong, W.: Technique for gray-scale visual light and infrared image fusion based on non-subsampled shearlet transform. Infrared Phys. Technol. 63, 110–118 (2014)

    Article  Google Scholar 

  13. Liu, X., Mei, W., Du, H.: Multi-modality medical image fusion based on image decomposition framework and nonsubsampled shearlet transform. Biomed. Signal Process. Control 40, 343–350 (2018)

    Article  Google Scholar 

  14. Yin, M., Liu, X., Liu, Y., Chen, X.: Medical image fusion with parameter-adaptive pulse coupled neural network in nonsubsampled shearlet transform domain. IEEE Trans. Instrum. Meas. 68(1), 49–64 (2019)

    Article  Google Scholar 

  15. Liu, Y., Chen, X., Wang, Z., Wang, Z.J., Ward, R.K., Wang, X.: Deep learning for pixel-level image fusion: Recent advances and future prospects. Inf. Fusion 42, 158–173 (2018)

    Article  Google Scholar 

  16. Liu, Y., Chen, X., Peng, H., Wang, Z.: Multi-focus image fusion with a deep convolutional neural network. Inf. Fusion 36, 191–207 (2017)

    Article  Google Scholar 

  17. Liu, Y., Chen, X., Cheng, J., Peng, H.: A medical image fusion method based on convolutional neural networks. In: 2017 20th International Conference on Information Fusion (Fusion), pp. 1–7 (2017)

  18. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D: Nonlinear Phenom. 60(1), 259–268 (1992)

  19. Wang, W.W., Shui, P.L., Feng, X.C.: Variational models for fusion and denoising of multifocus images. IEEE Signal Process. Lett. 15, 65–68 (2008)

    Article  Google Scholar 

  20. Yuan, J., Miles, B., Garvin, G., Tai, X.C., Fenster, A.: Efficient convex optimization approaches to variational image fusion. Numer. Math. Theory Methods Appl. 7(2), 234–250 (2014)

    Article  MATH  Google Scholar 

  21. Li, F., Zeng, T.: Variational image fusion with first and second-order gradient information. J. Comput. Math. 34, 200–222 (2016)

    Article  MATH  Google Scholar 

  22. Piella, G.: Image fusion for enhanced visualization: a variational approach. Int. J. Comput. Vision 83(1), 1–11 (2009)

    Article  MATH  Google Scholar 

  23. Wang, Q., Yang, X.: An efficient fusion algorithm combining feature extraction and variational optimization for CT and MR images. J. Appl. Clin. Med. Phys. 21(6), 139–150 (2020)

    Article  Google Scholar 

  24. Ma, J., Chen, C., Li, C., Huang, J.: Infrared and visible image fusion via gradient transfer and total variation minimization. Inf. Fusion 31, 100–109 (2016)

    Article  Google Scholar 

  25. Liu, X., Mei, W., Du, H.: Structure tensor and nonsubsampled shearlet transform based algorithm for CT and MRI image fusion. Neurocomputing 235, 131–139 (2017)

    Article  Google Scholar 

  26. Bredies, K., Kunisch, K., Pock, T.: Total generalized variation. SIAM J. Imag. Sci. 3, 492–526 (2010)

    Article  MATH  Google Scholar 

  27. Pock, T., Zebedin, L., Bischof, H.: TGV-fusion. In: Rainbow of Computer Science, pp. 245–258. Springer (2011)

  28. Wang, Q., Yang, X.: Variational image fusion approach based on TGV and local information. IET Comput. Vision 12(4), 535–541 (2018)

    Article  Google Scholar 

  29. Cai, J.F., Osher, S., Shen, Z.: Split bregman methods and frame based image restoration. Multiscale Model. Simul. 8(2), 337–369 (2009)

    Article  MATH  Google Scholar 

  30. Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vision 40(1), 120–145 (2011)

    Article  MATH  Google Scholar 

  31. Johnson, K.A., Becker, J.A.: Whole brain website of the harvard medical school. http://www.med.harvard.edu/aanlib/. Accessed 13 April (2018)

  32. Hossny, M., Nahavandi, S., Creighton, D.: Comments on’Information measure for performance of image fusion’. Electron. Lett. 44(18), 1066–1067 (2008)

    Article  Google Scholar 

  33. Yang, C., Zhang, J.Q., Wang, X.R., Liu, X.: A novel similarity based quality metric for image fusion. Inf. Fusion 9(2), 156–160 (2008)

    Article  Google Scholar 

  34. Xydeas, C., Petrovic, V.: Objective image fusion performance measure. Electron. Lett. 36(4), 308–309 (2000)

    Article  Google Scholar 

  35. Wang, Z., Bovik, A.C.: A universal image quality index. IEEE Signal Process. Lett. 9(3), 81–84 (2002)

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by National Natural Science Foundation of China ( No.11971229).

Author information

Authors and Affiliations

  1. Artificial Intelligence Research Institute, China University of Mining and Technology, Xuzhou, 221116, China

    Qinxia Wang & Mingcheng Zuo

  2. School of Science, Nanjing University of Science and Technology, Nanjing, 210094, China

    Qinxia Wang

Authors
  1. Qinxia Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mingcheng Zuo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qinxia Wang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Q., Zuo, M. A novel variational optimization model for medical CT and MR image fusion. SIViP 17, 183–190 (2023). https://doi.org/10.1007/s11760-022-02220-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-022-02220-4

Keywords

Search

Navigation