Skip to main content
SpringerLink
Log in

Improving performance of medical image fusion using histogram, dictionary learning and sparse representation

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Medical image fusion has attracted much attention in recent years, which aims to fuse different medical images into a more informative and clearer one. The fused image is able to help doctors to diagnose diseases rapidly and effectively. Among numerous fusion methods, sparse-representation-based image fusion is a new concept that has emerged over the past several years. However, the high-frequency components of low-resolution and the high-frequency components of source images are obtained equally, and sparse coefficients are solved by a minimization problem. As a result, it ignores the correlation between high-frequency components of low-resolution and the high-frequency components of source images, and solutions to the L0-norm minimization problem. To address these issues, we propose a new image fusion method based on histogram similarity and multi-view weighted sparse representation. By introducing a histogram similarity, different weights are assigned to the high-frequency components of low-resolution and the high-frequency components of source images to efficiently harness the complementary information. In addition, sparse coefficients solved by the L1-norm minimization problem are more accurate. This technique is further incorporated into medical image fusion. Experimental results demonstrate that the proposed method achieves state-of-the-art performance in terms of both visual quality and quantitative evaluation metrics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Banerjee R, Chatterjee S, Bit SD (2019) Performance of a partial discrete wavelet transform based path merging compression technique for wireless multimedia sensor networks[J]. Wirel Pers Commun 104(1):57–71

    Article  Google Scholar 

  2. Bar-Sinai Y, Brenner M, Getreuer P, et al. (2018) Using image super-resolution techniques as a coarse-graining method for physical systems[J]. Bull Am Phys Soc

  3. Chen S, Lu Y, Gao Q et al. (2018) Image fusion based on morphological component analysis via gradient[C]//2018 2nd IEEE advanced information management, communicates, electronic and automation control conference (IMCEC). IEEE: 1034–1037

  4. Cheng B, Powell WB (2018) Co-optimizing battery storage for the frequency regulation and energy arbitrage using multi-scale dynamic programming[J]. IEEE Trans Smart Grid 9(3):1997–2005

    Google Scholar 

  5. Chipman LJ, Orr TM, Graham LN (1995) Wavelets and image fusion. Proc of Int Conf on Image Processing Los Aiamitos:IEEE Computer Society,:248–251

  6. Dai W, Li Y, Zou J et al (2018) Fully decomposable compressive sampling with joint optimization for multidimensional sparse representation[J]. IEEE Trans Signal Process 66(3):603–616

    Article  MathSciNet  Google Scholar 

  7. Daniel E (2018) Optimum wavelet based homomorphic medical image fusion using hybrid genetic–Grey wolf optimization algorithm[J]. IEEE Sensors J

  8. Datta R, Joshi D, Li J, Wang JZ (2008) Image retrieval. ideas, influences, and trends of the new age. ACM Comput Survey 40(2):1–60

    Article  Google Scholar 

  9. Ding R, Wang X, Shang K, et al. (2018) Sparse representation-based intuitionistic fuzzy clustering approach to find the group intra-relations and group leaders for large-scale decision making[J]. IEEE Trans Fuzzy Syst

  10. Ding F, Meng D, Dai J et al (2018) Least squares based iterative parameter estimation algorithm for stochastic dynamical systems with ARMA noise using the model equivalence[J]. Int J Control Autom Syst 16(2):630–639

    Article  Google Scholar 

  11. Ding Z, Zhou D, Nie R et al (2018) Infrared and visible image fusion using modified PCNN and visual saliency detection[C]//2018 international conference on image and video processing, and artificial intelligence. International Society for Optics and Photonics 10836:108360E

    Google Scholar 

  12. Farid MS, Mahmood A, Al-Maadeed SA (2019) Multi-focus image fusion using content adaptive blurring[J]. Inform Fusion 45:96–112

    Article  Google Scholar 

  13. Fu W, Li S, Fang L et al (2018) Contextual online dictionary learning for hyperspectral image classification[J]. IEEE Trans Geosci Remote Sens 56(3):1336–1347

    Article  Google Scholar 

  14. Gharbia R, Hassanien AE, El-Baz AH et al. (2018) Multi-spectral and panchromatic image fusion approach using stationary wavelet transform and swarm flower pollination optimization for remote sensing applications[J]. Futur Gener Comput Syst

  15. Hagargi PA, Shubhangi DC (2018) Brain tumor MR image fusion using Most dominant features extraction from wavelet and Curvelet transforms[J]. Brain 5(05)

  16. Hua G, Zhao L, Zhang H, et al. (2018) Random matching pursuit for image watermarking[J]. IEEE Transactions on Circuits and Systems for Video Technology

  17. Kumar KS, Ram KNNS, Kiranmai K, et al. (2018) Denoising of Iris image using stationary wavelet transform[C]//2018 second international conference on inventive communication and computational technologies (ICICCT). IEEE: 1232–1237

  18. Lai W S, Huang J B, Ahuja N, et al. (2018) Fast and accurate image super-resolution with deep laplacian pyramid networks[J]. IEEE Trans Pattern Anal Mach Intell

  19. Lan X, Zhang S, Yuen PC et al (2018) Learning common and feature-specific patterns: a novel multiple-sparse-representation-based tracker[J]. IEEE Trans Image Process 27(4):2022–2037

    Article  MathSciNet  Google Scholar 

  20. Li S, Ye W, Liang H, et al. (2018) K-SVD based Denoising algorithm for DoFP polarization image sensors[C]//circuits and systems (ISCAS), 2018 IEEE international symposium on. IEEE : 1–5

  21. Li J, Yuan G, Fan H (2019) Multifocus image fusion using wavelet-domain-based deep CNN[J]. Computational Intelligence and Neuroscience 2019

  22. Liang RZ, Shi L, Wang H et al. (2016) Optimizing top precision performance measure of content-based image retrieval by learning similarity function[C]//pattern recognition (ICPR), 2016 23rd international conference on. IEEE : 2954–2958

  23. Liu S, Chen J, Rahardja S (2019) A new multi-focus image fusion algorithm and its efficient implementation[J]. IEEE Transactions on Circuits and Systems for Video Technology

  24. Lucas A, Lopez-Tapiad S, Molinae R, et al. (2019) Generative adversarial networks and perceptual losses for video super-resolution[J]. IEEE Trans Image Process

  25. Ma K, Zeng K, Wang Z (2015) Perceptual quality assessment for multi-exposure image fusion[J]. IEEE Trans Image Process 24(11):3345–3356

    Article  MathSciNet  Google Scholar 

  26. Ma J, Yu W, Liang P et al (2019) FusionGAN: a generative adversarial network for infrared and visible image fusion[J]. Inform Fusion 48:11–26

    Article  Google Scholar 

  27. Mancini M, Costante G, Valigi P et al (2018) J-MOD 2: joint monocular obstacle detection and depth estimation[J]. IEEE Robot Auto Lett 3(3):1490–1497

    Article  Google Scholar 

  28. Massa A, Bertolli M, Gottardi G et al. (2018) Compressive sensing as applied to antenna arrays: synthesis, diagnosis, and processing[C]//circuits and systems (ISCAS), 2018 IEEE international symposium on. IEEE : 1–5

  29. Nguyen LD, Lin D, Lin Z, et al. (2018) Deep CNNs for microscopic image classification by exploiting transfer learning and feature concatenation[C]//circuits and systems (ISCAS), 2018 IEEE international symposium on. IEEE : 1–5

  30. Piella G, Heijmans H (2003) A new quality metric for image fusion. Process IEEE Int Conf Image Process 3:173–176

    Google Scholar 

  31. Ra PK, Karmakar G, Eddy FYS et al. (2018) Dual tree complex wavelet transform based detection of power quality disturbances[C]//2018 IEEE innovative smart grid technologies-Asia (ISGT Asia). IEEE : 1177–1182

  32. Raj A, Pradhan J, Pal AK, et al. (2018) Multi-scale image fusion scheme based on non-sub sampled contourlet transform and four neighborhood Shannon entropy scheme[C]//2018 4th international conference on recent advances in information technology (RAIT). IEEE : 1–6

  33. Srinivasan A, Battacharjee P, Prasad A, et al. (2018) Brain MR image analysis using discrete wavelet transform with fractal feature analysis[C]//2018 second international conference on electronics, communication and aerospace technology (ICECA). IEEE : 1660–1664

  34. Wang Z, Bovik AC (2002) A universal image quality index. IEEE Signal Process Lett 9(3):81–84

    Article  Google Scholar 

  35. Wang X, Yu K, Dong C, et al. (2018) Recovering realistic texture in image super-resolution by deep spatial feature transform[J]. arXiv preprint arXiv:1804.02815

  36. Wang Y, Zheng Q, Heng PA (2018) Online robust projective dictionary learning: shape modeling for MR-TRUS registration[J]. IEEE Trans Med Imaging 37(4):1067–1078

    Article  Google Scholar 

  37. Wang M, Zhou S, Yang Z et al (2019) Image fusion based on wavelet transform and gray-level features[J]. J Mod Opt 66(1):77–86

    Article  Google Scholar 

  38. Wen Y, Sheng B, Li P et al (2019) Deep color guided coarse-to-fine convolutional network cascade for depth image super-resolution[J]. IEEE Trans Image Process 28(2):994–1006

    Article  MathSciNet  Google Scholar 

  39. Yan C, Xie H, Liu S et al (2018) Effective Uyghur language text detection in complex background images for traffic prompt identification[J]. IEEE Trans Intell Transp Syst 19(1):220–229

    Article  Google Scholar 

  40. Yang J, Chen X, Hu Y H, et al. (2018) Adaptive visual target tracking based on label consistent K-Svd sparse coding and kernel particle filter[C]//2018 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE: 1633–1637

  41. Yazdi SV, Douzal-Chouakria A (2018) Time warp invariant kSVD: sparse coding and dictionary learning for time series under time warp[J]. Pattern Recogn Lett 112:1–8

    Article  Google Scholar 

  42. Ye Q, Zhao H, Li Z et al (2018) L1-norm distance minimization-based fast robust twin support vector $ k $-plane clustering[J]. IEEE Trans Neural Netw Learn Syst 29(9):4494–4503

    Article  Google Scholar 

  43. Yin H (2018) Tensor sparse representation for 3-D medical image fusion using weighted average rule[J]. IEEE Trans Biomed Eng

  44. Yin H, Li S, Fang L (2013) Simultaneous image fusion and super-resolution using sparse representation[J]. Inform Fusion 14(3):229–240

    Article  Google Scholar 

  45. Zhang Q, Guo BL (2009) Multifocus image fusion using the nonsubsampled contourlet transform. Signal Process 89(7):1334–1346

    Article  Google Scholar 

  46. Zhang M, Li W, Du Q (2018) Diverse region-based CNN for hyperspectral image classification[J]. IEEE Trans Image Process 27(6):2623–2634

    Article  MathSciNet  Google Scholar 

  47. Zhang Y, Chandler DM, Mou X (2018) Quality assessment of screen content images via convolutional-neural-network-based synthetic/natural segmentation[J]. IEEE Trans Image Process 27(10):5113–5128

    Article  MathSciNet  Google Scholar 

  48. Zhao W, Lu H, Wang D (2018) Multisensor image fusion and enhancement in spectral total variation domain[J]. IEEE Trans Multimed 20(4):866–879

    Article  Google Scholar 

  49. Zhou T, Liu F, Bhaskar H et al (2018) Robust visual tracking via online discriminative and low-rank dictionary learning[J]. IEEE Trans Cybernet 48(9):2643–2655

    Article  Google Scholar 

  50. Zhu Z, Zheng M, Qi G, et al. (2019) A phase congruency and local Laplacian energy based multi-modality medical image fusion method in NSCT domain[J]. IEEE Access

Download references

Acknowledgements

The authors first sincerely thank the editors and anonymous reviewers for their constructive comments and suggestions, which are of great value to us. The authors would also like to thank Prof. Xiaojun Wu from Jiangnan University and researcher Zhenhua Feng from University of Surrey.

This work was supported in part by the National Natural Science Foundation of China under Grants 61702293.This work was supported in part by the Shandong Provincial Natural Science Foundation of China under Grants ZR2017QF015.

Author information

Authors and Affiliations

  1. College of Data Science Software Engineering, Qingdao University, Qingdao, Shandong, 266071, People’s Republic of China

    Yi Li, Zhihan Lv & Junli Zhao

  2. College of Computer Science & Technology, Qingdao University, Qingdao, Shandong, 266071, People’s Republic of China

    Zhenkuan Pan

Authors
  1. Yi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhihan Lv
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Junli Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhenkuan Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Yi Li or Zhihan Lv.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Y., Lv, Z., Zhao, J. et al. Improving performance of medical image fusion using histogram, dictionary learning and sparse representation. Multimed Tools Appl 78, 34459–34482 (2019). https://doi.org/10.1007/s11042-019-08027-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-08027-9

Keywords

Search

Navigation