Skip to main content
SpringerLink
Log in

A novel multiscale transform decomposition based multi-focus image fusion framework

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In this work, we propose a novel multiscale transform decomposition model for multi-focus image fusion to get a better fused performance. The motivation of the proposed fusion framework is to make full use of the decomposition characteristics of multiscale transform. The nonsubsampled contourlet transform (NSCT) is firstly used to decompose the source multi-focus images into low-frequency (LF) and several high-frequency (HF) bands to separate out the two basic characteristics of source images, i.e., principal information and edge details. The common “average” and “max-absolute” fusion rules are performed on low- and high-frequency components, respectively, and a basic fusion image is generated. Then the difference images between the basic fused image and the source images are calculated, and the energy of the gradient (EOG) of difference images are utilized to refine the basic fused image by integrating average filter and median filter. Visual and quantitative using fusion metrics like VIFF, QS, MI, QAB/F, SD, QPC and running time comparisons to state-of-the-art algorithms demonstrate the out-performance of the proposed fusion technique.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Arif M, Wang G (2020) Fast curvelet transform through genetic algorithm for multimodal medical image fusion. Soft Comput 24(3):1815–1836

    Article  Google Scholar 

  2. Cao L, Dey N, Ashour A (2020) Diabetic plantar pressure analysis using image fusion. Multimed Tools Appl 79(15–16):11213–11236

    Article  Google Scholar 

  3. Ch M, Riaz M, Iltaf N (2019) Weighted image fusion using cross bilateral filter and non-subsampled contourlet transform. Multidimensional Syst Signal Process 30(4):2199–2210

    Article  MATH  Google Scholar 

  4. Chen Y, Xu W, Zuo J (2019) The fire recognition algorithm using dynamic feature fusion and IV-SVM classifier. Cluster Comput 22(3):S7665–S7675

    Article  Google Scholar 

  5. Chen Y, Wang J, Liu S (2019) Multiscale fast correlation filtering tracking algorithm based on a feature fusion model. Concurrency and Computation: Practice and Experience e5533.

  6. Chen Y, Wang J, Xia R (2019) The visual object tracking algorithm research based on adaptive combination kernel. J Ambient Intell Human Comput 10(12):4855–4867

    Article  Google Scholar 

  7. Chen Y, Xiong J, Xu W (2019) A novel online incremental and decremental learning algorithm based on variable support vector machine. Cluster Comput 22(3):S7435–S7445

    Article  Google Scholar 

  8. Chen Y, Wang J, Chen X (2019) Single-image super-resolution algorithm based on structural self-similarity and deformation block features. IEEE Access 7:58791–58801

    Article  Google Scholar 

  9. Chen Y, Wang J, Chen X (2019) Image super-resolution algorithm based on dual-channel convolutional neural networks. Applied Sciences-Basel 9(11), UNSP 2316.

  10. Chen J, Li X, Luo L (2020) Infrared and visible image fusion based on target-enhanced multiscale transform decomposition. Inform Sci 508:64–78

    Article  Google Scholar 

  11. Chen Y, Tao J, Liu L (2020) Research of improving semantic image segmentation based on a feature fusion model. Journal of Ambient Intelligence and Humanized Computing. (Online)

  12. Chen Y, Tao J, Zhang Q (2020) Saliency detection via the improved hierarchical principal component analysis method. Wireless Commun Mobile Comput 2020:8822777

    Google Scholar 

  13. Chen Y, Liu L, Tao J (2020) The improved image inpainting algorithm via encoder and similarity constraint. The Visual Computer. (Online)

  14. Chu T, Tan Y, Liu Q (2020) Novel fusion method for SAR and optical images based on non-subsampled shearlet transform. Int J Remote Sens 41(12):4588–4602

    Article  Google Scholar 

  15. Da C, Zhou J, Do M (2006) The nonsubsampled contourlet transform: theory, design, and applications. IEEE Trans Image Process 15(10):3089–3101

    Article  Google Scholar 

  16. Du J, Li W (2020) Two-scale image decomposition based image fusion using structure tensor. Int J Imaging Syst Technol 30(2):271–284

    Article  Google Scholar 

  17. Han Y, Cai Y, Cao Y (2013) A new image fusion performance metric based on visual information fidelity. Inf Fusion 14(2):127–135

    Article  Google Scholar 

  18. Hayat N, Imran M (2020) Detailed and enhanced multi-exposure image fusion using recursive filter. Multimed Tools Appl 79(33–34):25067–25088

    Article  Google Scholar 

  19. He K, Zhou D, Zhang X (2019) Multi-focus image fusion combining focus-region-level partition and pulse-coupled neural network. Soft Comput 23(13):4685–4699

    Article  Google Scholar 

  20. Hu Q, Hu S, Zhang F (2020) Multi-modality medical image fusion based on separable dictionary learning and Gabor filtering. Signal Process: Image Commun 83:115758

    Google Scholar 

  21. Huang X, Qi G, Wei H (2019) A novel infrared and visible image information fusion method based on phase congruency and image entropy. Entropy 21(12):1135

    Article  MathSciNet  Google Scholar 

  22. Kaur M, Singh D (2020) Multi-modality medical image fusion technique using multi-objective differential evolution based deep neural networks. Journal of Ambient Intelligence and Humanized Computing. (Online)

  23. Kumar B (2015) Image fusion based on pixel significance using cross bilateral filter. Signal Image Video Process 9(5):1193–1204

    Article  Google Scholar 

  24. Li S, Kang XHJ (2013) Image fusion with guided filtering. IEEE Trans Image Process 22(7):2864–2875

    Article  Google Scholar 

  25. Li L, Si Y (2019) Enhancement of hyperspectral remote sensing images based on improved fuzzy contrast in nonsubsampled shearlet transform domain. Multimed Tools Appl 78(13):18077–18094

    Article  Google Scholar 

  26. Li L, Si Y, Jia Z (2018) A novel brain image enhancement method based on nonsubsampled contourlet transform. Int J Imaging Syst Technol 28(2):124–131

    Article  Google Scholar 

  27. Li L, Wang L, Wang Z (2019) A novel medical image fusion approach based on nonsubsampled shearlet transform. J Medical Imaging Health Inf 9(9):1815–1826

    Article  Google Scholar 

  28. Li L, Si Y, Wang L (2020) Brain image enhancement approach based on singular value decomposition in nonsubsampled shearlet transform domain. J Med Imaging Health Inf 10(8):1785–1794

    Article  Google Scholar 

  29. Li B, Peng H, Wang J (2020) Multi-focus image fusion based on dynamic threshold neural P systems and surfacelet transform. Knowledge-based Syst 196:105794

    Article  Google Scholar 

  30. Li L, Tan L, Li S (2020) Image fusion based on convolution sparse representation and pulse coupled neural network in non-subsampled contourlet domain. International Journal of Embedded Systems 12(1):1–10

    Article  MathSciNet  Google Scholar 

  31. Li L, Si Y, Wang L, Jia Z, Ma H (2020) A novel approach for multi-focus image fusion based on SF-PAPCNN and ISML in NSST domain. Multimed Tools Appl 79(33–34):24303–24328

    Article  Google Scholar 

  32. Li Y, Wang J, Miao Z (2020) Unsupervised densely attention network for infrared and visible image fusion. Multimedia Tools and Applications. (Online)

  33. Liao X, Yin J, Chen M (2020) Adaptive payload distribution in multiple images steganography based on image texture features. IEEE Transactions on Dependable and Secure Computing. Online.

  34. Liao X, Li K, Zhu X (2020) Robust detection of image operator chain with two-stream convolutional neural network. IEEE Journal of Selected Topics in Signal Processing 14(5):955–968

    Article  Google Scholar 

  35. Liu Z, Blasch E, Xue Z (2012) Objective assessment of multiresolution image fusion algorithms for context enhancement in night vision: a comparative study. IEEE Trans Pattern Analysis Machine Intell 34(1):94–109

    Article  Google Scholar 

  36. Liu Y, Liu S, Wang Z (2015) A general framework for image fusion based on multi-scale transform and sparse representation. Inf Fusion 24:147–164

    Article  Google Scholar 

  37. Liu Y, Chen X, Ward R (2016) Image fusion with convolutional sparse representation. IEEE Signal Process Lett 23(12):1882–1886

    Article  Google Scholar 

  38. Liu S, Wang J, Lu Y (2019) Multi-focus image fusion based on adaptive dual-channel spiking cortical model in non-subsampled shearlet domain. IEEE Access 7:56367–56388

    Article  Google Scholar 

  39. Liu Y, Chen X, Ward R (2019) Medical image fusion via convolutional sparsity based morphological component analysis. IEEE Signal Process Lett 26(3):485–489

    Article  Google Scholar 

  40. Luo Y, Qin J, Xiang X (2020) Coverless real-time image information hiding based on image block matching and dense convolutional network. Journal of Real-Time Image Processing 17(1):125–135

    Article  Google Scholar 

  41. Ma J, Ma Y, Li C (2019) Infrared and visible image fusion methods and applications: A survey. Inf Fusion 45:153–178

    Article  Google Scholar 

  42. Mustafa H, Zareapoor M, Yang J (2020) MLDNet: Multi-level dense network for multi-focus image fusion. Signal Process: Image Commun 85:115864

    Google Scholar 

  43. Panigrahy C, Seal A, Mahato N (2020) Fractal dimension based parameter adaptive dual channel PCNN for multi-focus image fusion. Optics and Lasers in Engineering 133, UNSP106141.

  44. Piella G, Heijmans H (2003) A new quality metric for image fusion. IEEE Int Conference Image Process 3:173–176

    Google Scholar 

  45. Qiu X, Li M, Zhang L (2019) Guided filter-based multi-focus image fusion through focus region detection. Signal Process: Image Commun 72:35–46

    Google Scholar 

  46. Qu X, Yan J, Xiao H (2008) Image fusion algorithm based on spatial frequency-motivated pulse coupled neural networks in nonsubsampled contourlet transform domain. Acta Automatica Sinica 34(12):1508–1514

    Article  MATH  Google Scholar 

  47. Ramlal S, Sachdeva J, Ahuja C (2019) An improved multimodal medical image fusion scheme based on hybrid combination of nonsubsampled contourlet transform and stationary wavelet transform. Int J Imaging Syst Technol 29(2):146–160

    Article  Google Scholar 

  48. Singh S, Anand R (2020) Multimodal medical image sensor fusion model using sparse K-SVD dictionary learning in nonsubsampled shearlet domain. IEEE Trans Instrumentation Measurement 69(2):593–607

    Article  Google Scholar 

  49. Singh R, Srivastava R, Prakash O (2012) Multimodal medical image fusion in dual tree complex wavelet transform domain using maximum and average fusion rules. J Med Imaging Health Inf 2(2):168–173

    Article  Google Scholar 

  50. Tan W, Zhou H, Song J (2019) Infrared and visible image perceptive fusion through multi-level Gaussian curvature filtering image decomposition. Appl optics 58(12):3064–3073

    Article  Google Scholar 

  51. Wang D, Xiao Y, Gao Y (2019) Image denoising method based on NSCT bivariate model and variational Bayes threshold estimation. Multimed Tools Appl 78(7):8927–8941

    Article  Google Scholar 

  52. Wang K, Zheng M, Wei H (2020) Multi-modality medical image fusion using convolutional neural network and contrast pyramid. Sensors 20(8):2169

    Article  Google Scholar 

  53. Yang Y, Park D, Huang S (2010) Fusion of CT and MR images using an improved wavelet based method. J X Ray Sci Technol 18(2):157–170

    Article  Google Scholar 

  54. Yin M, Liu X, Liu Y (2019) Medical image fusion with parameter-adaptive pulse coupled neural network in nonsubsampled shearlet transform domain. IEEE Trans Instrumentation Measurement 68(1):49–64

    Article  Google Scholar 

  55. Yu F, Liu L, Xiao L (2019) A robust and fixed-time zeroing neural dynamics for computing time-variant nonlinear equation using a novel nonlinear activation function. Neurocomputing 350:108–116

    Article  Google Scholar 

  56. Yu F, Liu L, He B (2019) Analysis and FPGA realization of a novel 5D hyperchaotic four-wing memristive system, active control synchronization, and secure communication application. Complexity 2019:4047957

    Article  Google Scholar 

  57. Zhang Q, Li G, Cao Y (2020) Multi-focus image fusion based on non-negative sparse representation and patch-level consistency rectification. Pattern Recognition 104:107325

    Article  Google Scholar 

  58. Zhang Y, Liu Y, Sun P (2020) IFCNN: A general image fusion framework based on convolutional neural network. Inf Fusion 54:99–118

    Article  Google Scholar 

  59. Zhang J, Xie Z, Sun J (2020) A cascaded R-CNN with multiscale attention and imbalanced samples for traffic sign detection. IEEE Access 8:29742–29754

    Article  Google Scholar 

  60. Zhang L, Jia Z, Koefoed L (2020) Remote sensing image enhancement based on the combination of adaptive nonlinear gain and the PLIP model in the NSST domain. Multimedia Tools and Applications 79(19–20):13647–13665

    Article  Google Scholar 

  61. Zhao J, Laganiere R, Liu Z (2007) Performance assessment of combinative pixel-level image fusion based on an absolute feature measurement. Int J Innovative Comput Inf Control 3(6A):1433–1447

    Google Scholar 

  62. Zhong Z, Gao W, Khattak A (2020) A novel multi-source image fusion method for pig-body multi-feature detection in NSCT domain. Multimed Tools Appl 79:26225–26244

    Article  Google Scholar 

  63. Zhou J, Hao M, Zhang D (2019) Fusion PSPnet image segmentation based method for multi-focus image fusion. IEEE Photonics J 11(6):6501412

    Article  Google Scholar 

  64. Zhu Z, Zheng M, Qi G (2019) A phase congruency and local laplacian energy based multi-modality medical image fusion method in NSCT domain. IEEE Access 7:20811–20824

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by the Shanghai Aerospace Science and Technology Innovation Fund under Grant No. SAST2019-048; the National Natural Science Foundation of China under Grant Nos. U1803261 and 61665012.

Author information

Authors and Affiliations

  1. Department of Electronic Engineering, Tsinghua University, Beijing, 100084, China

    Liangliang Li & Hongbing Ma

  2. College of Information Science and Engineering, Xinjiang University, Urumqi, 830046, China

    Zhenhong Jia

  3. College of Communication Engineering, Jilin University, Changchun, 130012, China

    Yujuan Si

Authors
  1. Liangliang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hongbing Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhenhong Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yujuan Si
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hongbing Ma.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, L., Ma, H., Jia, Z. et al. A novel multiscale transform decomposition based multi-focus image fusion framework. Multimed Tools Appl 80, 12389–12409 (2021). https://doi.org/10.1007/s11042-020-10462-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-10462-y

Keywords

Search

Navigation