Skip to main content
SpringerLink
Log in

An improved multi-focus image fusion algorithm based on multi-scale weighted focus measure

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

This paper focuses on developing an improved multi-focus image fusion (MFIF) algorithm. Existing spatial domain algorithms dependent on the obtained fusion decision map still lead to unexpected ghosting, blurred, edges as well as blocking effects such that the visual effect of image fusion is seriously degraded. To overcome these shortages, an improved MFIF algorithm is developed with the help of a novel multi-scale weighted focus measure and a decision map optimization technique. First, a novel multi-scale measurement template is designed in order to effectively extract the gradient information of rich texture regions, smooth regions as well as transitional regions between the aforementioned regions simultaneously. Then, an improved calculation scheme of the focus score matrix is designed based on the weighted sum of the focus measure maps in each region window centered on a concerned pixel, under which the advantage of pixel-by-pixel weighting is employed. In what follows, an initial decision map is obtained in light of the focus score matrix combined with threshold filtering, which is employed to eliminate the small isolated regions caused by some misclassified pixels. Furthermore, an accurate decision map is received with the help of the optimization capability of guided filtering to avoid edge unexpected artificial textures. In comparison with block-based fusion algorithms, our algorithm developed in this paper extracts the focus regions pixel-by-pixel, thereby helping to reduce the blocking effects that appear in the fusion image. Finally, some intensive comparison analysis based on common datasets is performed to verify the superiority over state-of-the-art methods in both visual qualitative and quantitative evaluations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Li H, He X, Tao D, Tang Y, Wang R (2018) Joint medical image fusion, denoising and enhancement via discriminative low-rank sparse dictionaries learning. Pattern Recongn 79:130–146

    Article  Google Scholar 

  2. Ma J, Ma Y, Li C (2019) Infrared and visible image fusion methods and applications: a survey. Inf Fusion 45:153–178

    Article  Google Scholar 

  3. Yin M, Duan P, Liu W, Liang X (2017) A novel infrared and visible image fusion algorithm based on shift-invariant dual-tree complex shearlet transform and sparse representation. Neurocomputing 226:182–191

    Article  Google Scholar 

  4. Tian J, Liu G, Liu J (2018) Multi-focus image fusion based on edges and focused region extraction. Optik 171:611–624

    Article  Google Scholar 

  5. Meher B, Agrawal S, Panda R, Abraham A (2019) A survey on region based image fusion methods. Inf Fusion 48:119–132

    Article  Google Scholar 

  6. Jin H, Xing B, Wang L, Wang Y (2015) Fusion of remote sensing images based on pyramid decomposition with Baldwinian Clonal Selection Optimization. Infrared Phys Technol 73 :204–211

    Article  Google Scholar 

  7. Yang Z, Mu X, Zhao F (2018) Scene classification of remote sensing image based on deep network and multi-scale features fusion. Optik 171:287–293

    Article  Google Scholar 

  8. Gupta K, Walia GS, Sharma K (2020) Quality based adaptive score fusion approach for multimodal biometric system. Appl Intell 50(4):1086–1099

    Article  Google Scholar 

  9. Li S, Kang X, Fang L, Hu J, Yin H (2017) Pixel-level image fusion: a survey of the state of the art. Inf Fusion 33:100–112

    Article  Google Scholar 

  10. Zhao C, Wang X, Zuo W, Shen F, et al (2020) Similarity learning with joint transfer constraints for person re-identification. Pattern Recogn 97:107014

    Article  Google Scholar 

  11. Zhao C, Chen K, Zang D, Zhang Z, et al (2019) Uncertainty-optimized deep learning model for small-scale person re-identification. Sci China Inf Sci 62:220102

    Article  Google Scholar 

  12. Zhao C, Chen K, Wei Z, Chen Y, et al (2018) Multilevel triplet deep learning model for person re-identification. Pattern Recogn Lett 117:161–168

    Article  Google Scholar 

  13. Zhang Y, Liu Y, Sun P, Yan H, et al (2020) IFCNN: A general image fusion framework based on convolutional neural network. Inf Fusion 54:99–118

    Article  Google Scholar 

  14. Burt P, Adelson E (1983) The Laplacian pyramid as a compact image code. IEEE Trans Commun 31(4):532–540

    Article  Google Scholar 

  15. Zhao W, Xu Z, Zhao J (2016) Gradient entropy metric and p-Laplace diffusion constraint-based algorithm for noisy multispectral image fusion. Inf Fusion 27:131–142

    Article  Google Scholar 

  16. Zhang Q, Guo BL (2009) Multifocus image fusion using the nonsubsampled contourlet transform. Signal Process 89(7):1334–1346

    Article  MATH  Google Scholar 

  17. Li H, Manjunath BS, Mitra SK (1995) Multisensor image fusion using the wavelet transform. Graph Mod Image Process 57(3):235–245

    Article  Google Scholar 

  18. Lewis JJ, O’Callaghan RJ, Nkiolov SG, Bull DR, Canagarajah N (2007) Pixel-and region-based image fusion with complex wavelets. Inf Fusion 8(2):119–130

    Article  Google Scholar 

  19. Yang B, Li S (2010) Multifocus image fusion and restoration with sparse representation. IEEE Trans Instrum Meas 59(4):884–892

    Article  Google Scholar 

  20. Liu Y, Liu S, Wang Z (2015) A general framework for image fusion based on multi-scale transform and sparse representation. Inf Fusion 24:147–164

    Article  Google Scholar 

  21. Tang H, Xiao B, Li W, Wang W (2018) Pixel convolutional neural network for multi-focus image fusion. Inf Sci 433-434:125–141

    Article  MathSciNet  Google Scholar 

  22. Mitianoudis N, Stathaki T (2007) Pixel-based and region-based image fusion schemes using ICA bases. Inf Fusion 8(2):131–142

    Article  MATH  Google Scholar 

  23. Liang J, He Y, Liu D, Zeng X (2012) Image fusion using higher order singular value decomposition. IEEE Trans Image Process 21(5):2898–2909

    Article  MathSciNet  MATH  Google Scholar 

  24. Saha A, Bhatnagar G, Wu QMJ (2013) Mutual spectral residual approach for multifocus image fusion. Digit Signal Process 23(4):1121–1135

    Article  MathSciNet  Google Scholar 

  25. De I, Chanda B (2013) Multi-focus image fusion using a morphology-based focus measure in a quad-tree structure. Inf Fusion 14(2):136–146

    Article  Google Scholar 

  26. Bai X, Zhang Y, Zhou F, Xue B (2015) Quadtree-based multi-focus image fusion using a weighted focus-measure. Inf Fusion 22:105–118

    Article  Google Scholar 

  27. Zhang Y, Bai X, Wang T (2017) Boundary finding based multi-focus image fusion through multi-scale morphological focus-measure. Inf Fusion 35:81–101

    Article  Google Scholar 

  28. Liu Y, Liu S, Wang Z (2015) Multi-focus image fusion with dense SIFT. Inf Fusion 23:139–155

    Article  Google Scholar 

  29. Zhou Z, Li S, Wang B (2014) Multi-scale weighted gradient-based fusion for multi-focus image. Inf Fusion 20:60–72

    Article  Google Scholar 

  30. Wang Z, Wang S, Zhu Y (2017) Multi-focus image fusion based on the improved PCNN and guided filter. Neural Process Lett 45(1):75–94

    Article  Google Scholar 

  31. Du C, Gao S (2018) Multi-focus image fusion algorithm based on pulse coupled neural networks and modified decision map. Optik 157:1003–1015

    Article  Google Scholar 

  32. Liu Y, Chen X, Peng H, Wang Z (2017) Multi-focus image fusion with a deep convolutional neural network. Inf Fusion 36 :191–207

    Article  Google Scholar 

  33. Naji MA, Aghagolzadeh A, Ezoji M (2019) Ensemble of CNN for multi-focus image fusion. Inf Fusion 51:201–214

    Article  Google Scholar 

  34. Huang W, Jing Z (2007) Evaluation of focus measures in multi-focus image fusion. Pattern Recogn Lett 28(4):493–500

    Article  Google Scholar 

  35. Nayar SK, Nakagawa Y (1994) Shape from focus. IEEE Trans Pattern Anal Mach Intell 16 (8):824–831

    Article  Google Scholar 

  36. Ahmed K T, Trtaza A, Iqbal MA (2017) Fusion of local and global features for effective image extraction. Appl Intell 47:526–543

    Article  Google Scholar 

  37. Zhang P, Gao W, Liu G (2018) Feature selection considering weighted relevancy. Appl Intell 48:4615–4625

    Article  Google Scholar 

  38. He K, Sun J, Tang X (2010) Guided image filtering. In: European Conference on Computer Vision. Heraklion, Greece, pp 1–14

  39. Qu G, Zhang D, Yan P (2002) Information measure for performance of image fusion. Electron Lett 38(7):313

    Article  Google Scholar 

  40. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13:600–612

    Article  Google Scholar 

  41. Zhao W, Wang D, Lu H (2019) Multi-focus image fusion with a natural enhancement via joint multi-level deeply supervised convolutional neural network. IEEE Trans Circ Syst Video Technol 29(4):1102–1115

    Article  Google Scholar 

  42. Nizami IF, Majid M, Khurshid K (2018) New feature selection algorithms for no-reference image quality assessment. Appl Intell 48:3482–3501

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Control Science and Engineering, University of Shanghai for Science and Technology, Shanghai, 200093, China

    Zhanhui Hu, Wei Liang & Derui Ding

  2. College of Science, University of Shanghai for Science and Technology, Shanghai, 200093, China

    Guoliang Wei

Authors
  1. Zhanhui Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Derui Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Guoliang Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Derui Ding.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported in part by the National Natural Science Foundation of China under Grants 61973219 and 61933007.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hu, Z., Liang, W., Ding, D. et al. An improved multi-focus image fusion algorithm based on multi-scale weighted focus measure. Appl Intell 51, 4453–4469 (2021). https://doi.org/10.1007/s10489-020-02066-8

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-02066-8

Keywords

Search

Navigation