Skip to main content
SpringerLink
Log in

Multifocus image fusion using a convolutional elastic network

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The aim of multifocus image fusion is to fuse two or more partially focused images into one fully focused image. To overcome the problem of a limited depth of field and blurred imaging of objects beyond the depth of field in optical imaging systems, a multifocus image fusion method based on a convolutional elastic network is proposed. Each source image is first decomposed into a base layer and a detail layer using the fast Fourier transform. Then, the convolutional elastic network performs fusion of the detail layers while applying the “choose-max” fusion rule to the base layers. Finally, the fused image is reconstructed by a two-dimensional inverse discrete Fourier transform. To verify the effectiveness of the proposed algorithm, we applied it and seven other popular methods to sets of multifocus images. The experimental results show that the proposed method overcomes the shortcomings of low spatial resolution and ambiguity in multifocus image fusion and achieves better contrast and clarity. In terms of both subjective visual effects and objective indicators, the performance of our method is optimal in comparation with other state-of-the-art fusion methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Boyd S, Parikh N (2010) Distributed optimization and statistical learning via the alternating direction method of multipliers. Found Trends Mach Learn 3:1–122

    Article  Google Scholar 

  2. Chai Y (2012) Multifocus image fusion based on features contrast of multiscale products in nonsubsampled contourlet transform domain. Optik - Int J Light Electron Opt 123(7):569–581

    Article  Google Scholar 

  3. Gao Z, Zhang C (2016) Texture clear multi-modal image fusion with joint sparsity model. Optik - Int J Light Electron Opt 121305:S0030402616310725

    Google Scholar 

  4. Goodfellow IJ, Pouget-Abadie J (2014) Generative adversarial nets. In: International conference on neural information processing systems

  5. Goshtasby AA, Nikolov S (2007) Guest editorial: image fusion: advances in the state of the art. Inf Fusion 8(2):114–118

    Article  Google Scholar 

  6. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based leaning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  7. Lei Z, Xu JL, Li HS, Ming LZ, E LX (2015) Fusion of infrared and visual images based on non-sampled contourlet transform and region classification. Opt Precis Eng 23:810–818

    Article  Google Scholar 

  8. Liu Z et al (2011) Objective assessment of multiresolution image fusion algorithms for context enhancement in night vision: a comparative study. IEEE Trans Pattern Anal Machine Intell 34(1):94–109

    Article  Google Scholar 

  9. Liu Y, Liu S, Wang Z (2015) A general framework for image fusion based on multi-scale transform and sparse representation. Inf Fusion 24:147–164

    Article  Google Scholar 

  10. Liu Y, Chen X, Ward RK (2016) Image fusion with convolutional sparse representation. IEEE Signal Process Lett 23:1882–1886

    Article  Google Scholar 

  11. Liu Y et al (2017) Multi-focus image fusion with a deep convolutional neural network. Inf Fusion 36:191–207

    Article  Google Scholar 

  12. Liu F, Chen L, Lu L, Ahmad A, Jeon G, Yang X (2019) Medical image fusion method by using Laplacian pyramid and convolutional sparse representation. Concurrency Computation Practice Experience

  13. Nirmalraj S, Nagarajanb G (2020) Fusion of visible and infrared image via compressive sensing using convolutional sparse representation. ICT Express online

  14. Piella G, Heijmans H (2003) A new quality metric for image fusion. In: International conference on image processing

  15. Qiu C, Peng W, Wang Y, Hong J, Xia S (2019) Fusion of misregistered GFP and phase contrast images with convolutional sparse representation and adaptive region energy rule. Microsc Res Tech 12:83

    Google Scholar 

  16. Qu G, Zhang D, Yan P (2002) Information measure for performance of image fusion. Electron Lett 38:313–315

    Article  Google Scholar 

  17. Wang PW, Bo L (2008) A novel image fusion metric based on multi-scale analysis. In: International conference on signal processing

  18. Wang Q, Shen Y, Jin J (2008) Performance evaluation of image fusion techniques. Image Fusion 469–492

  19. Wohlberg B (2016) Efficient algorithms for convolutional sparse representations. IEEE Trans Image Process 25:301–315

    Article  MathSciNet  Google Scholar 

  20. Xing C, Wang Z, Ouyang Q, Dong C, Duan C (2019) Image fusion method based on spatially masked convolutional sparse representation. Image Vis Comput 90:103806.1–103806.12

    Article  Google Scholar 

  21. Xing C, Wang M, Dong C, Duan C, Wang Z (2020) Using Taylor expansion and convolutional sparse representation for image fusion. Neurocomputing 402:437–455

    Article  Google Scholar 

  22. Xinxiang LI, Zhang L, Wang L, Zhou X (2019) Image fusion method based on convolutional sparse representation and morphological component analysis. Intell Comput Appl 13:24–31

    Google Scholar 

  23. Xydeas CS, Petrovic VS (2000) Objective image fusion performance measure. Electron Lett 36:308–309

    Article  Google Scholar 

  24. Yang Y (2011) A novel DWT based multi-focus image fusion method. Proc Eng 24(1):177–181

    Article  Google Scholar 

  25. Yang B, Li S (2006) Multifocus image fusion and restoration with sparse representation. IEEE Trans Image Process 15(2):3736–3745

    MathSciNet  Google Scholar 

  26. Yang B, Li S (2014) Visual attention guided image fusion with sparse representation. Optik—Int J Light Electron Opt 125:4881–4888

    Article  Google Scholar 

  27. Yin H, Li S (2011) Multimodal image fusion with joint sparsity model. Opt Eng 50:067007–1–067007-10

    Article  Google Scholar 

  28. Yu L, Wang Z (2014) Simultaneous image fusion and denoising with adaptive sparse representation. Image Process Iet 9:347–357

    Google Scholar 

  29. Zhang C (2020) Multifocus image fusion using multiscale transform and convolutional sparse representation. Int J Wavelets Multiresolution Inf Process 11:2050061–1–2050061-29

    MATH  Google Scholar 

  30. Zhang C, Yang X (2020) Visible and infrared image fusion based on masked online convolutional dictionary learning with frequency domain computation. In: The 1st international conference on agriculture and IT/iot/ICT (ICAIT2019)

  31. Zhang C, Yang X (2020) Visible and infrared image fusion based on online convolutional dictionary learning with sparse matrix computation. In: The 3rd international conference on wireless communications and applications (ICWCA2019)

  32. Zhang C, Yi L (2019) Multimodal image fusion with adaptive joint sparsity model. J Electron Imaging 28:013043

    Google Scholar 

  33. Zhang C, Feng Z, Gao Z, Jin X, Yan D, Yi L (2020) Salient feature multimodal image fusion with a joint sparse model and multiscale dictionary learning. Opt Eng 59:051402

    Google Scholar 

  34. Zhou ZQ, Wang B (2016) Perceptual fusion of infrared and visible images through a hybrid multi-scale decomposition with Gaussian and bilateral filters. Inf Fusion 30:15–26

    Article  Google Scholar 

  35. Zhou Z, Sun L, Bo W (2014) Multi-scale weighted gradient-based fusion for multi-focus images. Inf Fusion 20:60–72

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by the Sichuan Science and Technology Program (No.2020YFS0351), Luzhou Science and Technology Program (No.2019-SYF-34) and Scientific Research Project of Sichuan Public Security Department (No. 201917). We thank AJE (www.aje.com) for its linguistic assistance during the preparation of this manuscript.

Author information

Authors and Affiliations

  1. National Key Laboratory of Fundamental Science on Synthetic Vision, Sichuan University, Chengdu, 610039, China

    Chengfang Zhang

  2. The Center of Laboratory and Equipment, Sichuan Police College, Luzhou, 646000, China

    Chengfang Zhang

Authors
  1. Chengfang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chengfang Zhang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, C. Multifocus image fusion using a convolutional elastic network. Multimed Tools Appl 81, 1395–1418 (2022). https://doi.org/10.1007/s11042-021-11362-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11362-5

Keywords

Search

Navigation