Skip to main content

Advertisement

A multi-focus image fusion network with local-global joint attention module

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Multi-focus image fusion can obtain high-quality images by overcoming the limited depth of field of optical lenses. Benefiting from deep learning, we design a local-global joint attention module and propose a novel multi-focus image fusion network. The module essentially is an attention module. Local and global features are extracted respectively through point-wise convolution and spatial pyramid pooling. A joint attention map is produced by reducing the dimension and fusing these two features. The proposed network is mainly composed of a feature fusion module and two weight-shared dense feature extraction modules, each connected to six consecutive attention modules. Such design has two benefits: adequate extraction of initial features and capturing of local and global features. Subjective visual evaluation demonstrates that the proposed network can preserve the authenticity of fusion results. And it also reduces the appearance of artifacts and detail losses between the focus and defocus regions. Objective metric evaluation shows that the proposed network outperforms most of the existing models, such as SwinFusion, GACN, and UFA-FUSE, in Lytro, MFI-WHU, and MFFW datasets. Ablation experiments demonstrate that the design of attention and the overall framework of the network is reasonable. Overall, the proposed model can finish the multi-focus image fusion task with high quality.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

The authors declare that the data supporting the findings of this study are available within its supplementary information files.

References

  1. Zhang X (2021) Deep learning-based multi-focus image fusion: A survey and a comparative study. IEEE Trans Pattern Anal Mach Intell 44(9):4819–4838

    MATH  Google Scholar 

  2. Zhou W, He J, Li Y et al (2022) Multi-focus image fusion with enhancement filtering for robust vascular quantification using photoacoustic microscopy. Opt Lett 47(15):3732–3735

    MATH  Google Scholar 

  3. Manescu P, Shaw M, Neary-Zajiczek L et al (2022) Content aware multi-focus image fusion for high-magnification blood film microscopy. Biomed Opt Express 13(2):1005–1016

    Google Scholar 

  4. Zhou Y, Yu L, Zhi C et al (2022) A survey of multi-focus image fusion methods. Appl Sci 12(12):6281

    MATH  Google Scholar 

  5. Tan J, Zhang T, Zhao L et al (2021) Multi-focus image fusion with geometrical sparse representation. Signal Process Image Commun 92:116130

    MATH  Google Scholar 

  6. Li S, Kang X, Hu J et al (2013) Image matting for fusion of multi-focus images in dynamic scenes. Inf Fusion 14(2):147–162

    MATH  Google Scholar 

  7. Li S, Kang X, Hu J (2013) Image fusion with guided filtering. IEEE Trans Image Process 22(7):2864–2875

    MATH  Google Scholar 

  8. Zhou Z, Li S, Wang B (2014) Multi-scale weighted gradient-based fusion for multi-focus images. Inf Fusion 20:60–72

    MATH  Google Scholar 

  9. Liu Y, Chen X, Ward RK et al (2016) Image fusion with convolutional sparse representation. IEEE Signal Process Lett 23(12):1882–1886

    MATH  Google Scholar 

  10. Guan Z, Wang X, Nie R et al (2022) Ncdcn: multi-focus image fusion via nest connection and dilated convolution network. Appl Intell 1–16

  11. Choudhary G, Sethi D (2022) From conventional approach to machine learning and deep learning approach: An experimental and comprehensive review of image fusion techniques. Arch Comput Methods Eng 1–38

  12. Liu Y, Liu S, Wang Z (2015) Multi-focus image fusion with dense sift. Inf Fusion 23:139–155

    MATH  Google Scholar 

  13. Zhang Y, Zhao P, Ma Y et al (2021) Multi-focus image fusion with joint guided image filtering. Signal Process Image Commun 92:116128

    MATH  Google Scholar 

  14. Qiu X, Li M, Zhang L et al (2019) Guided filter-based multi-focus image fusion through focus region detection. Signal Process Image Commun 72:35–46

    MATH  Google Scholar 

  15. Hu Z, Liang W, Ding D et al (2021) An improved multi-focus image fusion algorithm based on multi-scale weighted focus measure. Appl Intell 51:4453–4469

    MATH  Google Scholar 

  16. Lu SY, Wang SH, Zhang YD (2023) Bcdnet: An optimized deep network for ultrasound breast cancer detection. IRBM 44(4):100774

    MATH  Google Scholar 

  17. Chakravarthy SS, Bharanidharan N, Rajaguru H (2023) Deep learning-based metaheuristic weighted k-nearest neighbor algorithm for the severity classification of breast cancer. IRBM 44(3):100749

    MATH  Google Scholar 

  18. Liu Y, Chen X, Peng H et al (2017) Multi-focus image fusion with a deep convolutional neural network. Inf Fusion 36:191–207

    MATH  Google Scholar 

  19. Ma B, Zhu Y, Yin X et al (2021) Sesf-fuse: An unsupervised deep model for multi-focus image fusion. Neural Comput Appl 33:5793–5804

    MATH  Google Scholar 

  20. Amin-Naji M, Aghagolzadeh A, Ezoji M (2019) Ensemble of cnn for multi-focus image fusion. Inf Fusion 51:201–214

    MATH  Google Scholar 

  21. Li H, Qian W, Nie R et al (2023) Siamese conditional generative adversarial network for multi-focus image fusion. Appl Intell 1–16

  22. Zhang Y, Liu Y, Sun P et al (2020) Ifcnn: A general image fusion framework based on convolutional neural network. Inf Fusion 54:99–118

    MATH  Google Scholar 

  23. Xu H, Ma J, Jiang J et al (2020) U2fusion: A unified unsupervised image fusion network. IEEE Trans Pattern Anal Mach Intell 44(1):502–518

    MATH  Google Scholar 

  24. Zhang H, Le Z, Shao Z et al (2021) Mff-gan: An unsupervised generative adversarial network with adaptive and gradient joint constraints for multi-focus image fusion. Inf Fusion 66:40–53

    MATH  Google Scholar 

  25. Ma L, Hu Y, Zhang B et al (2023) A new multi-focus image fusion method based on multi-classification focus learning and multi-scale decomposition. Appl Intell 53(2):1452–1468

    MATH  Google Scholar 

  26. Wang Y, Xu S, Liu J et al (2021) Mfif-gan: A new generative adversarial network for multi-focus image fusion. Signal Process Image Commun 96:116295

    Google Scholar 

  27. Liu Y, Wang L, Li H et al (2022) Multi-focus image fusion with deep residual learning and focus property detection. Inf Fusion 86–87:1–16

    MATH  Google Scholar 

  28. Zhao F, Zhao W, Lu H et al (2023) Depth-distilled multi-focus image fusion. IEEE Trans Multimed 25:966–978

    MATH  Google Scholar 

  29. Hu X, Jiang J, Liu X et al (2023) Zmff: Zero-shot multi-focus image fusion. Inf Fusion 92:127–138

    MATH  Google Scholar 

  30. Zang Y, Zhou D, Wang C et al (2021) Ufa-fuse: A novel deep supervised and hybrid model for multifocus image fusion. IEEE Trans Instrum Meas 70:1–17

    MATH  Google Scholar 

  31. Xu H, Ma J, Le Z et al (2020) Fusiondn: A unified densely connected network for image fusion. In: Proceedings of the AAAI conference on artificial intelligence, pp 12484–12491

  32. Ma B, Yin X, Wu D et al (2022) End-to-end learning for simultaneously generating decision map and multi-focus image fusion result. Neurocomputing 470:204–216

    MATH  Google Scholar 

  33. Ma J, Tang L, Fan F et al (2022) Swinfusion: Cross-domain long-range learning for general image fusion via swin transformer. IEEE/CAA J Autom Sin 9(7):1200–1217

    MATH  Google Scholar 

  34. Cheng C, Xu T, Wu XJ (2023) Mufusion: A general unsupervised image fusion network based on memory unit. Inf Fusion 92:80–92

    MATH  Google Scholar 

  35. Li H, Zhang B, Zhang Y et al (2021) A defense method based on attention mechanism against traffic sign adversarial samples. Inf Fusion 76:55–65

    MATH  Google Scholar 

  36. Qiu Y, Liu Y, Chen Y et al (2023) A2s ppnet: Attentive atrous spatial pyramid pooling network for salient object detection. IEEE Trans Multimed 25:1991–2006

    MATH  Google Scholar 

  37. Roy AG, Navab N, Wachinger C (2018) Concurrent spatial and channel ‘squeeze & excitation’in fully convolutional networks. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2018: 21st International Conference, Granada, Spain, September 16-20, 2018, Proceedings, Part I, Springer, pp 421–429

  38. Yu N, Li J, Hua Z (2022) Attention based dual path fusion networks for multi-focus image. Multimed Tools Appl 81(8):10883–10906

    MATH  Google Scholar 

  39. Niu Z, Zhong G, Yu H (2021) A review on the attention mechanism of deep learning. Neurocomputing 452:48–62

    MATH  Google Scholar 

  40. Zhou D, Jin X, Jiang Q et al (2022) Mcrd-net: An unsupervised dense network with multi-scale convolutional block attention for multi-focus image fusion. IET Image Process 16(6):1558–1574

    MATH  Google Scholar 

  41. Hou Q, Zhou D, Feng J (2021) Coordinate attention for efficient mobile network design. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp 13708–13717

  42. Jiang L, Fan H, Li J (2022) Multi-level receptive field feature reuse for multi-focus image fusion. Mach Vis Appl 33(6):92

    MATH  Google Scholar 

  43. Li Z, Li Y, Liu Y et al (2021) Deep learning based densely connected network for load forecasting. IEEE Trans Power Syst 36(4):2829–2840

    MATH  Google Scholar 

  44. Ji J, Li S, Liao X et al (2023) Semantic segmentation based on spatial pyramid pooling and multilayer feature fusion. IEEE Trans Cogn Dev Syst 15(3):1524–1535

    MATH  Google Scholar 

  45. Liu C, Ding W, Chen P et al (2022) Rb-net: Training highly accurate and efficient binary neural networks with reshaped point-wise convolution and balanced activation. IEEE Trans Circ Syst Video Technol 32(9):6414–6424

    MATH  Google Scholar 

  46. Dai Y, Gieseke F, Oehmcke S et al (2021) Attentional feature fusion. In: Proceedings of the IEEE/CVF winter conference on applications of computer Vision, pp 3560–3569

  47. Nejati M, Samavi S, Shirani S (2015) Multi-focus image fusion using dictionary-based sparse representation. Inf Fusion 25:72–84

    MATH  Google Scholar 

  48. Everingham M, Eslami SA, Van Gool L et al (2015) The pascal visual object classes challenge: A retrospective. Int J Comput Vision 111:98–136

    MATH  Google Scholar 

  49. Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv:1412.6980

  50. Xydeas CS, Petrovic V et al (2000) Objective image fusion performance measure. Electron Lett 36(4):308–309

  51. Chen Y, Blum RS (2009) A new automated quality assessment algorithm for image fusion. Image Vision Comput 27(10):1421–1432

    MATH  Google Scholar 

  52. Wang Q, Shen Y, Jin J (2008) Performance evaluation of image fusion techniques. Image Fusion Algorithms Appl 19:469–492

    MATH  Google Scholar 

  53. Xydeas CS, Petrovic VS (2000) Objective pixel-level image fusion performance measure. In: Sensor fusion: Architectures, algorithms, and applications IV, SPIE, pp 89–98

  54. Cvejic N, Bull D, Canagarajah C (2007) Metric for multimodal image sensor fusion. Electron Lett 43(2):95–96

    MATH  Google Scholar 

  55. Li H, Wu XJ (2019) Densefuse: A fusion approach to infrared and visible images. IEEE Trans Image Process 28(5):2614–2623

    MathSciNet  MATH  Google Scholar 

  56. Adu J, Gan J, Wang Y et al (2013) Image fusion based on nonsubsampled contourlet transform for infrared and visible light image. Infrared Phys Technol 61:94–100

    MATH  Google Scholar 

Download references

Funding

This work is supported by the National Natural Science Foundation of China (Grant No. 62003065), the Science and Technology Research Project of Chongqing Municipal Education Commission (Grant No. KJQN202200564, KJZD202200504), the Fund project of Chongqing Normal University (Grant No. 21XLB032) and the Chongqing Education Science 14th Five Year Plan Project (Grant No. 2022-576).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Hao Zhai.

Ethics declarations

Competing interests

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zou, X., Yang, Y., Zhai, H. et al. A multi-focus image fusion network with local-global joint attention module. Appl Intell 55, 113 (2025). https://doi.org/10.1007/s10489-024-06039-z

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10489-024-06039-z

Keywords