Abstract
Multi-focus image fusion can obtain high-quality images by overcoming the limited depth of field of optical lenses. Benefiting from deep learning, we design a local-global joint attention module and propose a novel multi-focus image fusion network. The module essentially is an attention module. Local and global features are extracted respectively through point-wise convolution and spatial pyramid pooling. A joint attention map is produced by reducing the dimension and fusing these two features. The proposed network is mainly composed of a feature fusion module and two weight-shared dense feature extraction modules, each connected to six consecutive attention modules. Such design has two benefits: adequate extraction of initial features and capturing of local and global features. Subjective visual evaluation demonstrates that the proposed network can preserve the authenticity of fusion results. And it also reduces the appearance of artifacts and detail losses between the focus and defocus regions. Objective metric evaluation shows that the proposed network outperforms most of the existing models, such as SwinFusion, GACN, and UFA-FUSE, in Lytro, MFI-WHU, and MFFW datasets. Ablation experiments demonstrate that the design of attention and the overall framework of the network is reasonable. Overall, the proposed model can finish the multi-focus image fusion task with high quality.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data Availability
The authors declare that the data supporting the findings of this study are available within its supplementary information files.
References
Zhang X (2021) Deep learning-based multi-focus image fusion: A survey and a comparative study. IEEE Trans Pattern Anal Mach Intell 44(9):4819–4838
Zhou W, He J, Li Y et al (2022) Multi-focus image fusion with enhancement filtering for robust vascular quantification using photoacoustic microscopy. Opt Lett 47(15):3732–3735
Manescu P, Shaw M, Neary-Zajiczek L et al (2022) Content aware multi-focus image fusion for high-magnification blood film microscopy. Biomed Opt Express 13(2):1005–1016
Zhou Y, Yu L, Zhi C et al (2022) A survey of multi-focus image fusion methods. Appl Sci 12(12):6281
Tan J, Zhang T, Zhao L et al (2021) Multi-focus image fusion with geometrical sparse representation. Signal Process Image Commun 92:116130
Li S, Kang X, Hu J et al (2013) Image matting for fusion of multi-focus images in dynamic scenes. Inf Fusion 14(2):147–162
Li S, Kang X, Hu J (2013) Image fusion with guided filtering. IEEE Trans Image Process 22(7):2864–2875
Zhou Z, Li S, Wang B (2014) Multi-scale weighted gradient-based fusion for multi-focus images. Inf Fusion 20:60–72
Liu Y, Chen X, Ward RK et al (2016) Image fusion with convolutional sparse representation. IEEE Signal Process Lett 23(12):1882–1886
Guan Z, Wang X, Nie R et al (2022) Ncdcn: multi-focus image fusion via nest connection and dilated convolution network. Appl Intell 1–16
Choudhary G, Sethi D (2022) From conventional approach to machine learning and deep learning approach: An experimental and comprehensive review of image fusion techniques. Arch Comput Methods Eng 1–38
Liu Y, Liu S, Wang Z (2015) Multi-focus image fusion with dense sift. Inf Fusion 23:139–155
Zhang Y, Zhao P, Ma Y et al (2021) Multi-focus image fusion with joint guided image filtering. Signal Process Image Commun 92:116128
Qiu X, Li M, Zhang L et al (2019) Guided filter-based multi-focus image fusion through focus region detection. Signal Process Image Commun 72:35–46
Hu Z, Liang W, Ding D et al (2021) An improved multi-focus image fusion algorithm based on multi-scale weighted focus measure. Appl Intell 51:4453–4469
Lu SY, Wang SH, Zhang YD (2023) Bcdnet: An optimized deep network for ultrasound breast cancer detection. IRBM 44(4):100774
Chakravarthy SS, Bharanidharan N, Rajaguru H (2023) Deep learning-based metaheuristic weighted k-nearest neighbor algorithm for the severity classification of breast cancer. IRBM 44(3):100749
Liu Y, Chen X, Peng H et al (2017) Multi-focus image fusion with a deep convolutional neural network. Inf Fusion 36:191–207
Ma B, Zhu Y, Yin X et al (2021) Sesf-fuse: An unsupervised deep model for multi-focus image fusion. Neural Comput Appl 33:5793–5804
Amin-Naji M, Aghagolzadeh A, Ezoji M (2019) Ensemble of cnn for multi-focus image fusion. Inf Fusion 51:201–214
Li H, Qian W, Nie R et al (2023) Siamese conditional generative adversarial network for multi-focus image fusion. Appl Intell 1–16
Zhang Y, Liu Y, Sun P et al (2020) Ifcnn: A general image fusion framework based on convolutional neural network. Inf Fusion 54:99–118
Xu H, Ma J, Jiang J et al (2020) U2fusion: A unified unsupervised image fusion network. IEEE Trans Pattern Anal Mach Intell 44(1):502–518
Zhang H, Le Z, Shao Z et al (2021) Mff-gan: An unsupervised generative adversarial network with adaptive and gradient joint constraints for multi-focus image fusion. Inf Fusion 66:40–53
Ma L, Hu Y, Zhang B et al (2023) A new multi-focus image fusion method based on multi-classification focus learning and multi-scale decomposition. Appl Intell 53(2):1452–1468
Wang Y, Xu S, Liu J et al (2021) Mfif-gan: A new generative adversarial network for multi-focus image fusion. Signal Process Image Commun 96:116295
Liu Y, Wang L, Li H et al (2022) Multi-focus image fusion with deep residual learning and focus property detection. Inf Fusion 86–87:1–16
Zhao F, Zhao W, Lu H et al (2023) Depth-distilled multi-focus image fusion. IEEE Trans Multimed 25:966–978
Hu X, Jiang J, Liu X et al (2023) Zmff: Zero-shot multi-focus image fusion. Inf Fusion 92:127–138
Zang Y, Zhou D, Wang C et al (2021) Ufa-fuse: A novel deep supervised and hybrid model for multifocus image fusion. IEEE Trans Instrum Meas 70:1–17
Xu H, Ma J, Le Z et al (2020) Fusiondn: A unified densely connected network for image fusion. In: Proceedings of the AAAI conference on artificial intelligence, pp 12484–12491
Ma B, Yin X, Wu D et al (2022) End-to-end learning for simultaneously generating decision map and multi-focus image fusion result. Neurocomputing 470:204–216
Ma J, Tang L, Fan F et al (2022) Swinfusion: Cross-domain long-range learning for general image fusion via swin transformer. IEEE/CAA J Autom Sin 9(7):1200–1217
Cheng C, Xu T, Wu XJ (2023) Mufusion: A general unsupervised image fusion network based on memory unit. Inf Fusion 92:80–92
Li H, Zhang B, Zhang Y et al (2021) A defense method based on attention mechanism against traffic sign adversarial samples. Inf Fusion 76:55–65
Qiu Y, Liu Y, Chen Y et al (2023) A2s ppnet: Attentive atrous spatial pyramid pooling network for salient object detection. IEEE Trans Multimed 25:1991–2006
Roy AG, Navab N, Wachinger C (2018) Concurrent spatial and channel ‘squeeze & excitation’in fully convolutional networks. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2018: 21st International Conference, Granada, Spain, September 16-20, 2018, Proceedings, Part I, Springer, pp 421–429
Yu N, Li J, Hua Z (2022) Attention based dual path fusion networks for multi-focus image. Multimed Tools Appl 81(8):10883–10906
Niu Z, Zhong G, Yu H (2021) A review on the attention mechanism of deep learning. Neurocomputing 452:48–62
Zhou D, Jin X, Jiang Q et al (2022) Mcrd-net: An unsupervised dense network with multi-scale convolutional block attention for multi-focus image fusion. IET Image Process 16(6):1558–1574
Hou Q, Zhou D, Feng J (2021) Coordinate attention for efficient mobile network design. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp 13708–13717
Jiang L, Fan H, Li J (2022) Multi-level receptive field feature reuse for multi-focus image fusion. Mach Vis Appl 33(6):92
Li Z, Li Y, Liu Y et al (2021) Deep learning based densely connected network for load forecasting. IEEE Trans Power Syst 36(4):2829–2840
Ji J, Li S, Liao X et al (2023) Semantic segmentation based on spatial pyramid pooling and multilayer feature fusion. IEEE Trans Cogn Dev Syst 15(3):1524–1535
Liu C, Ding W, Chen P et al (2022) Rb-net: Training highly accurate and efficient binary neural networks with reshaped point-wise convolution and balanced activation. IEEE Trans Circ Syst Video Technol 32(9):6414–6424
Dai Y, Gieseke F, Oehmcke S et al (2021) Attentional feature fusion. In: Proceedings of the IEEE/CVF winter conference on applications of computer Vision, pp 3560–3569
Nejati M, Samavi S, Shirani S (2015) Multi-focus image fusion using dictionary-based sparse representation. Inf Fusion 25:72–84
Everingham M, Eslami SA, Van Gool L et al (2015) The pascal visual object classes challenge: A retrospective. Int J Comput Vision 111:98–136
Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv:1412.6980
Xydeas CS, Petrovic V et al (2000) Objective image fusion performance measure. Electron Lett 36(4):308–309
Chen Y, Blum RS (2009) A new automated quality assessment algorithm for image fusion. Image Vision Comput 27(10):1421–1432
Wang Q, Shen Y, Jin J (2008) Performance evaluation of image fusion techniques. Image Fusion Algorithms Appl 19:469–492
Xydeas CS, Petrovic VS (2000) Objective pixel-level image fusion performance measure. In: Sensor fusion: Architectures, algorithms, and applications IV, SPIE, pp 89–98
Cvejic N, Bull D, Canagarajah C (2007) Metric for multimodal image sensor fusion. Electron Lett 43(2):95–96
Li H, Wu XJ (2019) Densefuse: A fusion approach to infrared and visible images. IEEE Trans Image Process 28(5):2614–2623
Adu J, Gan J, Wang Y et al (2013) Image fusion based on nonsubsampled contourlet transform for infrared and visible light image. Infrared Phys Technol 61:94–100
Funding
This work is supported by the National Natural Science Foundation of China (Grant No. 62003065), the Science and Technology Research Project of Chongqing Municipal Education Commission (Grant No. KJQN202200564, KJZD202200504), the Fund project of Chongqing Normal University (Grant No. 21XLB032) and the Chongqing Education Science 14th Five Year Plan Project (Grant No. 2022-576).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interests
The authors have no relevant financial or non-financial interests to disclose.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Zou, X., Yang, Y., Zhai, H. et al. A multi-focus image fusion network with local-global joint attention module. Appl Intell 55, 113 (2025). https://doi.org/10.1007/s10489-024-06039-z
Accepted:
Published:
DOI: https://doi.org/10.1007/s10489-024-06039-z