Skip to main content
SpringerLink
Log in

Multi-focus image fusion via adaptive fractional differential and guided filtering

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The goal of multi-focus image fusion is to make an image that is all-in-focus with a finite depth of field. Although multi-focus image fusion algorithms have advanced significantly, it is still difficult to avoid defocused pixels in fused results. To alleviate the problem, a multi-focus image fusion method via adaptive fractional differential and guided filtering is proposed in this paper. Specifically, by combining the guided filtering and the fractional differential, the base and detail layers can be obtained using the designed effective two-scale image decomposition approach. In addition, an image decomposition scheme based on the statistical characteristics concerning the source images is developed, which can retain more detailed information by adaptively adjusting parameters, thereby reducing defocused pixels. The fused all-in-focus images are obtained by merging features from different scales. Experiments demonstrate that compared with existing image fusion algorithms, the promising results on three public datasets are achieved by the proposed method in qualitative evaluation and quantitative measurement, especially in terms of AG, VIF, and SF. On the Lytro dataset, our method has 22%, 34%, and 14% improvement in AG, VIF, and SF compared to the sub-optimal method. On the MFFW dataset, compared with the sub-optimal method, the proposed method has 12%, 14%, and 19% improvement in AG, VIF, and SF, respectively. On the MFI-WHU dataset, the proposed method realizes a performance improvement of 22%, 4%, and 30% in AG, VIF, and SF, compared with the sub-optimal method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Data Availability

The Lytro dataset can be downloaded from https://mansournejati.ece.iut.ac.ir/content/lytro-multi-focus-dataset.The MFFW dataset can be found in [57]. The MFI-WHU dataset is provided in https://github.com/HaoZhang1018/MFI-WHU

References

  1. Song Y, Li M, Li Q, Sun L (2006) A new wavelet based multi-focus image fusion scheme and its application on optical microscopy. In: 2006 IEEE International Conference on Robotics and Biomimetics, pp. 401–405

  2. Chen Z, Wang D, Gong S, Zhao F (2017) Application of multi-focus image fusion in visual power patrol inspection. In: 2017 IEEE 2nd Advanced Information Technology, Electronic and Automation Control Conference (IAEAC), pp. 1688–1692. IEEE

  3. Li C, Liang X, Lu Y, Zhao N, Tang J (2019) Rgb-t object tracking: Benchmark and baseline. Pattern Recog 96:106977

    Google Scholar 

  4. Zhang H, Le Z, Shao Z, Xu H, Ma J (2021) Mff-gan: An unsupervised generative adversarial network with adaptive and gradient joint constraints for multi-focus image fusion. Inf Fusion 66:40–53

    Google Scholar 

  5. Farid MS, Mahmood A, Al-Maadeed SA (2019) Multi-focus image fusion using content adaptive blurring. Inf fusion 45:96–112

    Google Scholar 

  6. Li H, Zhang L, Jiang M, Li Y (2021) Multi-focus image fusion algorithm based on supervised learning for fully convolutional neural network. Pattern Recogn Lett 141:45–53

    ADS  Google Scholar 

  7. Singh H, Sanchez C, Cristobal G (2020) Construction of fused image with improved depth-of-field based on guided co-occurrence filtering. Digit Sign Process 104:102793

    Google Scholar 

  8. Tang D, Xiong Q, Yin H, Zhu Z, Li Y (2022) A novel sparse representation based fusion approach for multi-focus images. Expert Syst Appl 197:116737

    Google Scholar 

  9. Aymaz S, Köse C, Aymaz Ş (2022) A novel approach with the dynamic decision mechanism (ddm) in multi-focus image fusion. Multimedia Tools Appl 82:1–51

    Google Scholar 

  10. Zhang X, He H, Zhang J-X (2022) Multi-focus image fusion based on fractional order differentiation and closed image matting. ISA Trans

  11. Liu Y, Chen X, Wang Z, Wang ZJ, Ward RK, Wang X (2018) Deep learning for pixel-level image fusion: Recent advances and future prospects. Inf Fusion 42:158–173

    Google Scholar 

  12. Liu Y, Liu S, Wang Z (2015) Multi-focus image fusion with dense sift. Inf Fusion 23:139–155

    Google Scholar 

  13. De I, Chanda B (2013) Multi-focus image fusion using a morphology-based focus measure in a quad-tree structure. Inf Fusion 14(2):136–146

    Google Scholar 

  14. Huang W, Jing Z (2007) Multi-focus image fusion using pulse coupled neural network. Pattern Recogn Lett 28(9):1123–1132

    ADS  Google Scholar 

  15. Hu Z, Liang W, Ding D, Wei G (2021) An improved multi-focus image fusion algorithm based on multi-scale weighted focus measure. Appl Intell 51:4453–4469

    Google Scholar 

  16. Meher B, Agrawal S, Panda R, Abraham A (2019) A survey on region based image fusion methods. Inf Fusion 48:119–132

    Google Scholar 

  17. Liu W, Zheng Z, Wang Z (2021) Robust multi-focus image fusion using lazy random walks with multiscale focus measures. Signal Process 179:107850

    Google Scholar 

  18. Sharma S, Kumar K, Singh N (2022) Deep eigen space based ASL recognition system. IETE J Res 68(5):3798–3808

    Google Scholar 

  19. Sharma S, Shivhare SN, Singh N, Kumar K (2019) Computationally efficient ann model for small-scale problems. In: Machine intelligence and signal analysis, pp. 423–435

  20. Kumar K (2021) Text query based summarized event searching interface system using deep learning over cloud. Multimedia Tools Appl 80(7):11079–11094

    Google Scholar 

  21. Zhang X (2021) Deep learning-based multi-focus image fusion: A survey and a comparative study. IEEE Transactions on Pattern Analysis and Machine Intelligence, 4819–4838

  22. Lauriola I, Gallicchio C, Aiolli F (2020) Enhancing deep neural networks via multiple kernel learning. Pattern Recog 101:107194

    Google Scholar 

  23. Koppanati RK, Kumar K (2020) P-MEC: polynomial congruence-based multimedia encryption technique over cloud. IEEE Consum Electron Mag 10(5):41–46

    Google Scholar 

  24. Solanki A, Bamrara R, Kumar K, Singh N (2020) Vedl: a novel video event searching technique using deep learning. Soft Computing: Theories and Applications: Proceedings of SoCTA 2018:905–914

    Google Scholar 

  25. Mishra A, Kumar K, Kumar P, Mittal P (2020) A novel approach for handwritten character recognition using K-NN classifier. In: Soft Computing: Theories and Applications: Proceedings of SoCTA 2018, pp. 887–894

  26. Zhang X, Ye P, Xiao G (2020) Vifb: a visible and infrared image fusion benchmark. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 104–105

  27. Pu Y-F, Siarry P, Chatterjee A, Wang Z-N, Yi Z, Liu Y-G, Zhou J-L, Wang Y (2017) A fractional-order variational framework for retinex: fractional-order partial differential equation-based formulation for multi-scale nonlocal contrast enhancement with texture preserving. IEEE Trans Image Process 27(3):1214–1229

    ADS  MathSciNet  Google Scholar 

  28. He K, Sun J, Tang X (2012) Guided image filtering. IEEE transactions on pattern analysis and machine intelligence 35(6):1397–1409

    Google Scholar 

  29. Jiang L, Zhang D, Che L (2021) Texture analysis-based multi-focus image fusion using a modified pulse-coupled neural network (pcnn). Signal Processing: Image Commun 91:116068

    Google Scholar 

  30. He K, Zhou D, Zhang X, Nie R (2018) Multi-focus: Focused region finding and multi-scale transform for image fusion. Neurocomputing 320:157–170

    Google Scholar 

  31. Li S, Kang X, Hu J (2013) Image fusion with guided filtering. IEEE Trans Image Process 22(7):2864–2875

    ADS  PubMed  Google Scholar 

  32. Zhang Y, Zhao P, Ma Y, Fan X (2021) Multi-focus image fusion with joint guided image filtering. Signal Processing: Image Communication 92:116128

    Google Scholar 

  33. Ma J, Zhou Z, Wang B, Miao L, Zong H (2019) Multi-focus image fusion using boosted random walks-based algorithm with two-scale focus maps. Neurocomputing 335:9–20

    Google Scholar 

  34. Li X, Zhou F, Tan H, Chen Y, Zuo W (2021) Multi-focus image fusion based on nonsubsampled contourlet transform and residual removal. Signal Process 184:108062

    Google Scholar 

  35. Zhang Q, Liu Y, Blum RS, Han J, Tao D (2018) Sparse representation based multi-sensor image fusion for multi-focus and multi-modality images: A review. Inf Fusion 40:57–75

    Google Scholar 

  36. Ma X, Wang Z, Hu S (2021) Multi-focus image fusion based on multi-scale sparse representation. Journal of Visual Communication and Image Representation 81:103328

    Google Scholar 

  37. Liu Y, Wang Z (2015) Simultaneous image fusion and denoising with adaptive sparse representation. IET Image Process 9(5):347–357

    Google Scholar 

  38. Wang J, Peng J, Feng X, He G, Fan J (2014) Fusion method for infrared and visible images by using non-negative sparse representation. Infrared Phys Technol 67:477–489

    ADS  Google Scholar 

  39. Zhang Q, Shi T, Wang F, Blum RS, Han J (2018) Robust sparse representation based multi-focus image fusion with dictionary construction and local spatial consistency. Pattern Recogn 83:299–313

    ADS  Google Scholar 

  40. Fu G-P, Hong S-H, Li F-L, Wang L (2020) A novel multi-focus image fusion method based on distributed compressed sensing. J Vis Commun Image Represent 67:102760

    Google Scholar 

  41. Liu Y, Chen X, Ward RK, Wang ZJ (2016) Image fusion with convolutional sparse representation. IEEE Signal Process Lett 23(12):1882–1886

    ADS  Google Scholar 

  42. Tan J, Zhang T, Zhao L, Luo X, Tang YY (2021) Multi-focus image fusion with geometrical sparse representation. Signal Processing: Image Commun 92:116–130

    Google Scholar 

  43. Negi A, Kumar K, Chauhan P (2021) Deep neural network-based multi-class image classification for plant diseases. Agricultural informatics: automation using the IoT and machine learning, 117–129

  44. Sharma S, Kumar K (2021) ASL-3DCNN: American sign language recognition technique using 3-D convolutional neural networks. Multimedia Tools and Applications 80(17):26319–26331

    Google Scholar 

  45. Negi A, Kumar K, Chaudhari NS, Singh N, Chauhan P (2021) Predictive analytics for recognizing human activities using residual network and fine-tuning. In: Big Data Analytics: 9th International Conference, BDA 2021, Virtual Event, December 15-18, 2021, Proceedings 9, pp. 296–310

  46. Kumar K, Shrimankar DD (2017) F-DES: Fast and deep event summarization. IEEE Trans Multimed 20(2):323–334

    Google Scholar 

  47. Liu Y, Chen X, Peng H, Wang Z (2017) Multi-focus image fusion with a deep convolutional neural network. Inf Fusion 36:191–207

    Google Scholar 

  48. Yang Y, Nie Z, Huang S, Lin P, Wu J (2019) Multilevel features convolutional neural network for multifocus image fusion. IEEE Trans Comput Imaging 5(2):262–273

    Google Scholar 

  49. Zhang Y, Liu Y, Sun P, Yan H, Zhao X, Zhang L (2020) Ifcnn: A general image fusion framework based on convolutional neural network. Information Fusion 54:99–118

    Google Scholar 

  50. Mao Q, Yang X, Zhang R, Jeon G, Hussain F, Liu K (2022) Multi-focus images fusion via residual generative adversarial network. Multimed Tools Appl 81(9):12305–12323

    Google Scholar 

  51. Sayevand K, Golbabai A, Yildirim A (2012) Analysis of differential equations of fractional order. Appl Math Modell 36(9):4356–4364

    MathSciNet  Google Scholar 

  52. Azarang A, Ghassemian H (2018) Application of fractional-order differentiation in multispectral image fusion. Remote Sens Lett 9(1):91–100

    Google Scholar 

  53. He N, Wang J-B, Zhang L-L, Lu K (2015) An improved fractional-order differentiation model for image denoising. Signal Processing 112:180–188

    Google Scholar 

  54. Eskicioglu AM, Fisher PS (1995) Image quality measures and their performance. IEEE Transactions on communications 43(12):2959–2965

    Google Scholar 

  55. Cui G, Feng H, Xu Z, Li Q, Chen Y (2015) Detail preserved fusion of visible and infrared images using regional saliency extraction and multi-scale image decomposition. Optics Commun 341:199–209

    CAS  Google Scholar 

  56. Li ZKST-J, Liang Z-HT (2010) 2d fractional kalman filter and its application to image process. J Electron Inf Technol 12:3027–3031

    Google Scholar 

  57. Xu S, Wei X, Zhang C, Liu J, Zhang J (2020) MFFW: A new dataset for multi-focus image fusion. arXiv preprint arXiv:2002.04780

  58. Rao Y-J (1997) In-fibre bragg grating sensors. Meas Sci Technol 8(4):355

    ADS  CAS  Google Scholar 

  59. Han Y, Cai Y, Cao Y, Xu X (2013) A new image fusion performance metric based on visual information fidelity. Inf Fusion 14(2):127–135

    Google Scholar 

  60. Qu G, Zhang D, Yan P (2002) Information measure for performance of image fusion. Electron Lett 38(7):313–315

    ADS  Google Scholar 

  61. Qiu X, Li M, Zhang L, Yuan X (2019) Guided filter-based multi-focus image fusion through focus region detection. Signal Process: Image Commun 72:35–46

    Google Scholar 

  62. Bavirisetti DP, Xiao G, Zhao J, Dhuli R, Liu G (2019) Multi-scale guided image and video fusion: A fast and efficient approach. Circuits, Systems, and Signal Processing 38(12):5576–5605

    Google Scholar 

Download references

Acknowledgements

The authors would like to thank the anonymous reviewers and editors for their invaluable suggestions. The work was supported in part by the Fundamental Research Funds for the Central Universities (No. 2022YJS013), the National Natural Science Foundation of China (No. 62172029), and the Fundamental Research Funds for the Central Universities (No. 2020JBM008).

Author information

Authors and Affiliations

  1. School of Electronic and Information Engineering, Beijing Jiaotong University, 100044, Beijing, China

    Xiaoling Li, Houjin Chen, Yanfeng Li & Yahui Peng

Authors
  1. Xiaoling Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Houjin Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yanfeng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yahui Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Houjin Chen.

Ethics declarations

Competing of Interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, X., Chen, H., Li, Y. et al. Multi-focus image fusion via adaptive fractional differential and guided filtering. Multimed Tools Appl 83, 32923–32943 (2024). https://doi.org/10.1007/s11042-023-16785-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-16785-w

Keywords

Search

Navigation