Skip to main content
Springer Nature Link
Log in

A novel approach for multi-focus image fusion based on SF-PAPCNN and ISML in NSST domain

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In order to further improve the contrast and sharpness of fused image, a novel multi-focus image fusion algorithm based on spatial frequency-motivated parameter-adaptive pulse coupled neural network (SF-PAPCNN) and improved sum-modified-laplacian (ISML) in nonsubsampled shearlet transform (NSST) domain is proposed in this paper. In its procedural steps, at first, the source images are decomposed into low-frequency and high-frequency components by NSST. The low-frequency components are fused by SF-PAPCNN model, the PAPCNN is designed to estimate the PCNN parameters adaptively according to the input information, and the high-frequency components are fused by ISML model. Finally, the inverse NSST is employed to the fused coefficients to reconstruct the fused image. The superiority of the proposed fusion technique is confirmed by many analytical experimentations on the gray and color multi-focus image data sets. Compared with the state-of-the-art image fusion methods, the proposed fusion algorithm has superior performance in terms of visual inspection and objective evaluation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Amin-Naji M, Aghagolzadeh A, Ezoji M (2019) Ensemble of CNN for multi-focus image fusion. Inf Fusion 51:201–214

    Google Scholar 

  2. Chen Y, Park S, Ma Y (2011) A new automatic parameter setting method of a simplified PCNN for image segmentation. IEEE Trans Neural Netw 22(6):880–892

    Google Scholar 

  3. Chen Y, Deng N, Xin B (2019) Nonwovens structure measurement based on NSST multi-focus image fusion. Micron 123, UNSP 102684

  4. Chen Y, Xiong J, Xu W (2019) A novel online incremental and decremental learning algorithm based on variable support vector machine. Clust Comput 22(3):S7435–S7445

    Google Scholar 

  5. Chen Y, Xu W, Zuo J (2019) The fire recognition algorithm using dynamic feature fusion and IV-SVM classifier. Clust Comput 22(3):S7665–S7675

    Google Scholar 

  6. Chen Y, Wang J, Xia R (2019) The visual object tracking algorithm research based on adaptive combination kernel. J Ambient Intell Humaniz Comput 10(12):4855–4867

    Google Scholar 

  7. Chen Y, Wang J, Chen X (2019) Single-image super-resolution algorithm based on structural self-similarity and deformation block features. IEEE Access 7:58791–58801

    Google Scholar 

  8. Chen Y, Wang J, Liu S (2019) Multiscale fast correlation filtering tracking algorithm based on a feature fusion model. Concurr Comput Pract Exp e5533

  9. Cunha A, Zhou J, Do M (2006) The nonsubsampled contourlet transform: theory, design, and applications. IEEE Trans Image Process 15(10):3089–3101

    Google Scholar 

  10. Do M, Vetterli M (2005) The contourlet transform: an efficient directional multiresolution image representation. IEEE Trans Image Process 14(12):2091–2106

    Google Scholar 

  11. Du C, Gao S, Liu Y (2019) Multi-focus image fusion using deep support value convolutional neural network. Optik 176:567–578

    Google Scholar 

  12. Easley G, Labate D, Lim W (2008) Sparse directional image representations using the discrete shearlet transform. Appl Comput Harmon Anal 25(1):25–46

    MathSciNet  MATH  Google Scholar 

  13. Fan D, Lin Z, Zhao J, Liu Y (2020) Rethinking RGB-D salient object detection: models, datasets, and large-scale benchmarks. IEEE Transactions on Neural Networks and Learning Systems. (Accepted)

  14. Fu Z, Wang X, Xu J (2016) Infrared and visible images fusion based on RPCA and NSCT. Infrared Phys Technol 77:114–123

    Google Scholar 

  15. Gu K, Wu N, Yin B (2020) Secure data query framework for cloud and fog computing. IEEE Trans Netw Serv Manag 17(1):332–345

    Google Scholar 

  16. Gu K, Wu N, Yin B, Jia W (2020) Secure Data Sequence Query Framework Based on Multiple Fogs. IEEE Trans Emerg Top Comput 8(5):1–17

  17. Han Y, Cai Y, Cao Y (2013) A new image fusion performance metric based on visual information fidelity. Inf Fusion 14(2):127–135

  18. He K, Sun J, Tang X (2013) Guided image filtering. IEEE Trans Pattern Anal Mach Intell 35(6):1397–1409

    Google Scholar 

  19. He K, Zhou D, Zhang X (2019) Multi-focus image fusion combining focus-region-level partition and pulse-coupled neural network. Soft Comput 23(13):4685–4699

    Google Scholar 

  20. He G, Xing S, He X (2019) Image fusion method based on simultaneous sparse representation with non-subsampled contourlet transform. IET Comput Vis 13(2):240–248

    Google Scholar 

  21. Hermessi H, Mourali O, Zagrouba E (2018) Convolutional neural network-based multimodal image fusion via similarity learning in the shearlet domain. Neural Comput Appl 30(7):2029–2045

    Google Scholar 

  22. Kong W, Lei Y (2018) Multi-focus image fusion through DCNN and ELM. Electron Lett 54(22):1282–1283

    Google Scholar 

  23. Kou F, Chen W, Wen C (2015) Gradient domain guided image filtering. IEEE Trans Image Process 24(11):4528–4539

    MathSciNet  MATH  Google Scholar 

  24. Kumar B (2015) Image fusion based on pixel significance using cross bilateral filter. Signal Image Video Process 9(5):1193–1204

    Google Scholar 

  25. Li L, Si Y (2019) Enhancement of hyperspectral remote sensing images based on improved fuzzy contrast in nonsubsampled shearlet transform domain. Multimed Tools Appl 78(13):18077–18094

    Google Scholar 

  26. Li S, Yang B (2008) Multifocus image fusion by combining curvelet and wavelet transform. Pattern Recogn Lett 29(9):1295–1301

    Google Scholar 

  27. Li S, Kang X, Hu J (2013) Image fusion with guided filtering. IEEE Trans Image Process 22(7):2864–2875

    Google Scholar 

  28. Li Z, Zheng J, Zhu Z (2015) Weighted guided image filtering. IEEE Trans Image Process 24(1):120–129

    MathSciNet  MATH  Google Scholar 

  29. Li W, Zhao Z, Du J (2017) Edge-preserve filter image enhancement with application to medical image fusion. J Med Imaging Health Inf 7(1):16–24

    Google Scholar 

  30. Li Q, Yang X, Wu W (2018) Multi-focus image fusion method for vision sensor systems via dictionary learning with guided filter. Sensors 18(7):2143

    Google Scholar 

  31. Li L, Si Y, Wang L, Jia Z, Ma H (2020) Brain image enhancement approach based on singular value decomposition in nonsubsampled shearlet transform domain. J Med Imaging Health Inf 10(8):1785–1794

    Google Scholar 

  32. Liu X, Xiao D (2019) Multimodality image fusion based on quantum wavelet transform and sum-modified-laplacian rule. Int J Theor Phys 58(3):734–744

    MathSciNet  MATH  Google Scholar 

  33. Liu Y, Liu S, Wang Z (2015) A general framework for image fusion based on multi-scale transform and sparse representation. Inf Fusion 24:147–164

    Google Scholar 

  34. Liu S, Zhao J, Shi M (2015) Medical image fusion based on rolling guidance filter and spiking cortical model. Comput Math Methods Med 156043

  35. Liu Y, Chen X, Ward R (2016) Image fusion with convolutional sparse representation. IEEE Signal Process Lett 23(12):1882–1886

    Google Scholar 

  36. Liu Y, Chen X, Peng H (2017) Multi-focus image fusion with a deep convolutional neural network. Inf Fusion 36:191–207

    Google Scholar 

  37. Liu D, Zhou D, Nie R (2018) Infrared and visible image fusion based on convolutional neural network model and saliency detection via hybrid l(0)-l(1) layer decomposition. J Electron Imaging 27(6):063036

    Google Scholar 

  38. Liu S, Wang J, Lu Y (2019) Multi-focus image fusion based on adaptive dual-channel spiking cortical model in non-subsampled shearlet domain. IEEE Access 7:56367–56388

    Google Scholar 

  39. Liu Z, Lai Z, Ou W (2020) Structured optimal graph based sparse feature extraction for semi-supervised learning. Signal Process 170:107456

  40. Liu Z, Zhang W, Zhao P (2020) A cross-modal adaptive gated fusion generative adversarial network for RGB-D salient object detection. Neurocomputing 387:210–220

    Google Scholar 

  41. Ma J, Ma Y, Li C (2019) Infrared and visible image fusion methods and applications: A survey. Inf Fusion 45:153–178

    Google Scholar 

  42. Meher B, Agrawal S, Panda R (2019) A survey on region based image fusion methods. Inf Fusion 48:119–132

    Google Scholar 

  43. Nie G, Cheng M, Liu Y (2019) Multi-level context ultra-aggregation for stereo matching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR2019) 3278–3286

  44. Piao Y, Rong Z, Zhang M, Lu H (2020) Exploit and replace: an asymmetrical two-stream architecture for versatile light field saliency detection. In Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI2020). (Accepted)

  45. Qu X, Yan J, Xiao H (2008) Image fusion algorithm based on spatial frequency-motivated pulse coupled neural networks in nonsubsampled contourlet transform domain. Acta Automat Sin 34(12):1508–1514

    MATH  Google Scholar 

  46. Tan W, Zhou H, Rong S (2018) Fusion of multi-focus images via a Gaussian curvature filter and synthetic focusing degree criterion. Appl Opt 57(35):10092–10101

    Google Scholar 

  47. Wan W, Yang Y, Lee H (2018) Practical remote sensing image fusion method based on guided filter and improved SML in the NSST domain. Signal Image Video Process 12(5):959–966

    Google Scholar 

  48. Yang Y, Tong S, Huang S (2015) Multifocus image fusion based on NSCT and focused area detection. IEEE Sensors J 15(5):2824–2838

    Google Scholar 

  49. Yang Y, Wu J, Huang S (2019) Multimodal medical image fusion based on fuzzy discrimination with structural patch decomposition. IEEE J Biomed Health Inf 23(4):1647–1660

    Google Scholar 

  50. Ye F, Li X, Zhang X (2019) FusionCNN: a remote sensing image fusion algorithm based on deep convolutional neural networks. Multimed Tools Appl 78(11):14683–14703

    Google Scholar 

  51. Yin M, Liu X, Liu Y (2019) Medical image fusion with parameter-adaptive pulse coupled neural network in nonsubsampled shearlet transform domain. IEEE Trans Instrum Meas 68(1):49–64

    Google Scholar 

  52. Zhan K, Li Q, Teng J (2015) Multifocus image fusion using phase congruency. J Electron Imaging 24(3):033014

    Google Scholar 

  53. Zhan K, Shi J, Wang H (2017) Computational mechanisms of pulse-coupled neural networks: a comprehensive review. Arch Comput Methods Eng 24(3):573–588

    MathSciNet  MATH  Google Scholar 

  54. Zhang P, Yuan Y, Fei C (2018) Infrared and visible image fusion using co-occurrence filter. Infrared Phys Technol 93:223–231

    Google Scholar 

  55. Zhang J, Fan D, Dai Y (2020) UC-Net: Uncertainty inspired RGB-D saliency detection via conditional variational autoencoders. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR2020). (Accepted)

  56. Zhang L, Jia Z, Koefoed L, Yang J, Kasabov N (2020) Remote sensing image enhancement based on the combination of adaptive nonlinear gain and the PLIP model in the NSST domain. Multimed Tools Appl 79:13647–13665

    Google Scholar 

  57. Zhao J, Cao Y, Fan D, Cheng M (2019) Contrast prior and fluid pyramid integration for RGBD salient object detection. In Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR2019) 3922–3931

  58. Zhao J, Liu J, Fan D (2019) EGNet:Edge guidance network for salient object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV2019) 8778–8787

  59. Zhu Z, Zheng M, Qi G (2019) A phase congruency and local laplacian energy based multi-modality medical image fusion method in NSCT domain. IEEE Access 7:20811–20824

    Google Scholar 

Download references

Acknowledgments

This work was supported by the Science and Technology Development Plan Project of Jilin Province under Grant Nos. 20170414017GH and 20190302035GX; the Natural Science Foundation of Guangdong Province under Grant No. 2016A030313658; the Innovation and Strengthening School Project (provincial key platform and major scientific research project) supported by Guangdong Government under Grant No. 2015KTSCX175; the Premier-Discipline Enhancement Scheme Supported by Zhuhai Government under Grant No. 2015YXXK02-2; the Premier Key-Discipline Enhancement Scheme Supported by Guangdong Government Funds under Grant No. 2016GDYSZDXK036.

Author information

Authors and Affiliations

  1. College of Communication Engineering, Jilin University, Changchun, 130012, China

    Liangliang Li & Yujuan Si

  2. School of Electronic Information Engineering, Zhuhai College of Jilin University, Zhuhai, 519041, China

    Yujuan Si

  3. School of Mathematics and Information Science, Xinxiang University, Xinxiang, 453003, China

    Linli Wang

  4. College of Information Science and Engineering, Xinjiang University, Urumqi, 830046, China

    Zhenhong Jia

  5. Department of Electronic Engineering, Tsinghua University, Beijing, 100084, China

    Hongbing Ma

Authors
  1. Liangliang Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Yujuan Si
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Linli Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Zhenhong Jia
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Hongbing Ma
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Yujuan Si.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, L., Si, Y., Wang, L. et al. A novel approach for multi-focus image fusion based on SF-PAPCNN and ISML in NSST domain. Multimed Tools Appl 79, 24303–24328 (2020). https://doi.org/10.1007/s11042-020-09154-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09154-4

Keywords