Skip to main content

Advertisement

SpringerLink
Log in

Multi-focus image fusion based on nonsubsampled compactly supported shearlet transform

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Multi-focus image fusion, which aims to combine multi-focus images of a scene to construct an all-in-focus image, has become a major topic in image processing. Different methods have been proposed in spatial or transform domain. But many methods usually suffer from fusion quality degradations, such as contrast reduction, artificial edges, and discontinuous phenomena at boundaries of focused regions, which may cause issues when going for further processing. In order to overcome these problems, we introduce a nonsubsampled compactly supported shearlet transform (NSCSST), which possesses multi-scale, multi-direction, translation invariance and spatial localization characteristics that are very important for image fusion in transform domain. The transform can be implemented sequentially by the shear transform and the separable anisotropic nonsubsampled wavelet transform (SANSWT). Furthermore, we propose a new image fusion method based on NSCSST. It consists of two aspects: multi-direction fusion and transform domain fusion, which respectively correspond to the shear transform and the SANSWT of NSCSST. For each sheared image pair, the SANSWT coefficients are firstly fused by the transform domain fusion rules. And then, the final fused image is obtained by the multi-direction fusion rules, ranging from the simple averaging method to the proposed complex genetic algorithm based method. Experimental results show that our method outperforms some other methods, such as the method based on bilateral gradient, the method based on nonsubsampled contourlet transform, the method based on simultaneous empirical wavelet transform, and the method based on guided filtering.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Aiazzi B, Alparone L, Barducci A, Baronti S, Pippi I (1999) Multispectral fusion of multisensor image data by the generalized Laplacian pyramid. In: Proceeding of 1999 IEEE international geoscience and remote sensing symposium, vol 2. IEEE, pp 1183–1185

  2. Aslantas V, Kurban R (2009) A comparison of criterion functions for fusion of multi-focus noisy images. Opt Commun 282(16):3231–3242

    Article  Google Scholar 

  3. Aslantas V, Toprak AN (2014) A pixel based multi-focus image fusion method. Opt Commun 332(1):350–358

    Article  Google Scholar 

  4. Baradarani A, Wu QMJ, Ahmadi M, Mendapara P (2012) Tunable halfband-pair wavelet filter banks and application to multifocus image fusion. Pattern Recogn 45(2):657–671

    Article  MATH  Google Scholar 

  5. Candès E, Demanet L, Donoho D, Ying L (2006) Fast discrete curvelet transforms. Multiscale Model Simul 5(3):861–899

    Article  MathSciNet  MATH  Google Scholar 

  6. Cao Y, Li S, Hu J (2011) Multi-focus image fusion by nonsubsampled shearlet transform. In: Proceedings of the 6th international conference on image and graphics. IEEE, pp 17–21

  7. Chen L, Li J, Chen C (2013) Regional multifocus image fusion using sparse representation. Opt Express 21(4):5182–5197

    Article  Google Scholar 

  8. Cunha ALD, Zhou J, Do MN (2006) The nonsubsampled contourlet transform: theory, design, and applications. IEEE Trans Image Process 15(10):3089–3101

    Article  Google Scholar 

  9. Desai UY, Mizuki MM, Masaki I, Horn BKP (1996) Edge and mean based image compression. Technical Report AIM-1584, Massachusetts Institute of Technology, Cambridge, MA USA

  10. Ding G, Guo Y, Zhou J, Gao Y (2016) Large-scale cross-modality search via collective matrix factorization hashing. IEEE Trans Image Process 25(11):5427–5440

    Article  MathSciNet  Google Scholar 

  11. Do MN, Vetterli M (2002) Contourlets: a directional multiresolution image representation. In: Proceedings of 2002 international conference on image processing, vol 1. IEEE, pp 357–360

  12. Do MN, Vetterli M (2005) The contourlet transform: an efficient directional multiresolution image representation. IEEE Trans Image Process 14(12):2091–2106

    Article  Google Scholar 

  13. Easley G, Labate D, Lim WQ (2008) Sparse directional image representations using the discrete shearlet transform. Appl Comput Harmon Anal 25(1):25–46

    Article  MathSciNet  MATH  Google Scholar 

  14. Eltoukhy HA, Kavusi S (2003) A computationally efficient algorithm for multi-focus image reconstruction. In: Proceedings of SPIE electronic imaging, pp 332–341

  15. Eskicioglu AM, Fisher PS (1995) Image quality measures and their performance. IEEE Trans Commun 43(12):2959–2965

    Article  Google Scholar 

  16. Gao G, Xu L, Feng D (2013) Multi-focus image fusion based on non-subsampled shearlet transform. IET Image Process 7(6):633–639

    Article  Google Scholar 

  17. Gao Y, Wang M, Tao D, Ji R, Dai Q. (2012) 3-D object retrieval and recognition with hypergraph analysis. IEEE Trans Image Process 21(9):4290–4303

    Article  MathSciNet  MATH  Google Scholar 

  18. Geng P, Huang M, Liu S, Feng J, Bao P (2016) Multifocus image fusion method of ripplet transform based on cycle spinning. Multimed Tools Appl 75(17):10,583–10,593

    Article  Google Scholar 

  19. Goshtasby AA, Nikolov S (2007) Image fusion: advances in the state of the art. Information Fusion 8(2):114–118

    Article  Google Scholar 

  20. Guo D, Yan J, Qu X (2015) High quality multi-focus image fusion using self-similarity and depth information. Opt Commun 338:138–144

    Article  Google Scholar 

  21. Guo K, Kutyniok G, Labate D (2006) Sparse multidimensional representations using anisotropic dilation and shear operators. In: Wavelets and splines. Nashboro Press, Athens, GA, pp 189–201

    Google Scholar 

  22. Guo K, Labate D (2007) Optimally sparse multidimensional representation using shearlets. SIAM J Math Anal 39(1):298–318

    Article  MathSciNet  MATH  Google Scholar 

  23. Holland JH (1992) Adaptation in natural and artificial systems. MIT Press, Cambridge, MA USA

    Google Scholar 

  24. Holshneider M, Kronland-Martinet R, Morlet J, Tchamitchian P (1989) A real-time algorithm for signal analysis with the help of the wavelet transform. In: Combes JM, Grossmann A, Tchamitchian P (eds) Proceedings of the International conference time-frequency methods and phase space wavelets. Springer-Verlag, Berlin, pp 286–297

    Google Scholar 

  25. Huang W, Jing Z (2007) Evaluation of focus measures in multi-focus image fusion. Pattern Recogn Lett 28(4):493–500

    Article  Google Scholar 

  26. Ji X, Zhang G (2015) Image fusion method of SAR and infrared image based on curvelet transform with adaptive weighting. Multimed Tools Appl 1–17. doi:10.1007/s11042-015-2879-8

  27. Kutyniok G, Labate D (2009) Resolution of the wavefront set using continuous shearlets. Trans Am Math Soc 361(5):2719–2754

    Article  MathSciNet  MATH  Google Scholar 

  28. Kutyniok G, Lim WQ, Zhuang X (2012) Digital shearlet transforms. In: Kutyniok G, Labate D (eds) Shearlets: Multiscale analysis for multivariate data, chap 7.Birkhäuser Basel, pp 239–282

  29. Labate D, Lim WQ, Kutyniok G, Weiss G (2005) Sparse multidimensional representation using shearlets. In: Papadakis M, Laine AF, Unser MA (eds) Proceedings of the SPIE wavelets XI, vol 5914, pp 254–262

  30. Lewis JJ, O’callaghan RJ, Nikolov SG, Bull DR, Canagarajah N (2007) Pixel- and region-based image fusion with complex wavelets. Information Fusion 8(2):119–130

    Article  Google Scholar 

  31. Li C, Yang X, Chu B, Lu W, Pang L (2010) A new image fusion quality assessment method based on contourlet and SSIM. In: Proceedings of the 3rd IEEE international conference on computer science and information technology, vol 5. IEEE, pp 246–249

  32. Li S, Kang X (2012) Fast multi-exposure image fusion with median filter and recursive filter. IEEE Trans Consum Electron 58(2):626–632

    Article  Google Scholar 

  33. Li S, Kang X, Hu J (2013) Image fusion with guided filtering. IEEE Trans Image Process 22(7):2864–2875

    Article  Google Scholar 

  34. Li S, Kwok JT, Wang Y (2001) Combination of images with diverse focuses using the spatial frequency. Information Fusion 2(3):169–176

    Article  Google Scholar 

  35. Li S, Kwok JT, Wang Y (2002) Multifocus image fusion using artificial neural networks. Pattern Recogn Lett 23(8):985–997

    Article  MATH  Google Scholar 

  36. Li S, Yang B (2008) Multifocus image fusion using region segmentation and spatial frequency. Image Vis Comput 26(7):971–979

    Article  Google Scholar 

  37. Li S, Yang B, Hu J (2011) Performance comparison of different multi-resolution transforms for image fusion. Information Fusion 12(2):74–84

    Article  Google Scholar 

  38. Lim WQ (2010) The discrete shearlet transform: a new directional transform and compactly supported shearlet frames. IEEE Trans Image Process 19(5):1166–1180

    Article  MathSciNet  MATH  Google Scholar 

  39. Lin Z, Ding G, Han J, Wang J (2016) Cross-view retrieval via probability-based semantics-preserving hashing. IEEE Trans Cybern 1–14. doi:10.1109/TCYB.2016.2608906

  40. Lin Z, Ding G, Hu M, Lin Y, Ge SS (2014) Image tag completion via dual-view linear sparse reconstructions. Comput Vis Image Underst 124:42–60

    Article  Google Scholar 

  41. Liu X, Zhou Y, Wang J (2014) Image fusion based on shearlet transform and regional features. AEU-Int J Electron C 68(6):471–477

    Article  Google Scholar 

  42. Liu Y, Liu S, Wang Z (2015) A general framework for image fusion based on multi-scale transform and sparse representation. Information Fusion 24:147–164

    Article  Google Scholar 

  43. Miao Q, Shi C, Li W (2013) Image fusion based on shearlets. In: Miao Q (ed) New advances in image fusion, chap 7. Intech, pp 113–133

  44. Miao Q, Shi C, Xu P, Yang M, Shi Y (2011) Multi-focus image fusion algorithm based on shearlets. Chin Opt Lett 9(4):1–5

    Google Scholar 

  45. Miao Q, Shi C, Xu P, Yang M, Shi Y (2011) A novel algorithm of image fusion using shearlets. Opt Commun 284(6):1540–1547

    Article  Google Scholar 

  46. Miao Q, Wang B (2005) A novel adaptive multi-focus image fusion algorithm based on PCNN and sharpness. In: Proceedings of SPIE, vol 5778. pp 704–712

  47. Moonon AU, Hu J (2015) Multi-focus image fusion based on NSCT and NSST. Sensing and Imaging 16(1):1–16

    Article  Google Scholar 

  48. Naidu V, Raol J (2008) Pixel-level image fusion using wavelets and principal component analysis. Def Sci J 58(3):338–352

    Article  Google Scholar 

  49. Nayar SK, Nakagawa Y (1994) Shape from focus. IEEE Trans Pattern Anal Mach Intell 16(8):824–831

    Article  Google Scholar 

  50. Nejatia M, Samavi S, Shiranib S (2015) Multi-focus image fusion using dictionary-based sparse representation. Information Fusion 25:72–84

    Article  Google Scholar 

  51. Pajares G, de la Cruz JM (2004) A wavelet-based image fusion tutorial. Pattern Recogn 37(9):1855–1872

    Article  Google Scholar 

  52. Patel R, Rajput M, Parekh P (2015) Comparative study on multi-focus image fusion techniques in dynamic scene. Int J Comput Appl 109(6):5–9

    Google Scholar 

  53. Pennec EL, Mallat S (2005) Sparse geometric image representations with bandelets. IEEE Trans Image Process 14(4):423–438

    Article  MathSciNet  Google Scholar 

  54. Peyré G, Mallat S (2005) Discrete bandelets with geometric orthogonal filters. In: Proceedings of 2005 IEEE International Conference on Image Processing, IEEE, pp i–65–8

  55. Piella G, Heijmans H (2003) A new quality metric for image fusion. In: Proceedings of 2003 International Conference on Image Processing. IEEE, Barcelona, Spain, pp 173–176

    Google Scholar 

  56. Qu X, Yan J, Xiao H, Zhu Z (2008) Image fusion algorithm based on spatial frequency-motivated pulse coupled neural networks in nonsubsampled contourlet transform domain. Acta Automatica Sinica 34(12):1508–1514

    Article  MATH  Google Scholar 

  57. Qu X, Yan J, Xie G, Zhu Z, Chen B (2007) A novel image fusion algorithm based on bandelet transform. Chin Opt Lett 5(10):569–572

    Google Scholar 

  58. Qu XB, Yan JW, Yang GD (2005) Sum-modified-Laplacian-based multifocus image fusion method in cycle spinning sharp frequency localized contourlet transform domain. Opt Precis Eng 13(2)

  59. Sharma M (2016) A review: image fusion techniques and applications. Int J Comput Sci Inf Technol 7(3):1082–1085

    Google Scholar 

  60. Tian J, Chen L (2012) Adaptive multi-focus image fusion using a wavelet-based statistical sharpness measure. Signal Process 92(9):2137–2146

    Article  Google Scholar 

  61. Tian J, Chen L, Ma L, Yu W (2011) Multi-focus image fusion using a bilateral gradient-based sharpness criterion. Opt Commun 284(1):80–87

    Article  Google Scholar 

  62. Wang H, Nie C, Li Y, Zhang K, Chen L (2011) A novel fusion algorithm for multi-focus image. In: Zhang J (ed) Applied informatics and communication, communications in computer and information science, vol 227. Springer, Berlin Heidelberg, pp 641–647

    Google Scholar 

  63. Wang J, Wang W, Li B, Xu G, Zhang R, Zhang J (2016) Exposure fusion via sparse representation and shiftable complex directional pyramid transform. Multimed Tools Appl pp 1–21. doi:10.1007/s11042-016-3868-2

  64. Wang Z, Bovik AC (2002) A universal image quality index. IEEE Signal Process Lett 9(3):81–84

    Article  Google Scholar 

  65. Wang Z, Ma Y, Gu J (2010) Multi-focus image fusion using PCNN. Pattern Recogn 43(6):2003–2016

    Article  MATH  Google Scholar 

  66. Xiang T, Yan L, Gao R (2015) A fusion algorithm for infrared and visible images based on adaptive dual-channel unit-linking PCNN in NSCT domain. Infrared Phys Technol 69:53–61

    Article  Google Scholar 

  67. Xu J, Yang L, Wu D (2011) Ripplet: a new transform for image processing. J Vis Commun Image Represent 21(7):627–639

    Article  Google Scholar 

  68. Yang L, Guo BL, Ni W (2008) Multimodality medical image fusion based on multiscale geometric analysis of contourlet transform. Neurocomputing 72(1-3):203–211

    Article  Google Scholar 

  69. Yang Y, Huang S, Gao J, Qian Z (2014) Multi-focus image fusion using an effective discrete wavelet transform based algorithm. Meas Sci Rev 14(2):102–108

    Article  Google Scholar 

  70. Yang Y, Tong S, Huang S, Lin P (2015) Multifocus image fusion based on NSCT and focused area detection. IEEE Sensors J 15(5):2824–2838

    Google Scholar 

  71. You X, Chen Q, Fang B, Tang YY (2006) Thinning character using modulus minima of wavelet transform. Int J Pattern Recognit Artif Intell 20(3):361–375

    Article  Google Scholar 

  72. Zhang D, You X, Wang P, Yanushkevich SN, Tang YY (2009) Facial biometrics using nontensor product wavelet and 2D discriminant techniques. Int J Pattern Recognit Artif Intell 23(3):521–543

    Article  Google Scholar 

  73. Zhang Q, Guo BL (2009) Multifocus image fusion using the nonsubsampled contourlet transform. Signal Process 89(7):1334–1346

    Article  MATH  Google Scholar 

  74. Zhang X, Li X, Feng Y (2017) Image fusion based on simultaneous empirical wavelet transform. Multimed Tools Appl 76(6):8175–8193

    Article  Google Scholar 

  75. Zhao S, Yao H, Gao Y, Ding G, Chua TS (2016) Predicting personalized image emotion perceptions in social networks. IEEE Trans Affective Comput. doi:10.1109/TAFFC.2016.2628787

  76. Zhao S, Yao H, Gao Y, Ji R, Ding G (2017) Continuous probability distribution prediction of image emotions via multi-task shared sparse regression. IEEE Trans Multimedia 19(3):632–645

    Article  Google Scholar 

  77. Zhao S, Yao H, Yang Y, Zhang Y (2014) Affective image retrieval via multi-graph learning. In: Proceedings of the 22nd ACM international conference on multimedia. ACM, pp 1025–1028

  78. Zhao S, Yao H, Zhang Y, Wang Y, Liu S (2015) View-based 3D object retrieval via multi-modal graph learning. Signal Process 112:110–118

    Article  Google Scholar 

Download references

Acknowledgments

The authors would like to thank the editor and anonymous reviewers for their detailed review and valuable comments. This work was supported by the NSF of China (No. 11301137), the NSF of Hebei Province, China (No. A2014205100), the Educational Commission of Hebei Province, China (No. ZD2014062).

Author information

Authors and Affiliations

  1. College of Mathematics and Information Sciences, Hebei Normal University, Shijiazhuang, 050024, China

    Chunyu Wei, Bingyin Zhou & Wei Guo

  2. Hebei Key Laboratory of Computational Mathematics and Applications, Hebei Normal University, Shijiazhuang, 050024, China

    Bingyin Zhou & Wei Guo

Authors
  1. Chunyu Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bingyin Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wei Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chunyu Wei.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wei, C., Zhou, B. & Guo, W. Multi-focus image fusion based on nonsubsampled compactly supported shearlet transform. Multimed Tools Appl 77, 8327–8358 (2018). https://doi.org/10.1007/s11042-017-4731-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-4731-9

Keywords

Search

Navigation