Skip to main content
SpringerLink
Log in

Multi-scale Guided Image and Video Fusion: A Fast and Efficient Approach

  • Published:
Circuits, Systems, and Signal Processing Aims and scope Submit manuscript

Abstract

In this paper, we propose a general purpose, simple and fast fusion algorithm based on guided image filter. The proposed method can well combine useful source image information into the fused image supported by multi-scale image decomposition, structure transferring property, visual saliency detection, and weight map construction. Multi-scale image decomposition is appropriate to represent and manipulate image features at various scales. Structure transferring property enabled by our algorithm can induce structures of one source image into the other. A new visual saliency detection based on guided image filter introduced in this paper is able to extract significant regions from visually different images of the same scene. The choice of weight maps helped to integrate the complementary information pixel by pixel at each scale. Experimental outcomes of the proposed method are compared and analyzed with traditional and recent guided image filter-based fusion algorithms in terms of visual quality, fusion metrics and run time. In addition, to enhance fusion results further we made an effort to find a suitable image and video enhancement algorithm. The fusion performance analysis clearly indicates that the proposed method is very promising along with less run time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Notes

  1. https://sites.google.com/view/durgaprasadbavirisetti/datasets.

  2. https://figshare.com/articles/TNO_Image_Fusion_Dataset/1008029.

References

  1. D.P. Bavirisetti, R. Dhuli, Fusion of infrared and visible sensor images based on anisotropic diffusion and Karhunen-Loeve transform. IEEE Sens. J. 16(1), 203–209 (2016)

    Article  Google Scholar 

  2. D.P. Bavirisetti, R. Dhuli, Two-scale image fusion of visible and infrared images using saliency detection. Infrared Phys. Technol. 76, 52–64 (2016)

    Article  Google Scholar 

  3. D.P. Bavirisetti, R. Dhuli, Multi-filtering based edge preserving image fusion technique. Int. J. Serv. Technol. Manage 23(4), 275–289 (2017)

    Article  Google Scholar 

  4. D.P. Bavirisetti, V.K. Kollu, X. Gang, R. Dhuli, Fusion of MRI and CT images using guided image filter and image statistics. Int. J. Imaging Syst. Technol. 27, 227–237 (2017)

    Article  Google Scholar 

  5. D.P. Bavirisetti et al., A new image and video fusion method based on cross bilateral filter, in 2018 21st International Conference on Information Fusion (FUSION). IEEE (2018)

  6. G. Bhatnagar, Q.M.J. Wu, Z. Liu, Directive contrast based multimodal medical image fusion in NSCT domain. IEEE Trans. Multimed. 15(5), 1014–1024 (2013)

    Article  Google Scholar 

  7. R.S. Blum, Z. Liu (eds.), Multi-sensor image fusion and its applications (CRC Press, Boca Raton, 2005)

    Google Scholar 

  8. S. Chen, A. Ramli, Contrast enhancement using recursive mean-separate histogram equalization for scalable brightness preservation. IEEE Trans. Consum. Electron. 49(4), 1301–1309 (2003)

    Article  Google Scholar 

  9. J.-C. Chiang et al., High-dynamic-range image generation and coding for multi-exposure multi-view images. Circuits Syst. Signal Process. 36(7), 2786–2814 (2017)

    Article  MathSciNet  Google Scholar 

  10. R.R. Colditz et al., Influence of image fusion approaches on classification accuracy: a case study. Int. J. Remote Sens. 27(15), 3311–3335 (2006)

    Article  Google Scholar 

  11. N.D. Duong, S.D. Tio, A.S. Madhukumar, A cooperative spectrum sensing technique with dynamic frequency boundary detection and information-entropy-fusion for primary user detection. Circuits Syst. Signal Process. 30(4), 823–845 (2011)

    Article  MathSciNet  Google Scholar 

  12. W. Gan et al., Infrared and visible image fusion with the use of multi-scale edge-preserving decomposition and guided image filter. Infrared Phys. Technol. 72, 37–51 (2015)

    Article  Google Scholar 

  13. R.C. Gonzalez, R.E. Woods, in The Book, Digital Image Processing [M]. Publishing house of electronics industry 141.7 (2002)

  14. Y. Han et al., A new image fusion performance metric based on visual information fidelity. Inform. Fusion 14(2), 127–135 (2013)

    Article  Google Scholar 

  15. K. He, J. Sun, X. Tang, Guided image filtering. IEEE Trans. Pattern Anal. Mach. Intell. 35(6), 1397–1409 (2013)

    Article  Google Scholar 

  16. L. Itti, C. Koch, E. Niebur, A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998)

    Article  Google Scholar 

  17. A. Jameel, A. Ghafoor, M.M. Riaz, Improved guided image fusion for magnetic resonance and computed tomography imaging. Sci. World J. (2014). https://doi.org/10.1155/2014/695752

    Article  Google Scholar 

  18. A. Jameel, A. Ghafoor, M.M. Riaz, Wavelet and guided filter based multifocus fusion for noisy images. Optik Int. J. Light Electron Opt. 126(23), 3920–3923 (2015)

    Article  Google Scholar 

  19. A. Jameel, A. Ghafoor, M.M. Riaz, All in focus fusion using guided filter. Multidimension. Syst. Signal Process. 26(3), 879–889 (2015)

    Article  Google Scholar 

  20. U. Javed et al., Weighted fusion of MRI and PET images based on fractal dimension. Multidimension. Syst. Signal Process. 28(2), 679–690 (2017)

    Article  MathSciNet  Google Scholar 

  21. F.T. Jhohura, T. Howlader, S.M. Rahman, Bayesian fusion of ensemble of multifocused noisy images. Circuits Syst. Signal Process. 34(7), 2287–2308 (2015)

    Article  Google Scholar 

  22. Y. Kim, Contrast enhancement using brightness preserving bi-histogram equalization. IEEE Trans. Consum. Electron. 43(1), 1–8 (2002)

    Article  Google Scholar 

  23. M. Kim, M. Chung, Recursively separated and weighted histogram equalization for brightness preservation and contrast enhancement. IEEE Trans. Consum. Electron. 54(3), 1389–1397 (2008)

    Article  Google Scholar 

  24. S. Li, X. Kang, H. Jianwen, Image fusion with guided filtering. IEEE Trans. Image Process. 22(7), 2864–2875 (2013)

    Article  Google Scholar 

  25. J. Li et al., Visual saliency based on scale-space analysis in the frequency domain. IEEE Trans. Pattern Anal. Mach. Intell. 35(4), 996–1010 (2013)

    Article  Google Scholar 

  26. S. Liu et al., Image fusion based on complex-shearlet domain with guided filtering. Multidimension. Syst. Signal Process. 28(1), 207–224 (2017)

    Article  Google Scholar 

  27. X. Ma et al, Saliency analysis based on multi-scale wavelet decomposition, in 2013 16th International IEEE Conference on Intelligent Transportation Systems-(ITSC). IEEE (2013)

  28. K. Murahira, T. Kawakami, A. Taguchi, Modified histogram equalization for image contrast enhancement, in 4th International Symposium on Communications, Control and Signal Processing (ISCCSP). IEEE, pp. 1–5 (2010)

  29. S. Pachori, S. Raman, Multi-scale Saliency Detection using Dictionary Learning. arXiv preprint arXiv:1611.06307 (2016)

  30. M. Peng et al., Fault diagnosis of analog circuits using systematic tests based on data fusion. Circuits Syst. Signal Process. 32(2), 525–539 (2013)

    Article  MathSciNet  Google Scholar 

  31. Pritika, S. Budhiraja, Multimodal medical image fusion based on guided filtered multi-scale decomposition. Int. J. Biomed. Eng. Technol. 20(4), 285–301 (2016)

    Article  Google Scholar 

  32. S. Singh et al., Infrared and visible image fusion for face recognition, in Proceedings of SPIE, vol. 5404 (2004)

  33. H. Singh, V. Kumar, S. Bhooshan, A novel approach for detail-enhanced exposure fusion using guided filter. Sci. World J. (2014). https://doi.org/10.1155/2014/659217

  34. P. Shah, S.N. Merchant, U.B. Desai, Multifocus and multispectral image fusion based on pixel significance using multiresolution decomposition. Signal Image Video Process. 7(1), 95–109 (2013). https://doi.org/10.1007/s11760-011-0219-7

    Article  Google Scholar 

  35. P. Shah et al., Multimodal image/video fusion rule using generalized pixel significance based on statistical properties of the neighborhood. Signal Image Video Process. 8(4), 723–738 (2014)

    Article  Google Scholar 

  36. L. Shuaiqi, Z. Jie, S. Mingzhu, Medical image fusion based on rolling guidance filter and spiking cortical model. Comput. Math. Methods Med. (2015). https://doi.org/10.1155/2015/156043

  37. A. Toet, Iterative guided image fusion. Peer J. Comput. Sci. 2, e80 (2016)

    Article  Google Scholar 

  38. A. Toet, M.A. Hogervorst, Multiscale image fusion through guided filtering. SPIE Security + Defence. Int. Soc. Opt. Photonics (2016)

  39. N. Xu et al., Object tracking via deep multi-view compressive model for visible and infrared sequences, in 2018 21st International Conference on Information Fusion (FUSION). IEEE (2018)

  40. N. Xu et al., Relative object tracking algorithm based on convolutional neural network for visible and infrared video sequences, in Proceedings of the 4th International Conference on Virtual Reality. ACM (2018)

  41. C. Zhang, A.A. Sufi, Color enhancement in image fusion, in IEEE Workshop on Applications of Computer Vision, 2008. WACV 2008. IEEE (2008)

  42. T. Zhang et al., A novel method of signal fusion based on dimension expansion. Circuits Syst. Signal Process. 37(10), 4295–4318 (2018). https://doi.org/10.1007/s00034-018-0760-5

    Article  MathSciNet  Google Scholar 

  43. W. Zhou, A.C. Bovik, A universal image quality index. IEEE Signal Process. Lett. 9(3), 81–84 (2002)

    Article  Google Scholar 

  44. Z. Zhou et al., Fusion of infrared and visible images for night-vision context enhancement. Appl. Opt. 55(23), 6480–6490 (2016)

    Article  Google Scholar 

  45. K. Zuiderveld, Contrast limited adaptive histogram equalization. Graphic gems IV (Academic Press Professional, San Diego, 1994), pp. 474–485

    Google Scholar 

Download references

Acknowledgments

This work is sponsored by National Program on Key Basic Research Project (2014CB744903), National Natural Science Foundation of China (61673270), Shanghai Pujiang Program(16PJD028), Shanghai Industrial Strengthening Project (GYQJ-2017-5-08), Shanghai Science and Technology Committee Research Project (17DZ1204304) and Shanghai Engineering Research Center of Civil Aircraft Flight Testing. We would like to thank our postdoctoral researcher Dr. Xingchen Zhang for English proof reading of the manuscript.

Author information

Authors and Affiliations

  1. School of Aeronautics and Astronautics, Shanghai Jiao Tong University, Shanghai, China

    Durga Prasad Bavirisetti, Gang Xiao & Junhao Zhao

  2. School of Electronics Engineering, VIT University, Amaravati, A.P, India

    Ravindra Dhuli

  3. School of Automation Engineering, Shanghai University of Electrical Power, Shanghai, China

    Gang Liu

Authors
  1. Durga Prasad Bavirisetti
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gang Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Junhao Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ravindra Dhuli
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Gang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gang Xiao.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bavirisetti, D.P., Xiao, G., Zhao, J. et al. Multi-scale Guided Image and Video Fusion: A Fast and Efficient Approach. Circuits Syst Signal Process 38, 5576–5605 (2019). https://doi.org/10.1007/s00034-019-01131-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00034-019-01131-z

Keywords

Search

Navigation