Skip to main content
SpringerLink
Log in

Multimodal image/video fusion rule using generalized pixel significance based on statistical properties of the neighborhood

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Image fusion has been receiving increasing attention in the research community with the aim of investigating general formal solutions to a wide spectrum of applications such as multifocus, multiexposure, multispectral (\(IR\)-visible) and multimodal medical (CT and MRI) image and video fusion. While there exist many fusion techniques for each of these applications, it is difficult to formulate a common fusion technique that works equally well for all these applications. This is mainly because of the different characteristics of the images involved in various applications and the correspondingly different requirements on the fused image. In this work, we propose a common generalized fusion framework for all these classes, based on the statistical properties of local neighborhood of a pixel. As the eigenvalue of the unbiased estimate of the covariance matrix of an image block depends on the strength of edges in that block, we propose to employ it to compute a quantity we call as the significance of a pixel. This generalized pixel significance in turn can be used as a measure of the useful information content in that block, and hence can be used in the fusion process. Several data sets were fused to compare the results with various recently published methods. The analysis shows that for all the image types into consideration, the proposed methods improve the quality of the fused image, both visually and quantitatively, by preserving all the relevant information.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Blum, R., Liu, Z.: Multi-sensor Image Fusion and Its Applications. CRC Press, London (2005)

    Google Scholar 

  2. Wald, L.: Data Fusion Definitions and Architectures Fusion of Images of Different Spatial Resolutions. Ecole des Mines de Paris (2002)

  3. Li, S., Yang, B.: Multifocus image fusion by combining curvelet and wavelet transform. Pattern Recogn. Lett. 29, 1295–1301 (2008)

    Article  Google Scholar 

  4. Arivazhagan, S., Ganesan, L.: A modified statistical approach for image fusion using wavelet transform. Springer J. Signal Image Video Process. SIViP 3, 137–144 (2009)

    Article  Google Scholar 

  5. Shah, P., Merchant, S.N., Desai, U.B.: Fusion of surveillance images in infrared and visible band using curvelet, wavelet and wavelet packet transform. Int. J. Wavelets Multiresolution Inf. Process. (IJWMIP) 8(2), 271–292 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  6. Shah, P., Merchant, S.N., Desai, U.B.: Multifocus and multispectral image fusion based on pixel significance using multiresolution decomposition. Springer J. Signal Image Video Process. SIViP 7(1), 95–109 (2013)

    Article  Google Scholar 

  7. Shah, P., Chandra Sekhar Reddy, B.: Content enhancement for revealing a camouflaged target by fusion of multispectral surveillance videos. Springer J. Signal Image Video Process. SIViP 7(3), 537–552 (2013)

  8. De, I., Chanda, B.: A simple and efficient algorithm for multifocus image fusion using morphological wavelets. Signal Process. 86(5), 924–936 (2006)

    Article  MATH  Google Scholar 

  9. Yang, X., Yang, W., Pei, J.: Different focus points images fusion based on wavelet decomposition. Int. Conf. Inf. Fusion 1, 3–8 (2000)

    Google Scholar 

  10. Shangli, C, Junmin, H., Zhongwei, L.: Medical image of PET/CT weighted fusion based on wavelet transform. In: The 2nd International Conference on Bioinformatics and Biomedical Engineering (ICBBE) 2008, pp. 2523–2525, Shanghai, 16-18 May 2008. http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4535844&isnumber=4534880

  11. Yang, L., Xin, L., Yucui, Y.: Medical image fusion based on wavelet packet transform and self-adaptive operator. In: The 2nd International Conference on Bioinformatics and Biomedical Engineering (ICBBE) 2008, pp. 2647–2650, Shanghai, 16–18 May 2008

  12. Chai, Y., He, Y., Ying, C.: CT and MRI image fusion based on contourlet using novel rule. In: The 2nd International Conference on Bioinformatics and Biomedical Engineering (ICBBE) 2008, pp. 2064–2067. Shanghai, China (2008)

  13. Ibrahim, S., Wirth, M.: Visible and IR data fusion technique using the contourlet transform. In: IEEE International Conference on Computational Science and Engineering, pp. 42–47 (2009)

  14. Choi, M., Kim, R.Y., Kim, M.G.: The curvelet transform for image fusion. Soc. Photogramm. Remote Sens. B8, 59–64 (2004)

    Google Scholar 

  15. Sharma, R., Pavel, M., Leen, T.: Multi-stream video fusion using local principal components analysis. In: Proceedings of SPIE, p. 3436 (1998)

  16. Reinhard, E., Ward, G., Pattanaik, S., Debevec, P.: High Dynamic Range Imaging: Acquisition, Display and Image-Based Lighting. Morgan Kaufmann, Publishers Inc., San Francisco (2005)

  17. Mitsunaga, T., Nayar, S.: Radiometric self calibration. In: IEEE Computer Society Conference on Computer Vision and, Pattern Recognition, 1, 2 vol. xxiii+637+663 (1999)

  18. Debevec, P.E., Malik, J.: Recovering high dynamic range radiance maps from photographs. In: Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH, pp. 369–378. ACM Press/Addison-Wesley, NY, USA (1997)

  19. Drago, F., Myszkowski, K., Annen, T., Chiba, N.: Adaptive logarithmic mapping for displaying high contrast scenes. Comput. Graph. Forum 22, 419–426 (2003)

    Article  Google Scholar 

  20. Fattal, R., Lischinski, D., Werman, M.: Gradient domain high dynamic range compression. ACM Trans. Graph. 21, 249–256 (2002)

    Article  Google Scholar 

  21. Li, Y., Sharan, L., Adelson, E.H.: Compressing and companding high dynamic range images with subband architectures. In: ACM SIGGRAPH Papers, pp. 836-844, NY, USA (2005)

  22. Goshtasby, A.: Fusion of multi-exposure images. Image Vis. Comput. 23(6), 611–618 (2005)

    Article  Google Scholar 

  23. Mertens, T., Kautz, J., Van Reeth, F.: Exposure fusion. In: 15th Pacific Conference on Computer Graphics and Applications, pp. 382–390 (2007)

  24. Shah, P., Merchant, S.N., Desai, U.B.: An efficient spatial domain fusion scheme for multifocus images using statistical properties of neighborhood. In: IEEE International Conference and Multimedia Expo (ICME), Barcelona (2011)

  25. Brown, L.G.: A survey of image registration techniques. ACM Comput. Surv. 24, 325–376 (1992)

    Article  Google Scholar 

  26. Zitova, B., Flusser, J.: Image registration methods: a survey. Image Vis. Comput. 21, 977–1000 (2003)

    Article  Google Scholar 

  27. Antoine Maintz, J.B., Viergever, M.A.: A survey of medical image registration. Med. Image Anal. 2(1), 1–36 (1998). doi:10.1016/S1361-8415(01)80026-8; ISSN 1361-8415

  28. Li, H., Manjunath, B.S., Mitra, S.K.: A contour-based approach to multisensor image registration. IEEE Trans. Image Process. 4, 320–334 (1995)

    Article  Google Scholar 

  29. Ventura, A., Rampini, A., Schettini, R.: Image registration by recognition of corresponding structures. IEEE Trans. Geosci. Remote Sens. 28, 305–314 (1990)

    Article  Google Scholar 

  30. Goshtasby, A., Stockman, G., Page, C.: A region-based approach to digital image registration with subpixel accuracy. IEEE Trans. Geosci. Remote Sens. 24, 390–399 (1986)

    Google Scholar 

  31. Yaw Wee, C., Paramesran, R.: Measure of image sharpness using eigenvalues. Int. J. Inf. Sci. 177, 2533–2552 (2007)

  32. Devlin Susan, J., Gnanadesikan, R., Kettenring, J.R.: Robust estimation and outlier detection with correlation coefficients. Biometrika 62(3), 531–545 (1975)

    Google Scholar 

  33. Petrovic, V., Xydeas, C.: Objective image fusion performance characterisation. Proc. ICCV 2, 1866–1871 (2005)

    Google Scholar 

Download references

Acknowledgments

This work was supported by Microsoft Research India under the MSRI PhD Fellowship Award 2008.

Author information

Authors and Affiliations

  1. SPANN Lab, Electrical Engineering Department, Indian Institute of Technology Bombay, Mumbai, 400076, India

    Parul Shah, T. V. Srikanth & Shabbir N. Merchant

  2. Indian Institute of Technology Hyderabad, Ordnance Factory Estate, Yeddumailaram, 502205, Andhra Pradesh, India

    Uday B. Desai

Authors
  1. Parul Shah
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. T. V. Srikanth
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shabbir N. Merchant
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Uday B. Desai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Parul Shah.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Shah, P., Srikanth, T.V., Merchant, S.N. et al. Multimodal image/video fusion rule using generalized pixel significance based on statistical properties of the neighborhood. SIViP 8, 723–738 (2014). https://doi.org/10.1007/s11760-013-0585-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-013-0585-4

Keywords

Search

Navigation