Skip to main content
SpringerLink
Log in

Overview of Infrared and Visible Image Fusion Based on Deep Learning

  • Conference paper
  • First Online:
Computer Science and Education (ICCSE 2022)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1811))

Included in the following conference series:

  • 404 Accesses

Abstract

Deep learning can extract image features automatically and then fuse them under the constraint of loss function by training multi-layer and deep neural networks, which is more intelligent, and has been successfully applied to the field of infrared and visible image fusion. This paper gives an overview of infrared and visible image fusion methods, followed by a detailed analysis of the deep learning based infrared and visible image fusion framework and loss function, and points out the existing problems of infrared and visible image fusion methods and the development prospects.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ma, J., Zhang, H., et al.: GANMcC: a generative adversarial network with multiclassifification constraints for infrared and visible image fusion. IEEE Trans. Instrument. Measurm. 70, 1–14 (2021)

    Google Scholar 

  2. Chen, C., Meng, X., et al.: Infrared and visible image fusion method based on multiscale low-rank decomposition. Acta Optica Sinica, J. 40(11), 72–80 (2020)

    Google Scholar 

  3. Chen, G., Chen, Y.,  et al.: Infrared and visible image fusion based on discrete nonseparableshearlet transform and convolutional sparse representation. J. Jilin Univ. (Eng. Technol. Ed,), J. 51(03), 996–1010 (2021)

    Google Scholar 

  4. Liu, Y., Liu, S., et al.: A general framework for image fusion based on multi-scale transform and sparse representation. Inf. Fusion 24, 147–164 (2015)

    Article  Google Scholar 

  5. Xu, D., Wang, Y., et al.: Infrared and visible image fusion using a deep unsupervised framework with perceptual loss. IEEE Access, 206445–206458 (2020)

    Google Scholar 

  6. Liu, Y., Chen, X., et al.: Deep learning for pixel-level image fusion: Recent advances and future prospects. Inf. Fusion 42, 158–173 (2018)

    Article  Google Scholar 

  7. Liu, Y., Chen, X., et al.: Multi-focus image fusion with a deep convolutional neural network. Inf. Fusion 36, 191–207 (2017)

    Google Scholar 

  8. Yan, L., Cao, J., Rizvi, S.,  et al.: Improving the performance of image fusion based on visual saliency weightMap combined with CNN. IEEE Access, 59976–59986 (2020)

    Google Scholar 

  9. An, W., Wang, H.: Infrared and visible image fusion with supervised convolutional neural network. Optik 219, 165–120 (2020)

    Article  Google Scholar 

  10. Li, L., Xia, Z., et al.: Infrared and visible image fusion using a shallow CNN and structural similarity constraint. IET image Proc. J. 14(14), 3562–3571 (2020)

    Article  Google Scholar 

  11. Han, X., Ma, J., et al.: U2Fusion: a unified unsupervised imagefusion network. IEEE Trans. Pattern Anal. Mach. Intell. J. 44(1), 502–518 (2020)

    Google Scholar 

  12. Xie, C., Li, X.: Infrared and visible image fusion: a regionbased deep learning method. Intell. Robotics  Appli. 11744, 604–615 (2019)

    Google Scholar 

  13. Raza, A., Liu, J., et al.: IRMSDNet: infrared and visible image fusion based on infrared features and multiscale dense network. IEEE J. Selected Topics Appli. Earth Observations  Remote Sens. 14, 3426–3437 (2021)

    Google Scholar 

  14. Li, S., Hu, J.: Image fusion with guided fifiltering. IEEE Trans. Image Process. 22(7), 2864–2875 (2013)

    Google Scholar 

  15. Li, H., Wu, X.,  et al.: Infrared and visible image fusion using a deep learning framework. In: 14th International Conference on Pattern Recognition, ICPR, pp. 2705–2710 (2018)

    Google Scholar 

  16. Han, X., Ma, J., et al.: FusionDN: a unifified densely connected network for image fusion. In: AAAI Conference on Artificial Intelligence. Journal 34(7), 12484–12491 (2020)

    Google Scholar 

  17. Bin, S., Wen, Y., et al.: Infrared and visible image fusion based on convolutional neural network. Infrared Technol. J. 42(7), 660–669 (2020)

    Google Scholar 

  18. Shen, Y., Chen, X.: Infrared and visible image fusion based on a latent low-rank representation decomposition and VGG Net. J. Beijing Univ. Aeronaut. Astron. 47(06), 1105–1114 (2021)

    Google Scholar 

  19. Goodfellow, I., Pougetabadie, J., Mirza, M., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)

    Google Scholar 

  20. Ma, J., Wei, Y., et al.: FusionGAN: A generative adversarial network for infrared and visible image fusion. Inf. Fusion 48, 11–26 (2018)

    Article  MathSciNet  Google Scholar 

  21. Chen, Z., Fang, M., et al.: U-GAN model for infrared and visible images fusion. J. Northwestern Polytechnical Univ. 38(4), 904–912 (2020)

    Article  Google Scholar 

  22. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234–241 (2015)

    Google Scholar 

  23. Yang, Y., Xiang, J., et al.: Infrared and visible image fusion via texture conditional generative adversarial network. IEEE Trans. Circ. Syst. Video Technol. J. 31(12), 4771–4783 (2021)

    Google Scholar 

  24. Li, J., Huo, H., et al.: Infrared and visible image fusion using dual discriminators generative adversarial networks with wasserstein distance. Inf. Sci. 529, 28–41 (2020)

    Article  MathSciNet  Google Scholar 

  25. Li, Q., Li, Z., et al.: Coupled GAN with relativistic discriminators for infrared and visible images fusion. IEEE Sensors J. 21(6), 7458–7467 (2019)

    Google Scholar 

  26. Jiayi Ma, X., Han, J.J., et al.: DDcGAN: a dual-discriminator conditional generative adversarial network for multi-resolution image fusion. IEEE Trans. Image Process. 29, 4980–4995 (2020)

    Article  MATH  Google Scholar 

  27. Ram Prabhakar, K., Sai Srikar, V., Venkatesh Babu, R.: DeepFuse: a deep unsupervised approach for exposure fusion with exreme exposure image pairs. In: IEEE International Conference on Computer Vision, pp. 4724–4732 (2017)

    Google Scholar 

  28. Li, C., Sun, T., Xie, J.: EMF Deep learning based infrared and visible image fusion algorithm. Foreign Electronic Measurem. Technol. J. 39(10), 25–32 (2020)

    Google Scholar 

  29. K He, X Zhang, S Ren, J Sun. Deep residual learning for image recognition, pp. 770–778. IEEE (2016)

    Google Scholar 

  30. Huang, G., Liu, Z., Van Der Maatenet, L., et al.: Densely connected convolutional networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2261–2269 (2017)

    Google Scholar 

  31. Li, H., Xiaojun, W.: DenseFuse: a fusion approach to infrared and visible images. IEEE Trans. Image Process. J. 28(5), 2614–2623 (2019)

    Article  MathSciNet  Google Scholar 

  32. Guo, X., Meng, L., Mei, L., et al.: Multi-focus image fusion with Siamese self attention network. IET Image Process. J. 14(7), 1339–1446 (2020)

    Article  Google Scholar 

  33. Xie, C., Jian, X., et al.: Infrared and visible image fusion method based on deep learning. Command Inf. Syst. Technol. J. 11(2), 16–38 (2020)

    MathSciNet  Google Scholar 

  34. Huan, K., Li, X., et al.: Infrared and visible image fusion with convolutional neural network and NSST. Infrared  Laser Eng. J. 51(3), 512–519 (2022)

    MathSciNet  Google Scholar 

  35. Lou, X., Feng, X.: Infrared and visible image fusion in latent low rank representation framework based on convolution neural network and guided filtering. Acta Photonica Sinica J. 50(3), 188–201 (2021)

    Google Scholar 

  36. Li, H., Wu, X., Kittler, J.: MDLatLRR: a novel decomposition method for infrared and visible image fusion. IEEE Trans. Image Process. 29, 4733–4746 (2020)

    Article  MATH  Google Scholar 

  37. Xu, X.: Infrared and Visible Image Fusion Method Based on Sparse Representation. University of Electronic Science and Technology of China, Sichuan (2017)

    Google Scholar 

  38. Sun, C.,  Zhang, C.,  Xiong, N.: Infrared and visible image fusion techniques based on deep learning: A Review. Electronics 9(12), 2162 ( 2020)

    Google Scholar 

  39. Xu, S.: Research on Image Fusion Methods Based on Deep Learning. Jiangnan University, Jiangsu (2020)

    Google Scholar 

  40. Wang, J., Ke, C., Liu, M., et al.: Overview of infrared and visible image fusion algorithms based on neural network framework. Laser J. 41(7), 7–12 (2020)

    Google Scholar 

  41. Xiong, M.: Research on image fusion method based on deep neural network. Jiangnan University, Jiangsu (2019)

    Google Scholar 

Download references

Acknowledgements

This work is supported by the Ningxia Natural Science Foundation (No. 2022AAC03236), by the National Natural Science Foundation of China (No. 11961001, No. 61907012), by the First-Class Disciplines Foundation of Ningxia (No. NXYLXK2017B09), and by the Special project of North Minzu University (No. FWNX01), and by the Master Degree Candidate Innovation Program (No. YCX22106).

Author information

Authors and Affiliations

  1. Ningxia Key Laboratory of Intelligent Information and Big Data Processing School of Math and Information Science, North Minzu University, Yinchuan, China

    Haixia Zhao, Xia Chang & Yuelin Gao

Authors
  1. Haixia Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xia Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuelin Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xia Chang .

Editor information

Editors and Affiliations

  1. Xiamen University, Xiamen, China

    Wenxing Hong

  2. Sichuan University, Chengdu, China

    Yang Weng

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhao, H., Chang, X., Gao, Y. (2023). Overview of Infrared and Visible Image Fusion Based on Deep Learning. In: Hong, W., Weng, Y. (eds) Computer Science and Education. ICCSE 2022. Communications in Computer and Information Science, vol 1811. Springer, Singapore. https://doi.org/10.1007/978-981-99-2443-1_7

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-2443-1_7

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-2442-4

  • Online ISBN: 978-981-99-2443-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation