Skip to main content
SpringerLink
Log in

FUFusion: Fuzzy Sets Theory for Infrared and Visible Image Fusion

  • Conference paper
  • First Online:
Pattern Recognition and Computer Vision (PRCV 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14426))

Included in the following conference series:

  • 502 Accesses

Abstract

Infrared and visible image fusion combines the high resolution, rich structure of visible images with the remarkable information of infrared images for a wide range of applications in tasks such as target detection and segmentation. However, the representation and retention of significant and non-significant structures in fused images is still a big challenge. To address this issue, we proposed a novel infrared and visible image fusion approach based on the theory of fuzzy sets. First, we proposed a novel filter that integrates the SV-bitonic filter into a least squares model. By leveraging both global and local image features, this new filter achieved edge preservation and smoothness while effectively decomposing the image into structure and base layers. Moreover, for the fusion of the structure layers, according to the salient structural characteristics, we proposed a feature extraction method based on fuzzy inference system. Additionally, the intuitionistic fuzzy sets similarity measure was utilized to extract details from the residual structure layer. Extensive experiments demonstrate that the proposed model outperforms the state-of-the-art methods on publicly available datasets. The code is available at https://github.com/JEI981214/FUFusion_PRCV.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://github.com/hanna-xu/RoadScene.

  2. 2.

    https://github.com/JinyuanLiu-CV/TarDAL.

References

  1. Zhou, H., Sun, M., Ren, X., Wang, X.Y.: Visible-thermal image object detection via the combination of illumination conditions and temperature information. Remote Sens. 13, 3656 (2021)

    Article  Google Scholar 

  2. Chandrakanth, V., Murthy, V., Channappayya, S.S.: Siamese cross-domain tracker design for seamless tracking of targets in RGB and thermal videos. IEEE Trans. Artif. Intelli. 4, 161–172 (2022)

    Google Scholar 

  3. Wu, W., Qiu, Z.M., Zhao, M., Huang, Q.H., Lei, Y.: Visible and infrared image fusion using NSST and deep Boltzmann machine. Optik 157, 334–342 (2018)

    Article  Google Scholar 

  4. Zhang, X., Demiris, Y.: Visible and infrared image fusion using deep learning. IEEE Trans. Pattern Anal. Mach. Intelli. 45, 10535–10554 (2023)

    Article  Google Scholar 

  5. Gao, Y., Ma, S.W., Liu, J.J.: DCDR-GAN: a densely connected disentangled representation generative adversarial network for infrared and visible image fusion. IEEE Trans. Circ. Syst. Video Technol. 33, 549–561 (2023)

    Article  Google Scholar 

  6. Zhao, Z.X., Xu, S., Zhang, J.S., Liang, C.Y., Zhang, C.X., Liu, J.M.: Efficient and model-based infrared and visible image fusion via algorithm unrolling. IEEE Trans. Circ. Syst. Video Technol. 32, 1186–1196 (2022)

    Article  Google Scholar 

  7. Chang, Z.H., Feng, Z.X., Yang, S.Y., Gao, Q.W.: AFT: adaptive fusion transformer for visible and infrared images. IEEE Trans. Image Process. 32, 2077–2092 (2023)

    Article  Google Scholar 

  8. Tang, L.F., Xiang, X.Y., Zhang, H., Gong, M.Q., Ma, J.Y.: DIVFusion: darkness-free infrared and visible image fusion. Inform. Fusion 91, 477–493 (2023)

    Article  Google Scholar 

  9. Yang, Y., Zhang, Y., Huang, S., Zuo, Y., Sun, J.: Infrared and visible image fusion using visual saliency sparse representation and detail injection model. IEEE Trans. Instrum. Meas. 70, 1–15 (2021)

    Article  Google Scholar 

  10. Li, X., Zhou, F., Tan, H.: Joint image fusion and denoising via three-layer decomposition and sparse representation. Know.-Based Syst. 224, 107087 (2021)

    Article  Google Scholar 

  11. Li, H., Wu, X.J., Kittler, J.: MDLatLRR: a novel decomposition method for infrared and visible image fusion. IEEE Trans. Image Process. 29, 4733–4746 (2020)

    Article  Google Scholar 

  12. Mo, Y., Kang, X.D., Duan, P.H., Sun, B., Li, S.T.: Attribute filter based infrared and visible image fusion. Inform. Fusion 75, 41–54 (2021)

    Article  Google Scholar 

  13. Ren, L., Pan, Z., Cao, J., Zhang, H., Wang, H.: Infrared and visible image fusion based on edge-preserving guided filter and infrared feature decomposition. Signal Process. 186, 108108 (2021)

    Article  Google Scholar 

  14. Treece, G.: Morphology-based noise reduction: structural variation and thresholding in the bitonic filter. IEEE Trans. Image Process. 186, 336–350 (2020)

    Article  MathSciNet  Google Scholar 

  15. Liu, W., Zhang, P.P., Chen, X.G., Shen, C.H., Huang, X.L., Yang, J.: Embedding bilateral filter in least squares for efficient edge-preserving image smoothing. IEEE Trans. Circuits Syst. Video Technol. 30, 23–35 (2020)

    Article  Google Scholar 

  16. Alshennawy, A.A., Aly, A.A.: Edge detection in digital images using fuzzy logic technique. Int. J. Comput. Inform. Eng. 3, 540–548 (2009)

    Google Scholar 

  17. Li, S., Kang, X., Hu, J.: Image fusion with guided filtering. IEEE Trans. Image Process. 22, 2864–2875 (2013)

    Article  Google Scholar 

  18. Liu, J.Y., Fan, X., Jiang, J., Liu, R.S., Luo, Z.X.: Learning a deep multi-scale feature ensemble and an edge-attention guidance for image fusion. IEEE Trans. Circuits Syst. Video Technol. 32, 105–119 (2022)

    Article  Google Scholar 

  19. Zhou, H.B., Wu, W., Zhang, Y.D., Ma, J.Y., Ling, H.B.: Semantic-supervised infrared and visible image fusion via a dual-discriminator generative adversarial network. IEEE Trans. Multimedia 25, 635–648 (2023)

    Article  Google Scholar 

  20. Tang, L.F., Yuan, J.T., Ma, J.Y.: Image fusion in the loop of high-level vision tasks: a semantic-aware real-time infrared and visible image fusion network. Inform. Fusion 82, 28–42 (2022)

    Article  Google Scholar 

  21. Chen, J., Li, X.J., Luo, L.B., Mei, X.G., Ma, J.Y.: Infrared and visible image fusion based on target-enhanced multiscale transform decomposition. Inf. Sci. 508, 64–78 (2020)

    Article  Google Scholar 

  22. Ma, J., Zhang, H., Shao, Z., Liang, P., Han, X.: GANMcC: A generative adversarial network with multiclassification constraints for infrared and visible image fusion. IEEE Trans. Instrum. Meas. 70, 1–14 (2021)

    Google Scholar 

  23. Li, H., Wu, X.-J.: Infrared and visible image fusion using Latent low-rank representation. arXiv preprint arXiv:1804.08992 (2022)

  24. Qu, G., Zhang, D., Yan, P.: Information measure for performance of image fusion. Electron. lett. 38, 313–315 (2002)

    Article  Google Scholar 

  25. Wang, Q., Shen, Y., Jin, J.: Performance evaluation of image fusion techniques. Image Fusion Algorithms Appl. 19, 469–492 (2008)

    Article  Google Scholar 

  26. Xydeas, C.S., Petrovic, V.: Objective image fusion performance measure. Electron. Lett. 36, 308–309 (2000)

    Article  Google Scholar 

  27. Wang, P.-W., Liu, B.: A novel image fusion metric based on multi-scale analysis. In: Proceedings of the 2008 9th International Conference on Signal Processing, pp. 965–968 (2008)

    Google Scholar 

  28. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error measurement to structural similarity. IEEE Trans. Image Process. 13, 600–613 (2004)

    Article  Google Scholar 

  29. Han, Y., Cai, Y.Z., Cao, Y., Xu, X.M.: A new image fusion performance metric based on visual information fidelity. Inform. Fusion 14, 127–135 (2013)

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported by the National Natural Science Foundation of China (Grant No. 62201149).

Author information

Authors and Affiliations

  1. School of Physics and Optoelectronic Engineering, Foshan University, Foshan, 528225, China

    Yuchan Jie, Xiaosong Li & Haishu Tan

  2. School of Mechatronic Engineering and Automation, Foshan University, Foshan, 528225, China

    Yong Chen & Xiaoqi Cheng

  3. Jiangsu Shuguang Photoelectric Co., Ltd, Yangzhou, 225009, China

    Peng Yi

Authors
  1. Yuchan Jie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaosong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Peng Yi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Haishu Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Xiaoqi Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaosong Li .

Editor information

Editors and Affiliations

  1. Nanjing University of Information Science and Technology, Nanjing, China

    Qingshan Liu

  2. Xiamen University, Xiamen, China

    Hanzi Wang

  3. Beijing University of Posts and Telecommunications, Beijing, China

    Zhanyu Ma

  4. Sun Yat-sen University, Guangzhou, China

    Weishi Zheng

  5. Peking University, Beijing, China

    Hongbin Zha

  6. Chinese Academy of Sciences, Beijing, China

    Xilin Chen

  7. Chinese Academy of Sciences, Beijing, China

    Liang Wang

  8. Xiamen University, Xiamen, China

    Rongrong Ji

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Jie, Y., Chen, Y., Li, X., Yi, P., Tan, H., Cheng, X. (2024). FUFusion: Fuzzy Sets Theory for Infrared and Visible Image Fusion. In: Liu, Q., et al. Pattern Recognition and Computer Vision. PRCV 2023. Lecture Notes in Computer Science, vol 14426. Springer, Singapore. https://doi.org/10.1007/978-981-99-8432-9_37

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-8432-9_37

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-8431-2

  • Online ISBN: 978-981-99-8432-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation