skip to main content
10.1145/3603781.3603847acmotherconferencesArticle/Chapter ViewAbstractPublication PagescniotConference Proceedingsconference-collections
research-article

RepVGGFuse: an approach for infrared and visible image fusion network based on RepVGG architecture

Published: 27 July 2023 Publication History

Abstract

In this paper, we propose an infrared and visible image fusion network based on RepVGG architecture. This network adopts an encoder-decoder structure. The encoding network, which contains five RepVGG blocks, is utilized to extract deep features of infrared and visible images. Each layer of RepVGG blocks is constructed with 3x3, 1x1 and identity branches while training and converted to single-branch architecture constructed with 3x3 convolutional layers while inferring. These extracted features are added and the fusion image is reconstructed by the decoding network. The proposed method was compared with seven fusion methods and the result shows that the proposed fusion method can retain more contour and texture information with less noise. The proposed method is superior to the comparison methods. The code of the proposed fusion network is available at https://github.com/xiongzhangzzz/repvggfuse.

References

[1]
J. Ma, Y. Ma, and C. Li, “Infrared and visible image fusion methods and applications: A survey,” Inf. Fusion, vol. 45, pp. 153–178, Jan. 2019.
[2]
C. Sun, C. Zhang, and N. Xiong, “Infrared and Visible Image Fusion Techniques Based on Deep Learning: A Review,” Electronics, vol. 9, no. 12, p. 2162, 2020.
[3]
X. Ding, X. Zhang, N. Ma, J. Han, G. Ding, and J. Sun, “RepVGG: Making VGG-style ConvNets Great Again,” in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, Jun. 2021, pp. 13728–13737.
[4]
K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” presented at the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778.
[5]
K. R. Prabhakar, V. S. Srikar, and R. V. Babu, “DeepFuse: A Deep Unsupervised Approach for Exposure Fusion with Extreme Exposure Image Pairs,” presented at the 2017 IEEE International Conference on Computer Vision (ICCV), 2017, pp. 4724–4732.
[6]
H. Li and X.-J. Wu, “DenseFuse: A Fusion Approach to Infrared and Visible Images,” IEEE Trans. IMAGE Process., vol. 28, no. 5, p. 10, 2019.
[7]
Z. Tang, Y. Gao, Y. Zhu, Z. Zhang, M. Li and D. Metaxas, "CrossNorm and SelfNorm for Generalization under Distribution Shifts," 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 52-61.
[8]
T.-Y. Lin, “Microsoft COCO: Common Objects in Context,” in ECCV, Sep. 2014.
[9]
X. Zhang, P. Ye, and G. Xiao, “VIFB: A Visible and Infrared Image Fusion Benchmark,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Seattle, WA, USA, Jun. 2020, pp.
[10]
D. P. Bavirisetti and R. Dhuli, “Fusion of Infrared and Visible Sensor Images Based on Anisotropic Diffusion and Karhunen-Loeve Transform,” IEEE Sens. J., vol. 16, no. 1, pp. 203–209, 2015.
[11]
D. P. Bavirisetti, G. Xiao, J. Zhao, X. Zhang, and P. Wang, “A New Image and Video Fusion Method Based on Cross Bilateral Filter,” in 2018 21st International Conference on Information Fusion (FUSION), Jul. 2018, pp. 1–8.
[12]
Dr. V. Naidu, “Image Fusion Technique using Multi-resolution Singular Value Decomposition,” Def. Sci. J., vol. 61, Aug. 2011.
[13]
H. Li and X.-J. Wu, “Infrared and visible image fusion using Latent Low-Rank Representation,” ArXiv180408992 Cs, Aug. 2021.
[14]
J. Ma, W. Yu, P. Liang, C. Li, and J. Jiang, “FusionGAN: A generative adversarial network for infrared and visible image fusion,” Inf. Fusion, vol. 48, pp. 11–26, Aug. 2019.
[15]
J. Ma, H. Zhang, Z. Shao, P. Liang, and H. Xu, “GANMcC: A Generative Adversarial Network With Multiclassification Constraints for Infrared and Visible Image Fusion,” IEEE Trans. Instrum. Meas., vol. 70, pp. 1–14, 2021.
[16]
G. Qu, D. Zhang, and P. Yan, “Information measure for performance of image fusion,” Electron. Lett., vol. 38, no. 7, p. 313, 2002.
[17]
M. Haghighat and M. A. Razian, “Fast-FMI: Non-reference image fusion metric,” in 2014 IEEE 8th International Conference on Application of Information and Communication Technologies (AICT), Oct. 2014.
[18]
P. Jagalingam and A. V. Hegde, “A Review of Quality Metrics for Fused Image,” Aquat. Procedia, vol. 4, pp. 133–142, Jan. 2015.
[19]
Z. Wang, A. Bovik, H. R. Sheikh, and E. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE Trans Image Process, vol. 13, pp. 600–612, Jan. 2014.
[20]
Z. Wang, E. P. Simoncelli, and A. C. Bovik, “Multiscale structural similarity for image quality assessment,” in The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003, Pacific Grove, CA, USA, 2003, pp. 1398–1402.
[21]
V. Aslantas and E. Bendes, “A new image quality metric for image fusion: The sum of the correlations of differences,” AEU - Int. J. Electron. Commun., vol. 69, no. 12, pp. 1890–1896, Dec. 2015.

Index Terms

  1. RepVGGFuse: an approach for infrared and visible image fusion network based on RepVGG architecture
        Index terms have been assigned to the content through auto-classification.

        Recommendations

        Comments

        Information & Contributors

        Information

        Published In

        cover image ACM Other conferences
        CNIOT '23: Proceedings of the 2023 4th International Conference on Computing, Networks and Internet of Things
        May 2023
        1025 pages
        ISBN:9798400700705
        DOI:10.1145/3603781
        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 27 July 2023

        Permissions

        Request permissions for this article.

        Check for updates

        Author Tags

        1. RepVGG
        2. deep learning
        3. encoder-decoder
        4. image fusion

        Qualifiers

        • Research-article
        • Research
        • Refereed limited

        Conference

        CNIOT'23

        Acceptance Rates

        Overall Acceptance Rate 39 of 82 submissions, 48%

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • 0
          Total Citations
        • 19
          Total Downloads
        • Downloads (Last 12 months)7
        • Downloads (Last 6 weeks)0
        Reflects downloads up to 30 Jan 2025

        Other Metrics

        Citations

        View Options

        Login options

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        HTML Format

        View this article in HTML Format.

        HTML Format

        Figures

        Tables

        Media

        Share

        Share

        Share this Publication link

        Share on social media