skip to main content
10.1145/3655532.3655543acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicrsaConference Proceedingsconference-collections
research-article

End to end single image fog removal through a cycle consistent adversarial network guided by an assist network

Published: 28 June 2024 Publication History

Abstract

Many methods try to improve the generalization ability of the model by training on unpaired data. The most representative method is to use the generative adversarial network to remove the fog from a single image, and to constrain generative networks to generate fogfree images, by resisting loss. However, the simple adversarial loss cannot constraint the network to generate fogfree images, resulting in the loss of image structure texture. To solve this problem, we embed an Assist Network in the Cycle-Consistent Adversarial Networks, called ACDGan(Assist Cycle Defog Generative Adversarial Network), which can generate transmission graphs and atmospheric light, and then reconstruct the fog image to constrain the network to produce a fogfree images. ACDGan uses unpaired fog and fogfree training images, adversarial discriminators and cycle consistency losses to automatically construct a fog removal system. We design the CSAU(Coordinate Space Attention Unit) module which combines spatial attention and coordinate attention. CSAU can transform various deformation data in space and automatically capture the features of important regions. We have carried out a large number of experiments on synthetic images and fog images, and the proposed method has achieved significant improvement compared with the most advanced methods.

References

[1]
McCartney, Earl J. "Optics of the atmosphere: scattering by molecules and particles." New York (1976).
[2]
S.G. Narasimhan, S.K. Nayar, Vision and the atmosphere, Int. J. Comput. Vis. 48(3) (2002) 233–254.
[3]
Berman, Dana, Tali Treibitz, and Shai Avidan. "Air-light estimation using haze-lines." 2017 IEEE International Conference on Computational Photography (ICCP). IEEE, 2017.
[4]
W. Ren, S. Liu, H. Zhang, J. Pan, X. Cao, M.-H. Yang, Single image dehazing via multi-scale convolutional neural networks, in: European Conference on Computer Vision, Springer, 2016, pp. 154–169.
[5]
R. Fattal, Dehazing using color-lines, ACM Trans. Graph. 34 (1) (2014) 13.
[6]
D. Berman, S. Avidan, Non-local image dehazing, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 1674–1682
[7]
Kaiming He, Jian Sun, and Xiaoou Tang. Single image haze removal using dark channel prior. IEEE TPAMI,33(12):2341–2353, 2010.Kaiming He, Jian Sun, and Xiaoou Tang. Single image haze removal using dark channel prior. IEEE TPAMI,33(12):2341–2353, 2010.
[8]
Zhu Q, Mai J, Shao L. A fast single image haze removal algorithm using color attenuation prior[J]. IEEE transactions on image processing, 2015, 24(11): 3522-3533.Bolun Cai, Xiangmin Xu, Kui Jia, Chunmei Qing, and Dacheng Tao. Dehazenet: An end-to-end system for single image haze removal. IEEE TIP, 25(11):5187–5198, 2016.
[9]
Bolun Cai, Xiangmin Xu, Kui Jia, Chunmei Qing, and Dacheng Tao. Dehazenet: An end-to-end system for single image haze removal. IEEE TIP, 25(11):5187–5198, 2016.Li, B., Peng, X., Wang, Z., Xu, J., Dan, F.: AOD-Net: all-in-one dehazing network. In: 2017 IEEE International Conference on Computer Vision (ICCV). IEEE (2017)
[10]
Ren, W., Liu, S., Zhang, H., Pan, J., Cao, X., Yang, M.-H.Single image dehazing via multi-scale convolutional neural networks. In: Lecture Notes in Computer Science. Springer Science and Business Media, Cham, pp. 154–169 (2016)
[11]
Chen D, He M, Fan Q, Gated context aggregation network for image dehazing and deraining[C]//2019 IEEE winter conference on applications of computer vision (WACV). IEEE, 2019: 1375-1383.
[12]
Ullah H, Muhammad K, Irfan M, Light-DehazeNet: a novel lightweight CNN architecture for single image dehazing[J]. IEEE transactions on image processing, 2021, 30: 8968-8982.
[13]
He, Z., Patel, V.M.: Densely connected pyramid dehazing network. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2018)
[14]
Peng, Z., Li, Z., Zhang, J. et al. Few-shot image recognition with knowledge transfer. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 441–449 (2019)
[15]
Yang, X., Xu, Z., Luo, J.: Towards perceptual image dehazing by physics-based disentanglement and adversarial training. In: The Thirty-Second AAAI Conference on Artifcial Intelligence (AAAI-18) (2018)
[16]
Isola, P., Zhu, J.-Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, 21–26 July 2017, pp. 5967–5976 (2017)
[17]
Li, R., Pan, J., Li, Z., Tang, J.: Single image dehazing via conditional generative adversarial network. In: Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, 18–22 June 2018, pp. 8202–8211 (2018)
[18]
Engin, D., Genc, A., Ekenel, H.K.: Cycle-dehaze: enhanced CycleGAN for single image dehazing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Salt Lake City, 18–22 June 2018, pp. 938–9388 (2018)
[19]
Yang Y, Wang C, Liu R, Self-augmented unpaired image dehazing via density and depth decomposition[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022: 2037-2046.
[20]
Liu W, Hou X, Duan J,et al.End-to-End Single Image Fog Removal using Enhanced Cycle Consistent Adversarial Networks[J]. 2019.
[21]
Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycleconsistent adversarial networks. In ICCV, pages 2223–2232, 2017.
[22]
Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-toimage translation using cycle-consistent adversarial networks. IEEE (2017)
[23]
Qingsong Zhu, Jiaming Mai, and Ling Shao. A fast single image haze removal algorithm using color attenuation prior. IEEE TIP, 24(11):3522–3533, 2015.
[24]
Wenqi Ren, Si Liu, Hua Zhang, Jinshan Pan, Xiaochun Cao,and Ming-Hsuan Yang. Single image dehazing via multiscale convolutional neural networks. In ECCV, pages 154–169, 2016.
[25]
Boyun Li, Yuanbiao Gou, Jerry Zitao Liu, Hongyuan Zhu,Joey Tianyi Zhou, and Xi Peng. Zero-shot image dehazing. IEEE TIP, 29:8457–8466, 2020.
[26]
Alona Golts, Daniel Freedman, and Michael Elad. Unsupervised single image dehazing using dark channel prior loss. IEEE TIP, 29:2692–2701, 2019.
[27]
Ju M, Ding C, Ren W,et al.IDE: Image Dehazing and Exposure Using an Enhanced Atmospheric Scattering Model[J].IEEE Transactions on Image Processing, 2021.
[28]
Zhang H, Patel V M .Densely Connected Pyramid Dehazing Network[J].IEEE, 2018.
[29]
K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. International Conference on Learning Representations (ICLR), 2015.
[30]
Boyi Li, Wenqi Ren, Dengpan Fu, Dacheng Tao, Dan Feng, Wenjun Zeng, and Zhangyang Wang. Benchmarking single-image dehazing and beyond. IEEE Transactions on Image Processing, 28(1):492–505, 2018.
[31]
Yanfu Zhang, Li Ding, and Gaurav Sharma. Hazerd: an outdoor scene dataset and benchmark for single image dehazing. In Image Processing(ICIP), 2017 IEEE International Conference on, pages 3205–3209.IEEE, 2017.
[32]
Codruta O. Ancuti, Cosmin Ancuti, Mateu Sbert, and Radu Timofte. Dense haze: A benchmark for image dehazing with dense-haze and haze-free images. In ICIP, 2019.

Index Terms

  1. End to end single image fog removal through a cycle consistent adversarial network guided by an assist network

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    ICRSA '23: Proceedings of the 2023 6th International Conference on Robot Systems and Applications
    September 2023
    335 pages
    ISBN:9798400708039
    DOI:10.1145/3655532
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 28 June 2024

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Assist Network, Attention Mechanism
    2. Gan
    3. image defogging

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Conference

    ICRSA 2023

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 12
      Total Downloads
    • Downloads (Last 12 months)12
    • Downloads (Last 6 weeks)2
    Reflects downloads up to 20 Feb 2025

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media