skip to main content
10.1145/3474085.3475432acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

DehazeFlow: Multi-scale Conditional Flow Network for Single Image Dehazing

Published: 17 October 2021 Publication History

Abstract

Single image dehazing is a crucial and preliminary task for many computer vision applications, making progress with deep learning. The dehazing task is an ill-posed problem since the haze in the image leads to the loss of information. Thus, there are multiple feasible solutions for image restoration of a hazy image. Most existing methods learn a deterministic one-to-one mapping between a hazy image and its ground-truth, which ignores the ill-posedness of the dehazing task. To solve this problem, we propose DehazeFlow, a novel single image dehazing framework based on conditional normalizing flow. Our method learns the conditional distribution of haze-free images given a hazy image, enabling the model to sample multiple dehazed results. Furthermore, we propose an attention-based coupling layer to enhance the expression ability of a single flow step, which converts natural images into latent space and fuses features of paired data. These designs enable our model to achieve state-of-the-art performance while considering the ill-posedness of the task. We carry out sufficient experiments on both synthetic datasets and real-world hazy images to illustrate the effectiveness of our method. The extensive experiments indicate that DehazeFlow surpasses the state-of-the-art methods in terms of PSNR, SSIM, LPIPS, and subjective visual effects.

Supplementary Material

MP4 File (MM21-fp1489.mp4)
Presentation video

References

[1]
Dana Berman, Shai Avidan, et al. 2016. Non-local image dehazing. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1674--1682.
[2]
Bolun Cai, Xiangmin Xu, Kui Jia, Chunmei Qing, and Dacheng Tao. 2016. Dehazenet: An end-to-end system for single image haze removal. IEEE Transactions on Image Processing, Vol. 25, 11 (2016), 5187--5198.
[3]
Dongdong Chen, Mingming He, Qingnan Fan, Jing Liao, Liheng Zhang, Dongdong Hou, Lu Yuan, and Gang Hua. 2019. Gated context aggregation network for image dehazing and deraining. In 2019 IEEE winter conference on applications of computer vision (WACV). IEEE, 1375--1383.
[4]
Qili Deng, Ziling Huang, Chung-Chi Tsai, and Chia-Wen Lin. 2020. HardGAN: A Haze-Aware Representation Distillation GAN for Single Image Dehazing. In European Conference on Computer Vision. Springer, 722--738.
[5]
Laurent Dinh, David Krueger, and Yoshua Bengio. 2014. Nice: Non-linear independent components estimation. arXiv preprint arXiv:1410.8516 (2014).
[6]
Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. 2016. Density estimation using real nvp. arXiv preprint arXiv:1605.08803 (2016).
[7]
Hang Dong, Jinshan Pan, Lei Xiang, Zhe Hu, Xinyi Zhang, Fei Wang, and Ming-Hsuan Yang. 2020. Multi-scale boosted dehazing network with dense feature fusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2157--2167.
[8]
Raanan Fattal. 2014. Dehazing using color-lines. ACM transactions on graphics (TOG), Vol. 34, 1 (2014), 1--14.
[9]
Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative Adversarial Networks. arxiv: 1406.2661 [stat.ML]
[10]
Kaiming He, Jian Sun, and Xiaoou Tang. 2010. Single image haze removal using dark channel prior. IEEE transactions on pattern analysis and machine intelligence, Vol. 33, 12 (2010), 2341--2353.
[11]
Diederik P Kingma and Prafulla Dhariwal. 2018. Glow: Generative flow with invertible 1x1 convolutions. arXiv preprint arXiv:1807.03039 (2018).
[12]
Boyi Li, Xiulian Peng, Zhangyang Wang, Jizheng Xu, and Dan Feng. 2017a. An all-in-one network for dehazing and beyond. arXiv preprint arXiv:1707.06543 (2017).
[13]
Boyi Li, Xiulian Peng, Zhangyang Wang, Jizheng Xu, and Dan Feng. 2017b. Aod-net: All-in-one dehazing network. In Proceedings of the IEEE international conference on computer vision. 4770--4778.
[14]
Boyi Li, Wenqi Ren, Dengpan Fu, Dacheng Tao, Dan Feng, Wenjun Zeng, and Zhangyang Wang. 2018b. Benchmarking single-image dehazing and beyond. IEEE Transactions on Image Processing, Vol. 28, 1 (2018), 492--505.
[15]
Runde Li, Jinshan Pan, Zechao Li, and Jinhui Tang. 2018a. Single image dehazing via conditional generative adversarial network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 8202--8211.
[16]
Yu Li, Robby T Tan, and Michael S Brown. 2015. Nighttime haze removal with glow and multiple light colors. In Proceedings of the IEEE international conference on computer vision. 226--234.
[17]
Rui Liu, Yu Liu, Xinyu Gong, Xiaogang Wang, and Hongsheng Li. 2019 a. Conditional adversarial generative flow for controllable image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7992--8001.
[18]
Xiaohong Liu, Yongrui Ma, Zhihao Shi, and Jun Chen. 2019 b. Griddehazenet: Attention-based multi-scale network for image dehazing. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 7314--7323.
[19]
Andreas Lugmayr, Martin Danelljan, Luc Van Gool, and Radu Timofte. 2020. Srflow: Learning the super-resolution space with normalizing flow. In European Conference on Computer Vision. Springer, 715--732.
[20]
Earl J McCartney. 1976. Optics of the atmosphere: scattering by molecules and particles. New York (1976).
[21]
Kangfu Mei, Aiwen Jiang, Juncheng Li, and Mingwen Wang. 2018. Progressive feature fusion network for realistic image dehazing. In Asian conference on computer vision. Springer, 203--215.
[22]
Srinivasa G Narasimhan and Shree K Nayar. 2002. Vision and the atmosphere. International journal of computer vision, Vol. 48, 3 (2002), 233--254.
[23]
Ko Nishino, Louis Kratz, and Stephen Lombardi. 2012. Bayesian defogging. International journal of computer vision, Vol. 98, 3 (2012), 263--278.
[24]
Albert Pumarola, Stefan Popov, Francesc Moreno-Noguer, and Vittorio Ferrari. 2020. C-flow: Conditional generative flow models for images and 3d point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7949--7958.
[25]
Xu Qin, Zhilin Wang, Yuanchao Bai, Xiaodong Xie, and Huizhu Jia. 2020. FFA-Net: Feature fusion attention network for single image dehazing. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 11908--11915.
[26]
Yanyun Qu, Yizi Chen, Jingying Huang, and Yuan Xie. 2019. Enhanced pix2pix dehazing network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8160--8168.
[27]
Wenqi Ren, Si Liu, Hua Zhang, Jinshan Pan, Xiaochun Cao, and Ming-Hsuan Yang. 2016. Single image dehazing via multi-scale convolutional neural networks. In European conference on computer vision. Springer, 154--169.
[28]
Wenqi Ren, Lin Ma, Jiawei Zhang, Jinshan Pan, Xiaochun Cao, Wei Liu, and Ming-Hsuan Yang. 2018. Gated fusion network for single image dehazing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3253--3261.
[29]
Danilo Rezende and Shakir Mohamed. 2015. Variational inference with normalizing flows. In International Conference on Machine Learning. PMLR, 1530--1538.
[30]
Yuanjie Shao, Lerenhan Li, Wenqi Ren, Changxin Gao, and Nong Sang. 2020. Domain adaptation for image dehazing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2808--2817.
[31]
Yafei Song, Jia Li, Xiaogang Wang, and Xiaowu Chen. 2017. Single image dehazing using ranking convolutional neural network. IEEE Transactions on Multimedia, Vol. 20, 6 (2017), 1548--1560.
[32]
Robby T Tan. 2008. Visibility in bad weather from a single image. In 2008 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 1--8.
[33]
Jean-Philippe Tarel and Nicolas Hautiere. 2009. Fast visibility restoration from a single color or gray level image. In 2009 IEEE 12th International Conference on Computer Vision. IEEE, 2201--2208.
[34]
Xintao Wang, Ke Yu, Shixiang Wu, Jinjin Gu, Yihao Liu, Chao Dong, Yu Qiao, and Chen Change Loy. 2018. Esrgan: Enhanced super-resolution generative adversarial networks. In Proceedings of the European conference on computer vision (ECCV) workshops. 0--0.
[35]
Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon. 2018. Cbam: Convolutional block attention module. In Proceedings of the European conference on computer vision (ECCV). 3--19.
[36]
Guandao Yang, Xun Huang, Zekun Hao, Ming-Yu Liu, Serge Belongie, and Bharath Hariharan. 2019. Pointflow: 3d point cloud generation with continuous normalizing flows. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 4541--4550.
[37]
Xitong Yang, Zheng Xu, and Jiebo Luo. 2018. Towards perceptual image dehazing by physics-based disentanglement and adversarial training. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32.
[38]
He Zhang and Vishal M Patel. 2018. Densely connected pyramid dehazing network. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3194--3203.
[39]
He Zhang, Vishwanath Sindagi, and Vishal M Patel. 2018b. Multi-scale single image dehazing using perceptual pyramid deep network. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops. 902--911.
[40]
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. 2018a. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition. 586--595.
[41]
Dong Zhao, Long Xu, Lin Ma, Jia Li, and Yihua Yan. 2020. Pyramid Global Context Network for Image Dehazing. IEEE Transactions on Circuits and Systems for Video Technology (2020).
[42]
Qingsong Zhu, Jiaming Mai, and Ling Shao. 2015. A fast single image haze removal algorithm using color attenuation prior. IEEE transactions on image processing, Vol. 24, 11 (2015), 3522--3533.

Cited By

View all
  • (2025)Image dehazing via self-supervised depth guidancePattern Recognition10.1016/j.patcog.2024.111051158(111051)Online publication date: Feb-2025
  • (2025)Multiscale hybrid feature guided normalizing flow for low-light image enhancementComputers and Electrical Engineering10.1016/j.compeleceng.2024.109922122(109922)Online publication date: Mar-2025
  • (2025)HUPE: Heuristic Underwater Perceptual Enhancement with Semantic Collaborative LearningInternational Journal of Computer Vision10.1007/s11263-024-02318-xOnline publication date: 4-Jan-2025
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
MM '21: Proceedings of the 29th ACM International Conference on Multimedia
October 2021
5796 pages
ISBN:9781450386517
DOI:10.1145/3474085
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. attention
  2. normalizing flow
  3. single image dehazing

Qualifiers

  • Research-article

Funding Sources

Conference

MM '21
Sponsor:
MM '21: ACM Multimedia Conference
October 20 - 24, 2021
Virtual Event, China

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)59
  • Downloads (Last 6 weeks)4
Reflects downloads up to 14 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Image dehazing via self-supervised depth guidancePattern Recognition10.1016/j.patcog.2024.111051158(111051)Online publication date: Feb-2025
  • (2025)Multiscale hybrid feature guided normalizing flow for low-light image enhancementComputers and Electrical Engineering10.1016/j.compeleceng.2024.109922122(109922)Online publication date: Mar-2025
  • (2025)HUPE: Heuristic Underwater Perceptual Enhancement with Semantic Collaborative LearningInternational Journal of Computer Vision10.1007/s11263-024-02318-xOnline publication date: 4-Jan-2025
  • (2024)Adaptive Multi-Feature Attention Network for Image DehazingElectronics10.3390/electronics1318370613:18(3706)Online publication date: 18-Sep-2024
  • (2024)Real-World Scene Image Enhancement with Contrastive Domain Adaptation LearningACM Transactions on Multimedia Computing, Communications, and Applications10.1145/369497320:12(1-23)Online publication date: 26-Nov-2024
  • (2024)HazeSpace2M: A Dataset for Haze Aware Single Image DehazingProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681382(9155-9164)Online publication date: 28-Oct-2024
  • (2024)Advancing Real-World Image Dehazing: Perspective, Modules, and TrainingIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.341673146:12(9303-9320)Online publication date: Dec-2024
  • (2024)Reliable Event Generation With Invertible Conditional Normalizing FlowIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.332653846:2(927-943)Online publication date: Feb-2024
  • (2024)Bridging the Gap Between Haze Scenarios: A Unified Image Dehazing ModelIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.341467734:11(11070-11085)Online publication date: Nov-2024
  • (2024)Fooling the Image Dehazing Models by First Order GradientIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.335798734:7(6265-6278)Online publication date: Jul-2024
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media