Skip to main content
SpringerLink
Log in

MFAF-Net: image dehazing with multi-level features and adaptive fusion

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Hazy images often cause blurring, detail loss and color distortion, which makes it difficult to address the other visual tasks such as tracking, classification and object detection. In recent years, significant advances have been made in image dehazing task, dominated by convolutional neural networks (CNNs). Most existing CNNs methods tend to estimate the transmission map and atmospheric light and then recover the haze-free image based on atmospheric scattering model. However, its dehazing performance is limited due to inaccurate estimation. To this end, we present a new architecture called multi-level features and adaptive fusion network (MFAF-Net) for single image dehazing, which can obtain the haze-free image in an end-to-end manner. For one thing, we utilize a novel context enhanced module as the core of feature extraction, and it combines multi-scale dilation convolution layers with feature attention module, which enables to acquire much informative contextual information. For another, we present a new fusion approach called adaptive fusion module for both low- and high-level feature fusion, and it can provide more flexibility when handling features of inconsistent semantic and level; thus, our network restores images with more detailed information. Experimental results on both synthetic and real-world datasets demonstrate that MFAF-Net outperforms existing state-of-the-art methods in terms of quantitative and qualitative evaluation metrics. The code will be made publicly available on Github.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Redmon, J., Farhadi, A.: Yolov3: an incremental improvement. arXiv:1804.02767 (2018)

  2. Chen, C., Wang, G., Peng, C., Fang, Y., Qin, H.: Exploring rich and efficient spatial temporal interactions for real-time video salient object detection. IEEE Trans. Image Process. PP(99), 1–1 (2021)

    Google Scholar 

  3. Das, D.K., Shit, S., Ray, D.N., Majumder, S.: CGAN: closure-guided attention network for salient object detection. Vis. Comput. 38, 1–15 (2021)

    Google Scholar 

  4. Li, J., Zhang, S., Huang, T.: Multi-scale temporal cues learning for video person re-identification. IEEE Trans. Image Process. PP(99), 1–1 (2020)

    Google Scholar 

  5. Xie, J., Ge, Y., Zhang, J., Huang, S., Wang, H.: Low-resolution assisted three-stream network for person re-identification. Vis. Comput. 10, 1–11 (2021)

    Google Scholar 

  6. Jia, Z., Li, Y., Tan, Z., Wang, W., Wang, Z., Yin, G.: Domain-invariant feature extraction and fusion for cross-domain person re-identification. Vis. Comput. 39, 1–12 (2022)

    Google Scholar 

  7. McCartney, E.J.: Optics of the atmosphere: scattering by molecules and particles. nyjw (1976)

  8. Narasimhan, S.G., Nayar, S.K.: Chromatic framework for vision in bad weather. In: IEEE Computer Society Conference on Computer Vision & Pattern Recognition (2000)

  9. Narasimhan, S.G., Nayar, S.K.: Vision and the atmosphere. Int. J. Comput. Vis. 48(3), 233–254 (2002)

    Article  Google Scholar 

  10. He, K., Sun, J., Fellow, I.E.E.E., Tang, X.: Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2341–2353 (2011)

    Article  Google Scholar 

  11. Zhu, Q., Mai, J., Shao, L.: A fast single image haze removal algorithm using color attenuation prior. IEEE Trans. Image Process. 24(11), 3522–3533 (2015)

    Article  MathSciNet  Google Scholar 

  12. Berman, D., Treibitz, T., Avidan, S.: Non-local image dehazing. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)

  13. Cai, B., Xu, X., Jia, K., Qing, C., Tao, D.: Dehazenet: an end-to-end system for single image haze removal. IEEE Trans. Image Process. 25(11), 5187–5198 (2016)

    Article  MathSciNet  Google Scholar 

  14. Zhang, H., Patel, V.M.: Densely connected pyramid dehazing network (2018)

  15. Ren, W., Ma, L., Zhang, J., Pan, J., Cao, X., Liu, W., Yang, M.H.: Gated fusion network for single image dehazing (2018)

  16. Ren, W., Liu, S., Zhang, H., Pan, J., Cao, X., Yang, M.H.: Single image dehazing via multi-scale convolutional neural networks (2016)

  17. Zhang, S., He, F.: DRCDN: learning deep residual convolutional dehazing networks. Vis. Comput. 36(3), 1797–1808 (2019)

    Google Scholar 

  18. Yang, F., Zhang, Q.: Depth aware image dehazing. Vis. Comput. 11, 1–9 (2021)

    Google Scholar 

  19. Li, X., Hua, Z., Li, J.: Attention-based adaptive feature selection for multi-stage image dehazing. Vis. Comput. 39, 663–678 (2022)

    Article  Google Scholar 

  20. Zhang, S., Zhang, J., He, F., Hou, N.: DRDDN: dense residual and dilated dehazing network. Vis. Comput. 39, 1–17 (2022)

    Google Scholar 

  21. Xie, B., Guo, F., Cai, Z.: Improved single image dehazing using dark channel prior and multi-scale retinex. In: International Conference on Intelligent System Design & Engineering Application (2010)

  22. Singh, D., Kumar, V., Kaur, M.: Single image dehazing using gradient channel prior. Appl. Intell. 49(8), 4276–4293 (2019)

    Article  Google Scholar 

  23. Hu, Q., Zhang, Y., Zhu, Y., Jiang, Y., Song, M.: Single image dehazing algorithm based on sky segmentation and optimal transmission maps. Vis. Comput. 39, 1–17 (2022)

    Google Scholar 

  24. Huang, T., Li, S., Jia, X., Lu, H., Liu, J.: Neighbor2neighbor: self-supervised denoising from single noisy images (2021)

  25. Khongkraphan, K., Phonon, A., Nuiphom, S.: An efficient blind image deblurring using a smoothing function. Appl. Comput. Intell. Soft Comput. 2021(9), 1–10 (2021)

    Google Scholar 

  26. Fan, W., Wu, Y., Wang, C.: Single image rain streak removal via layer similarity prior. Appl Intell 51, 5822–5835 (2021)

    Article  Google Scholar 

  27. Yang, W., Tan, R.T., Wang, S., Fang, Y., Liu, J.: Single image deraining: from model-based to data-driven and beyond. IEEE Trans. Pattern Anal. Mach. Intell. PP(99), 1–1 (2020)

    Google Scholar 

  28. Li, B., Peng, X., Wang, Z., Xu, J., Dan, F.: Aod-net: all-in-one dehazing network. In: 2017 IEEE International Conference on Computer Vision (ICCV) (2017)

  29. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234–241. Springer, Berlin (2015)

  30. Chen, D., He, M., Fan, Q., Liao, J., Zhang, L., Hou, D., Yuan, L., Hua, G.: Gated context aggregation network for image dehazing and deraining. In: 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1375–1383. IEEE (2019)

  31. Ancuti, C.O., Ancuti, C., Vasluianu, F.A., Timofte, R.: Ntire 2020 challenge on nonhomogeneous dehazing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 490–491 (2020)

  32. Ancuti, C.O., Ancuti, C., Vasluianu, F.A., Timofte, R.: Ntire 2021 nonhomogeneous dehazing challenge report. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 627–646 (2021)

  33. Das, S.D., Dutta, S.: Fast deep multi-patch hierarchical network for nonhomogeneous image dehazing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 482–483 (2020)

  34. Engin, D., Gen, A., Ekenel, H.K.: Cycle-dehaze: enhanced cyclegan for single image dehazing. IEEE (2018)

  35. Li, L., Dong, Y., Ren, W., Pan, J., Gao, C., Sang, N., Yang, M.H.: Semi-supervised image dehazing. IEEE Trans. Image Process. 29, 2766–2779 (2020)

    Article  Google Scholar 

  36. Qin, X., Wang, Z., Bai, Y., Xie, X., Jia, H.: Ffa-net: feature fusion attention network for single image dehazing (2019)

  37. Wang, C., Wu, Y., Cai, Y., Yao, G., Su, Z., Wang, H.: Single image deraining via deep pyramid network with spatial contextual information aggregation. Appl. Intell. 50(5), 1437–1447 (2020)

    Article  Google Scholar 

  38. Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions. In: ICLR (2016)

  39. Lin, T.Y., Dollar, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)

  40. Wang, Z.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13, 600–612 (2004)

    Article  Google Scholar 

  41. Qu, Y., Chen, Y., Huang, J., Xie, Y.: Enhanced pix2pix dehazing network. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019)

  42. Zhang, X., Wang, T., Wang, J., Tang, G., Zhao, L.: Pyramid channel-based feature attention network for image dehazing. Comput. Vis. Image Underst. 197–198, 103003 (2020)

    Article  Google Scholar 

  43. Dong, Y., Liu, Y., Zhang, H., Chen, S., Qiao, Y.: FD-GAN: generative adversarial networks with fusion-discriminator for single image dehazing. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 7, pp. 10729–10736 (2020)

  44. Shao, Y., Li, L., Ren, W., Gao, C., Sang, N.: Domain adaptation for image dehazing. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020)

  45. Ullah, H., Muhammad, K., Irfan, M., Anwar, S., Sajjad, M., Imran, A.S., de Albuquerque, V.H.C.: Light-DehazeNet: a novel lightweight CNN architecture for single image dehazing. IEEE Trans. Image Process. 30, 8968–8982 (2021)

    Article  Google Scholar 

  46. Liu, X., Ma, Y., Shi, Z., Chen, J.: GridDehazeNet: attention-based multi-scale network for image dehazing. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 7314–7323 (2019)

  47. Li, B., Ren, W., Fu, D., Tao, D., Feng, D., Zeng, W., Wang, Z.: Benchmarking single-image dehazing and beyond. IEEE Trans. Image Process. 28, 492–505 (2018)

    Article  MathSciNet  Google Scholar 

  48. Ancuti, C., Ancuti, C.O., Timofte, R., Vleeschouwer, C.D.: I-haze: a dehazing benchmark with real hazy and haze-free indoor images. In: International Conference on Advanced Concepts for Intelligent Vision Systems, pp. 620–631. Springer (2018)

  49. Ancuti, C.O., Ancuti, C., Timofte, R., De Vleeschouwer, C.: O-haze: a dehazing benchmark with real hazy and haze-free outdoor images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 754–762 (2018)

  50. Huynh-Thu, Q., Ghanbari, M.: Scope of validity of PSNR in image/video quality assessment. Electron. Lett. 44(13), 800–801 (2008)

    Article  Google Scholar 

  51. Blau, Y., Mechrez, R., Timofte, R., Michaeli, T., Zelnik-Manor, L.: The 2018 PIRM challenge on perceptual image super-resolution. In: Proceedings of the European Conference on Computer Vision (ECCV) Workshops, pp. 0–0 (2018)

  52. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  53. Fu, J., Xu, J., Tasaka, K., Chen, Z.: Residual squeeze-and-excitation network for fast image deraining. arXiv:2006.00757 (2020)

  54. Wang, C., Fan, W., Zhu, H., Su, Z.: Single image deraining via nonlocal squeeze-and-excitation enhancing network. Appl. Intell. 50(3), 2932–2944 (2020)

    Article  Google Scholar 

  55. Xia, L., Wu, J., Lin, Z., Hong, L., Zha, H.: Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining. Springer, Cham (2018)

    Google Scholar 

Download references

Acknowledgements

This work was supported by the Beijing Key Laboratory of Precision Photoelectric Measuring Instrument and Technology, the Winter Olympics Key Project Technology Fund (2018YFF0300804).

Author information

Authors and Affiliations

  1. Beijing Key Laboratory for Precision Optoelectronic Measurement Instrument and Technology, School of Optics and Photonics, Beijing Institute of Technology, Beijing, 100081, China

    Weichao Yi, Liquan Dong, Ming Liu, Mei Hui, Lingqin Kong & Yuejin Zhao

  2. Yangtze Delta Region Academy of Beijing Institute of Technology, Jiaxing, 314019, China

    Liquan Dong, Ming Liu, Lingqin Kong & Yuejin Zhao

Authors
  1. Weichao Yi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Liquan Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ming Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mei Hui
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lingqin Kong
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yuejin Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Liquan Dong or Ming Liu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yi, W., Dong, L., Liu, M. et al. MFAF-Net: image dehazing with multi-level features and adaptive fusion. Vis Comput 40, 2293–2307 (2024). https://doi.org/10.1007/s00371-023-02917-8

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-023-02917-8

Keywords

Search

Navigation