Skip to main content
SpringerLink
Log in

Multi-scale depth information fusion network for image dehazing

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

According to the atmospheric physical model, we can use accurate transmittance and atmospheric light information to convert a hazy image into a clean one. The scene-depth information is very important for image dehazing due to the transmittance directly corresponds to the scene depth. In this paper, we propose a multi-scale depth information fusion network based on the U-Net architecture. The model uses hazy images as inputs and extracts the depth information from these images; then, it encodes and decodes this information. In this process, hazy image features of different scales are skip-connected to the corresponding positions. Finally, the model outputs a clean image. The proposed method does not rely on atmospheric physical models, and it directly outputs clean images in an end-to-end manner. Through numerous experiments, we prove that the multi-scale deep information fusion network can effectively remove haze from images; it outperforms other methods in the synthetic dataset experiments and also performs well in the real-scene test set.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. McCartney EJ (1977) Optics of the atmosphere: Scattering by molecules and particles. Int J Comput Vis 28(11):521–521

    Google Scholar 

  2. Narasimhan SG, Nayar SK (2000) Chromatic framework for vision in bad weather. In: Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No. PR00662), vol 1. IEEE, pp 598–605

  3. Narasimhan SG, Nayar SK (2002) Vision and the atmosphere. Int J Comput Vis 48(3):233–254

    Article  Google Scholar 

  4. He K, Sun J, Tang X (2010) Single image haze removal using dark channel prior. IEEE Trans Pattern Anal Mach Intell 33(12):2341–2353

    Google Scholar 

  5. Berman D, Avidan S, et al. (2016) Non-local image dehazing. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1674–1682

  6. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  7. Li J, Li G, Fan H (2018) Image dehazing using residual-based deep cnn. IEEE Access 6:26831–26842

    Article  Google Scholar 

  8. Cai B, Xu X, Jia K, Qing C, Tao D (2016) Dehazenet: An end-to-end system for single image haze removal. IEEE Trans Image Process 25(11):5187–5198

    Article  MathSciNet  Google Scholar 

  9. Li B, Peng X, Wang Z, Xu J, Feng D (2017) Aod-net: All-in-one dehazing network. In: Proceedings of the IEEE international conference on computer vision, pp 4770–4778

  10. Qu Y, Chen Y, Huang J, Xie Y (2019) Enhanced pix2pix dehazing network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 8160–8168

  11. Chen D, He M, Fan Q, Liao J, Zhang L, Hou D, Yuan L, Hua G (2019) Gated context aggregation network for image dehazing and deraining. In: 2019 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, pp 1375–1383

  12. Liu X, Ma Y, Shi Z, Chen J (2019) Griddehazenet: Attention-based multi-scale network for image dehazing. In: Proceedings of the IEEE International Conference on Computer Vision, pp 7314–7323

  13. Qin X, Wang Z, Bai Y, Xie X, Jia H (2020) Ffa-net: Feature fusion attention network for single image dehazing. In: AAAI, pp 11908–11915

  14. Dong H, Pan J, Xiang L, Hu Z, Zhang X, Wang F, Yang M-H (2020) Multi-scale boosted dehazing network with dense feature fusion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 2157–2167

  15. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention. Springer, pp 234–241

  16. Zhu K, Jiang X, Fang Z, Gao Y, Fujita H, Hwang J-N (2020) Photometric transfer for direct visual odometry. Knowl-Based Syst:106671

  17. Bissonnette LR (1992) Imaging through fog and rain. Opt Eng 31(5):1045–1053

    Article  Google Scholar 

  18. Narasimhan SG, Nayar SK (2003) Contrast restoration of weather degraded images. IEEE Trans Pattern Anal Mach Intell 25(6):713–724

    Article  Google Scholar 

  19. Ibrahim H, Kong NSP (2007) Brightness preserving dynamic histogram equalization for image contrast enhancement. IEEE Trans Consum Electron 53(4):1752–1758

    Article  Google Scholar 

  20. Kim J-Y, Kim L-S, Hwang S-H (2001) An advanced contrast enhancement using partially overlapped sub-block histogram equalization. IEEE Trans Circ Syst Video Technol 11(4):475–484

    Article  Google Scholar 

  21. Stark JA (2000) Adaptive image contrast enhancement using generalizations of histogram equalization. IEEE Trans Image Process 9(5):889–896

    Article  Google Scholar 

  22. Khan MF, Khan E, Abbasi ZA (2014) Segment dependent dynamic multi-histogram equalization for image contrast enhancement. Digital Signal Process 25:198–223

    Article  Google Scholar 

  23. Richter R (1996) Atmospheric correction with haze removal including a haze/clear transition region. In: Algorithms for Multispectral and Hyperspectral Imagery II, vol 2758. International Society for Optics and Photonics, pp 254–262

  24. Seow M-J, Asari VK (2006) Ratio rule and homomorphic filter for enhancement of digital colour image. Neurocomputing 69(7-9):954–958

    Article  Google Scholar 

  25. Daubechies I (1990) The wavelet transform, time-frequency localization and signal analysis. IEEE Trans Inf Theory 36(5):961–1005

    Article  MathSciNet  Google Scholar 

  26. Ma J, Plonka G (2010) The curvelet transform. IEEE Signal Process Mag 27(2):118–133

    Article  Google Scholar 

  27. Fattal R (2008) Single image dehazing. ACM Trans Graph (TOG) 27(3):1–9

    Article  Google Scholar 

  28. Arigela S, Asari V K (2014) Enhancement of hazy color images using a self-tunable transformation function. In: International Symposium on Visual Computing. Springer, pp 578–587

  29. Ren W, Liu S, Zhang H, Pan J, Cao X, Yang M-H (2016) Single image dehazing via multi-scale convolutional neural networks. In: European conference on computer visionSpringer, pp 154– 169

  30. Zhu H, Peng X, Chandrasekhar V, Li L, Lim J-H (2018) Dehazegan: When image dehazing meets differential programming. In: Proceedings of the twenty-seventh international joint conference on artificial intelligence, IJCAI-18. International Joint Conferences on Artificial Intelligence Organization, pp 1234–1240. https://doi.org/10.24963/ijcai.2018/172

  31. Wang T, Zhang X, Jiang R, Zhao L, Chen H, Luo W (2020) Video deblurring via spatiotemporal pyramid network and adversarial gradient prior. Comput Vis Image Underst 203:103135

    Article  Google Scholar 

  32. Ren W, Ma L, Zhang J, Pan J, Cao X, Liu W, Yang M-H (2018) Gated fusion network for single image dehazing. In: Proceedings of the IEEE Conference on Computerd Vision and Pattern Recognition, pp 3253–3261

  33. Zhang X, Wang T, Wang J, Tang G, Zhao L (2020) Pyramid channel-based feature attention network for image dehazing. Comput Vis Image Underst:103003

  34. Liu F, Shen C, Lin G, Reid I (2015) Learning depth from single monocular images using deep convolutional neural fields. IEEE Trans Pattern Anal Mach Intell 38(10):2024–2039

    Article  Google Scholar 

  35. Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507

    Article  MathSciNet  Google Scholar 

  36. Glorot X, Bordes A, Bengio Y (2011) Deep sparse rectifier neural networks. In: Proceedings of the fourteenth international conference on artificial intelligence and statistics, pp 315–323

  37. Nair V, Hinton GE (2010) Rectified linear units improve restricted boltzmann machines vinod nair. In: Proceedings of the 27th International Conference on Machine Learning (ICML-10), Haifa

  38. Maas AL, Hannun AY, Ng AY (2013) Rectifier nonlinearities improve neural network acoustic models. In: Proc. icml, vol 30, pp 3

  39. He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE international conference on computer vision, pp 1026–1034

  40. Clevert D-A, Unterthiner T, Hochreiter S (2015) Fast and accurate deep network learning by exponential linear units (elus). arXiv:1511.07289

  41. Chen Y, Dai X, Liu M, Chen D, Yuan L, Liu Z (2020) Dynamic relu. arXiv:2003.10027

  42. Hua Z, Fan G, Li J (2020) Iterative residual network for image dehazing. IEEE Access 8:167693–167710

    Article  Google Scholar 

  43. Silberman N, Hoiem D, Kohli P, Fergus R (2012) Indoor segmentation and support inference from rgbd images. In: European conference on computer vision. Springer, pp 746–760

  44. Li B, Ren W, Fu D, Tao D, Feng D, Zeng W, Wang Z (2017) Reside: A benchmark for single image dehazing, vol 1. arXiv:1712.04143

Download references

Acknowledgments

The authors acknowledge the National Natural Science Foundation of China (Grant nos. 61772319, 62002200, 61976125, 61873177 and 61773244), and Shandong Natural Science Foundation of China (Grant no. ZR2017MF049).

Author information

Author notes

  1. Guodong Fan & Zhen Hua

    Present address: School of Information and Electronic Engineering, Shandong Technology and Business University, Yantai, 264005, China

Authors and Affiliations

  1. School of Computer Science and Technology, Shandong Technology and Business University, Yantai, 264005, China

    Jinjiang Li

Authors
  1. Guodong Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhen Hua
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jinjiang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhen Hua.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fan, G., Hua, Z. & Li, J. Multi-scale depth information fusion network for image dehazing. Appl Intell 51, 7262–7280 (2021). https://doi.org/10.1007/s10489-021-02236-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02236-2

Keywords

Search

Navigation