Skip to main content
SpringerLink
Log in

Effects of haze and dehazing on deep learning-based vision models

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Most deep-learning-based vision models are trained and tested on clear images, avoiding noisy, or hazy, images. However, these models may encounter degraded images. So, it is important to recover and enhance them using a dehazing process. Dehazing usually serves as a preprocessing step for low-, medium-, and high-level vision tasks. Therefore, this article empirically studies the impact of haze and dehazing on high-level vision tasks and considers the degree to which dehazing algorithms can improve a vision model’s performance. For this purpose, we created two synthetic hazy datasets and trained several detection and classification models on both clear and hazy images. We found that haze and fog can easily affect a vision model’s performance and observed that using dehazing directly as a preprocessing step for high-level vision tasks did not substantially improve vision model’s performance but also renders performance unreliable and unpredictable. Therefore, when developing deep vision models, the research community should maintain aspects of bad weather conditions, such as haze, mist, fog, and rain, to avoid the failure of their proposed outdoor vision models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Wu H, Qu Y, Lin S, Zhou J, Qiao R, Zhang Z, ... Ma L (2021) Contrastive learning for compact single image dehazing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 10551–10560)

  2. Zhang S, He F, Ren W, Yao J (2020) Joint learning of image detail and transmission map for single image dehazing. Vis Comput 36(2):305–316

    Article  Google Scholar 

  3. Zhang J, He F, Chen Y (2020) A new haze removal approach for sky/river alike scenes based on external and internal clues. Multimedia Tools and Applications 79(3):2085–2107

    Article  Google Scholar 

  4. Yoon KJ, Shyam P, Kim KS (2021, February) Towards domain invariant single image dehazing. In AAAI Conference on Aritificial Intelligence. Association for the Advancement of Artificial Intelligence

  5. Wang C, Huang Y, Zou Y, Xu Y (2021, June) FWB-Net: front white balance network for color shift correction in single image dehazing via atmospheric light estimation. In ICASSP 2021–2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 2040–2044). IEEE.

  6. Yi X, Ma B, Zhang Y, Liu L, Wu J (2021) Two-step image dehazing with intra-domain and inter-domain adaptation. arXiv preprint arXiv:2102.03501

  7. Pei Y, Huang Y, Zou Q, Zang H, Zhang X, Wang S (2018) Effects of image degradations to cnn-based image classification. arXiv preprint arXiv:1810.05552

  8. Pei, Y., Huang, Y., Zou, Q., Lu, Y., & Wang, S. (2018). Does haze removal help cnn-based image classification?. In Proceedings of the European Conference on Computer Vision (ECCV) (pp. 682–697).

  9. Zhang S, He F, Ren W (2020) NLDN: Non-local dehazing network for dense haze removal. Neurocomputing 410:363–373

    Article  Google Scholar 

  10. Zhang S, He F (2020) DRCDN: learning deep residual convolutional dehazing networks. Vis Comput 36(9):1797–1808

    Article  Google Scholar 

  11. Yang, J., Yu, K., Gong, Y., & Huang, T. (2009, June). Linear spatial pyramid matching using sparse coding for image classification. In 2009 IEEE Conference on computer vision and pattern recognition (pp. 1794–1801). IEEE.

  12. Deng, J., Dong, W., Socher, R., Li, L. J., Li, K., & Fei-Fei, L. (2009, June). Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition (pp. 248–255). Ieee.

  13. Everingham M, Van Gool L, Williams CK, Winn J, Zisserman A (2010) The pascal visual object classes (voc) challenge. Int J Comput Vision 88(2):303–338

    Article  Google Scholar 

  14. Lin, T. Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., ... & Zitnick, C. L. (2014, September). Microsoft coco: Common objects in context. In European conference on computer vision (pp. 740–755). Springer, Cham.

  15. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097–1105

    Google Scholar 

  16. Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.

  17. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., ... & Rabinovich, A. (2015). Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1–9).

  18. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770–778).

  19. Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., ... & Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861.

  20. Liu, L., Shen, C., & Van Den Hengel, A. (2015). The treasure beneath convolutional layers: Cross-convolutional-layer pooling for image classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 4749–4757).

  21. Wang, J., Yang, Y., Mao, J., Huang, Z., Huang, C., & Xu, W. (2016). Cnn-rnn: A unified framework for multi-label image classification. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2285–2294).

  22. Durand, T., Mordan, T., Thome, N., & Cord, M. (2017). Wildcat: Weakly supervised learning of deep convnets for image classification, pointwise localization and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 642–651).

  23. Wang, F., Jiang, M., Qian, C., Yang, S., Li, C., Zhang, H., ... & Tang, X. (2017). Residual attention network for image classification. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3156–3164).

  24. Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014). Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 580–587).

  25. He K, Zhang X, Ren S, Sun J (2015) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intell 37(9):1904–1916

    Article  Google Scholar 

  26. Girshick, R. (2015). Fast r-cnn. In Proceedings of the IEEE international conference on computer vision (pp. 1440–1448).

  27. Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: Towards real-time object detection with region proposal networks. Adv Neural Inf Process Syst 28:91–99

    Google Scholar 

  28. Yu, J., Jiang, Y., Wang, Z., Cao, Z., & Huang, T. (2016, October). Unitbox: An advanced object detection network. In Proceedings of the 24th ACM international conference on Multimedia (pp. 516–520).

  29. Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). You only look once: Unified, real-time object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 779–788).

  30. Redmon, J., & Farhadi, A. (2017). YOLO9000: better, faster, stronger. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 7263–7271).

  31. Redmon, J., & Farhadi, A. (1804). YOLOv3: an incremental improvement (2018). arXiv preprint arXiv:1804.02767, 20.

  32. Tan, M., Pang, R., & Le, Q. V. (2020). Efficientdet: Scalable and efficient object detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 10781–10790).

  33. Ren CX, Dai DQ, Yan H (2012) Coupled kernel embedding for low-resolution face image recognition. IEEE Trans Image Process 21(8):3770–3783

    Article  MathSciNet  MATH  Google Scholar 

  34. Zou WW, Yuen PC (2011) Very low resolution face recognition problem. IEEE Trans Image Process 21(1):327–340

    Article  MathSciNet  MATH  Google Scholar 

  35. Basu S, Karki M, Ganguly S, DiBiano R, Mukhopadhyay S, Gayaka S, ... Nemani R (2017) Learning sparse feature representations using probabilistic quadtrees and deep belief nets. Neural Processing Letters, 45(3), 855-867

  36. Xiao H, Rasul K, Vollgraf R (2017) Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747

  37. Karam LJ, Zhu T (2015, March) Quality labeled faces in the wild (QLFW): a database for studying face recognition in real-world environments. In Human Vision and Electronic Imaging XX (Vol. 9394, p. 93940B). International Society for Optics and Photonics

  38. Roy P, Ghosh S, Bhattacharya S, Pal U (2018) Effects of degradations on deep neural network architectures. arXiv preprint arXiv:1807.10108

  39. Wang Z, Chang S, Yang Y, Liu D, Huang TS (2016) Studying very low resolution recognition using deep networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4792–4800)

  40. Diamond S, Sitzmann V, Boyd S, Wetzstein G, Heide F (2017) Dirty pixels: Optimizing image classification architectures for raw sensor data. arXiv e-prints, arXiv-1701

  41. Li, B., Peng, X., Wang, Z., Xu, J., & Feng, D. (2017). Aod-net: All-in-one dehazing network. In Proceedings of the IEEE international conference on computer vision (pp. 4770–4778).

  42. Griffin, G., Holub, A., & Perona, P. (2007). Caltech-256 object category dataset.

  43. Zhang, N., Zhang, L., & Cheng, Z. (2017, November). Towards simulating foggy and hazy images and evaluating their authenticity. In International Conference on Neural Information Processing (pp. 405–415). Springer, Cham.

  44. Chen, D., He, M., Fan, Q., Liao, J., Zhang, L., Hou, D., ... & Hua, G. (2019, January). Gated context aggregation network for image dehazing and deraining. In 2019 IEEE winter conference on applications of computer vision (WACV) (pp. 1375–1383). IEEE.

  45. Qin, X., Wang, Z., Bai, Y., Xie, X., & Jia, H. (2020, April). FFA-Net: Feature fusion attention network for single image dehazing. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 34, No. 07, pp. 11908–11915).

  46. Mei, K., Jiang, A., Li, J., & Wang, M. (2018, December). Progressive feature fusion network for realistic image dehazing. In Asian conference on computer vision (pp. 203–215). Springer, Cham.

  47. Meng, G., Wang, Y., Duan, J., Xiang, S., & Pan, C. (2013). Efficient image dehazing with boundary constraint and contextual regularization. In Proceedings of the IEEE international conference on computer vision (pp. 617–624).

  48. Tarel, J. P., & Hautiere, N. (2009, September). Fast visibility restoration from a single color or gray level image. In 2009 IEEE 12th international conference on computer vision (pp. 2201–2208). IEEE.

  49. He K, Sun J, Tang X (2010) Single image haze removal using dark channel prior. IEEE Trans Pattern Anal Mach Intell 33(12):2341–2353

    Google Scholar 

  50. Zhu Q, Mai J, Shao L (2015) A fast single image haze removal algorithm using color attenuation prior. IEEE Trans Image Process 24(11):3522–3533

    Article  MathSciNet  MATH  Google Scholar 

  51. Li B, Ren W, Fu D, Tao D, Feng D, Zeng W, Wang Z (2018) Benchmarking single-image dehazing and beyond. IEEE Trans Image Process 28(1):492–505

    Article  MathSciNet  MATH  Google Scholar 

  52. Bochkovskiy, A., Wang, C. Y., & Liao, H. Y. M. (2020). Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934.

  53. Liu, Y., Zhao, G., Gong, B., Li, Y., Raj, R., Goel, N., ... & Tao, D. (2018). Improved techniques for learning to dehaze and beyond: A collective study. arXiv preprint arXiv:1807.00202.

  54. Liu, D., Wen, B., Liu, X., Wang, Z., & Huang, T. S. (2017). When image denoising meets high-level vision tasks: A deep learning approach. arXiv preprint arXiv:1706.04284.

  55. Cheng, B., Wang, Z., Zhang, Z., Li, Z., Liu, D., Yang, J., ... & Huang, T. S. (2017, October). Robust emotion recognition from low quality and low bit rate video: A deep learning approach. In 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII) (pp. 65–70). IEEE.

  56. Carlini, N., & Wagner, D. (2017, November). Adversarial examples are not easily detected: Bypassing ten detection methods. In Proceedings of the 10th ACM workshop on artificial intelligence and security (pp. 3–14).

Download references

Acknowledgements

This work was supported by National Science Foundation of China (NSFC-61860206004).

Author information

Authors and Affiliations

  1. College of Applied Sciences, Shenzhen University, Shenzhen, 518060, China

    Haseeb Hassan & Bingding Huang

  2. Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, School of Biomedical Engineering, Shenzhen University Health Science Center, Shenzhen, 518060, China

    Haseeb Hassan

  3. College of Big Data and Internet, Shenzhen Technology University, Shenzhen, 518188, China

    Haseeb Hassan & Bingding Huang

  4. Department of Computer Science & Engineering, SJB Institute of Technology, Bangalore, India

    Pranshu Mishra

  5. Department of Computer Science, National University of Computer and Emerging Sciences, Islamabad, Chiniot-Faisalabad Campus, Chiniot, 35400, Pakistan

    Muhammad Ahmad

  6. Department of Computing and Mathematics, Manchester Metropolitan University, Manchester, UK

    Ali Kashif Bashir

  7. School of Computer Science and Technology, Anhui University, Hefei, China

    Bin Luo

Authors
  1. Haseeb Hassan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pranshu Mishra
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Muhammad Ahmad
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ali Kashif Bashir
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bingding Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Bin Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Bingding Huang or Bin Luo.

Ethics declarations

Competing of interest

The authors whose names are listed declare no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hassan, H., Mishra, P., Ahmad, M. et al. Effects of haze and dehazing on deep learning-based vision models. Appl Intell 52, 16334–16352 (2022). https://doi.org/10.1007/s10489-022-03245-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03245-5

Keywords

Search

Navigation