Skip to main content
SpringerLink
Log in

PSPAN:pyramid spatially weighted pixel attention network for image dehazing

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Haze-free images are the prerequisites for many high-level visual tasks, and thus image dehazing has become an active topic in computer vision. However, the existing image dehazing algorithms are limited in face of unevenly distributed haze and dense haze in some scenes. In this paper, we propose a Pyramid Spatially Weighted Pixel Attention Network (PSPAN) for single image dehazing by leveraging complementarity among different levels of features in a pyramid manner with unique attention methods. The proposed PSPAN utilizes the feature pyramid as the core network and consists of three modules: an efficient Multi-scale Feature Extraction Attention module, a pyramid Spatially Weighted Pixel Attention module, and an image reconstruction module. Specifically, PSPAN preprocesses hazy images first before acquiring abundant shared features. After that, these features are sent to different branches. To effectively fuse useful information from these different branches and obtain better-dehazed results, we propose an efficient feature aggregation attention module. Finally, the image reconstruction module is used to restore clear images. Meanwhile, a loss function that combines a mean square error loss part, an edge loss part, and a perceptual loss part is employed in PSPAN which can better preserve image details. Experimental results demonstrate that the proposed PSPAN achieves superior performance to other existing state-of-the-art algorithms in terms of accuracy and visual effect.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data Availability Statements

The datasets analysed during the current study are available in the public RESIDE Dataset and public LIVE Image Defogging Database. And the different algorithms’ results which performed in datasets during the current study are available from the public paper or the corresponding author on reasonable request.

References

  1. Cai B, Xu X, Jia K, Qing C, Tao D (2016) Dehazenet: An end-to-end system for single image haze removal. IEEE Transactions on Image Processing 25(11):5187–5198. https://doi.org/10.1109/TIP.2016.2598681

    Article  MathSciNet  Google Scholar 

  2. Kansal, I, Kasana, SS (2018) Minimum preserving subsampling-based fast image de-fogging. Journal of Modern Optics, 65(18):2103–2123. https://doi.org/10.1080/09500340.2018.1499976

  3. Fattal, R (2008) Single image dehazing. ACM transactions on graphics (TOG) 27(3):1–9. https://doi.org/10.1145/1360612.1360671

  4. Tan, RT (2008) Visibility in bad weather from a single image. In: 2008 IEEE conference on computer vision and pattern recognition, pp 1–8. https://doi.org/10.1109/CVPR.2008.4587643. IEEE

  5. Fattal, R (2014) Dehazing using color-lines. ACM transactions on graphics (TOG), 34(1):1–14. https://doi.org/10.1145/2651362

  6. Zhu, Q, Mai, J, Shao, L (2015) A fast single image haze removal algorithm using color attenuation prior. IEEE transactions on image processing, 24(11):3522–3533 . https://doi.org/10.1109/TIP.2015.2446191

  7. Dong Y, Liu Y, Zhang H, Chen S, Qiao Y (2020) Fd-gan: Generative adversarial networks with fusion-discriminator for single image dehazing. Proceedings of the AAAI Conference on Artificial Intelligence 34:10729–10736. https://doi.org/10.1609/aaai.v34i07.6701

    Article  Google Scholar 

  8. Fattal R (2008) Single image dehazing. ACM transactions on graphics (TOG) 27(3):1–9. https://doi.org/10.1145/1360612.1360671

    Article  Google Scholar 

  9. Fattal R (2014) Dehazing using color-lines. ACM transactions on graphics (TOG) 34(1):1–14. https://doi.org/10.1145/2651362

    Article  Google Scholar 

  10. Yang, H, Pan, J, Yan, Q, Sun, W, Ren, J, Tai, Y-W (2017) Image dehazing using bilinear composition loss function. https://doi.org/10.48550/arXiv.1710.00279

  11. Li, C, Guo, J, Porikli, F, Fu, H, Pang, Y (2018) A cascaded convolutional neural network for single image dehazing. IEEE Access, 6:24877–24887. https://doi.org/10.1109/ACCESS.2018.2818882

  12. Li, B, Peng, X, Wang, Z, Xu, J, Feng, D (2017) Aod-net: All-in-one dehazing network. In: Proceedings of the IEEE international conference on computer vision, pp 4770–4778

  13. He K, Sun J, Tang X (2011) Single image haze removal using dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(12):2341–2353. https://doi.org/10.1109/TPAMI.2010.168

    Article  Google Scholar 

  14. Mei, K, Jiang, A, Li, J, Wang, M (2018) Progressive feature fusion network for realistic image dehazing. In: Asian Conference on Computer Vision, pp 203–215. https://doi.org/10.1007/978-3-030-20887-5_13. Springer

  15. Qu, Y, Chen, Y, Huang, J, Xie, Y (2019) Enhanced pix2pix dehazing network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 8160–8168

  16. Dong, H, Pan, J, Xiang, L, Hu, Z, Zhang, X, Wang, F, Yang, M-H (2020) Multi-scale boosted dehazing network with dense feature fusion. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 2157–2167. https://doi.org/10.48550/arXiv.2004.1338

  17. Kansal I, Kasana SS (2018) Minimum preserving subsampling-based fast image de-fogging. J Modern Optics 65(18):2103–2123. https://doi.org/10.1080/09500340.2018.1499976

    Article  Google Scholar 

  18. Kansal I, Kasana SS (2018) Fusion-based image de-fogging using dual tree complex wavelet transform. Int J Wavelets Multiresolution Inf Process 16(06):1850054. https://doi.org/10.1142/S0219691318500546

    Article  MathSciNet  Google Scholar 

  19. Kansal I, Kasana SS (2020) Improved color attenuation prior based image de-fogging technique. Multimed Tools Appl 79(17–18):12069–12091. https://doi.org/10.1007/s11042-019-08240-6

    Article  Google Scholar 

  20. Lan Y, Cui Z, Su Y, Wang N, Li A, Zhang W, Li Q, Zhong X (2022) Online knowledge distillation network for single image dehazing. Scientific Reports 12(1):1–13. https://doi.org/10.1038/s41598-022-19132-5

    Article  Google Scholar 

  21. Hu, J, Shen, L, Sun, G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). https://doi.org/10.48550/arXiv.1709.01507

  22. Li C, Guo J, Porikli F, Fu H, Pang Y (2018) A cascaded convolutional neural network for single image dehazing. IEEE Access 6:24877–24887. https://doi.org/10.1109/ACCESS.2018.2818882

    Article  Google Scholar 

  23. Li B, Ren W, Fu D, Tao D, Feng D, Zeng W, Wang Z (2018) Benchmarking single-image dehazing and beyond. IEEE Transactions on Image Processing 28(1):492–505. https://doi.org/10.1109/TIP.2018.2867951

    Article  MathSciNet  Google Scholar 

  24. Mnih, V, Heess, N, Graves, A, et al (2014) Recurrent models of visual attention. Advances in neural information processing systems, vol 27

  25. Lu H, Li Y, Nakashima S, Serikawa S (2016) Single image dehazing through improved atmospheric light estimation. Multimed Tools Appl 75(24):17081–17096. https://doi.org/10.1007/s11042-015-2977-7

    Article  Google Scholar 

  26. Qin, X, Wang, Z, Bai, Y, Xie, X, Jia, H (2020) Ffa-net: Feature fusion attention network for single image dehazing. In: Proceedings of the AAAI conference on artificial intelligence, vol 34, pp 11908–11915. https://doi.org/10.1609/aaai.v34i07.6865

  27. He, K, Zhang, X, Ren, S, Sun, J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 770–778

  28. Qin X, Wang Z, Bai Y, Xie X, Jia H (2020) Ffa-net: Feature fusion attention network for single image dehazing. Proceedings of the AAAI conference on artificial intelligence 34:11908–11915. https://doi.org/10.1609/aaai.v34i07.6865

    Article  Google Scholar 

  29. Wang, Q, Wu, B, Zhu, P, Li, P, Zuo, W, Hu, Q (2020) Supplementary material for ’eca-net: Efficient channel attention for deep convolutional neural networks. In: Proceedings of the 2020 IEEE/CVF conference on computer vision and pattern recognition, IEEE, Seattle, WA, USA, pp 13–19

  30. Xu, K, Ba, J, Kiros, R, Cho, K, Courville, A, Salakhudinov, R, Zemel, R, Bengio, Y (2015) Show, attend and tell: Neural image caption generation with visual attention. In: International conference on machine learning, pp 2048–2057. PMLR

  31. Vaswani, A, Shazeer, N, Parmar, N, Uszkoreit, J, Jones, L, Gomez, A.N, Kaiser, Ł, Polosukhin, I (2017) Attention is all you need. Advances in neural information processing systems, 30

  32. Wang, X, Girshick, R, Gupta, A, He, K (2018) Non-local neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 7794–7803. https://doi.org/10.48550/arXiv.1711.07971

  33. Hong, M, Xie, Y, Li, C, Qu, Y (2020) Distilling image dehazing with heterogeneous task imitation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3462–3471

  34. Gilbarg, D, Trudinger, NS (1983) Elliptic partial differential equations of second order, grundlehren der mathematischen wissenschaften. Berlin Heidelberg New York ed

  35. Simonyan, K, Zisserman, A (2014) Very deep convolutional networks for large-scale image recognition. https://doi.org/10.48550/arXiv:1409.1556

  36. Johnson, J, Alahi, A, Fei-Fei, L (2016) Perceptual losses for real-time style transfer and super-resolution. In: European conference on computer vision, pp 694–711. https://doi.org/10.1007/978-3-319-46475-6_43. Springer

  37. Zhang, H, Patel, VM (2018) Densely connected pyramid dehazing network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3194–3203. https://doi.org/10.48550/arXiv.1803.08396

  38. Ren, W, Liu, S, Zhang, H, Pan, J, Cao, X, Yang, M-H (2016) Single image dehazing via multi-scale convolutional neural networks. In: European conference on computer vision, pp 154–169. Springer

  39. Chen, D, He, M, Fan, Q, Liao, J, Zhang, L, Hou, D, Yuan, L, Hua, G (2019) Gated context aggregation network for image dehazing and deraining. In: 2019 IEEE winter conference on applications of computer vision (WACV):pp 1375–1383. https://doi.org/10.1109/WACV.2019.00151

  40. Lan, Y, Cui, Z, Su, Y, Wang, N, Li, A, Zhang, W, Li, Q, Zhong, X (2022) Online knowledge distillation network for single image dehazing. Scientific Reports, 12(1):1–13. https://doi.org/10.1038/s41598-022-19132-5

  41. Kansal, I, Kasana, SS (2018) Fusion-based image de-fogging using dual tree complex wavelet transform. International Journal of Wavelets, Multiresolution and Information Processing, 16(06):1850054. https://doi.org/10.1142/S0219691318500546

  42. Choi, LK, A.C.B. You, J (2015) Referenceless prediction of perceptual fog density and perceptual image defogging. Image Process, 24(11). https://doi.org/10.1109/TIP.2015.2456502

  43. Li, B, Ren, W, Fu, D, Tao, D, Feng, D, Zeng, W, Wang, Z (2018) Benchmarking single-image dehazing and beyond. IEEE Transactions on Image Processing 28(1):492–505. https://doi.org/10.1109/TIP.2018.2867951

  44. Zhang X, Jiang R, Wang T, Huang P, Zhao L (2021) Attention-based interpolation network for video deblurring. Neurocomputing 453:865–875. https://doi.org/10.1016/j.neucom.2020.04.147

    Article  Google Scholar 

  45. Zhu Q, Mai J, Shao L (2015) A fast single image haze removal algorithm using color attenuation prior. IEEE transactions on image processing 24(11):3522–3533. https://doi.org/10.1109/TIP.2015.2446191

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This work is partially supported by Heilongjiang Province Natural Science Foundation (LH2022F005) and Northeast Petroleum University Guiding Innovation Fund (No.15071202202).

Author information

Authors and Affiliations

  1. School of Electrical Information Engineering, Northeast Petroleum University, 163319, Daqing, China

    YuBo Zhang, Tongxiang Xu & Kang Tian

Authors
  1. YuBo Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tongxiang Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kang Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tongxiang Xu.

Ethics declarations

Conflicts of interest

The authors declare that they have no conflict of interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Y., Xu, T. & Tian, K. PSPAN:pyramid spatially weighted pixel attention network for image dehazing. Multimed Tools Appl 83, 11367–11385 (2024). https://doi.org/10.1007/s11042-023-15844-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-15844-6

Keywords

Search

Navigation