Skip to main content
SpringerLink
Log in

A multi-scale attentive recurrent network for image dehazing

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Image dehazing is a pre-processing step in computer vision tasks, that has attracted considerable attention from the research community. Existing CNN-based methods ignore haze-related priors and rarely use a coarse-to-fine scheme in a feed-forward architecture to remove haze due to increasing network depth and parameters. This results in sub-optimal dehazing results. To address these problems, a multi-scale attentive recurrent network is proposed for image dehazing, which consists of a haze attention map predicted network and a recurrent encoder-decoder network. First, by assuming that haze in an image is formed by multiple layers with different depths, the haze attention map predicted network is designed for generating the map with multiple stages via a multi-scale recurrent framework. Second, the haze attention map is viewed as the haze-related prior and guides the subsequent recurrent encoder-decoder network to be aware of haze concentration information. Finally, for leveraging the intermediate information and optimizing the dehazing result with less parameters and more robust features, the recurrent residual operations which pass the features of selected layers at the current time step to the corresponding layers at the next time step are applied in the recurrent encoder-decoder network for removing haze following a coarse-to-fine strategy. Experiments on synthetic and real images demonstrate that our method outperforms state-of-the-art methods considering both visual and quantitative evaluations. In addition, our method is also suitable for real-time processing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Alajarmeh A, Salam R, Abdulrahim K, Marhusin M, Zaidan A, Zaidan B (2018) Real-time framework for image dehazing based on linear transmission and constant-time airlight estimation. Inf Sci 436:108–130

    Article  MathSciNet  Google Scholar 

  2. Berman D, Avidan S et al (2016) Non-local image dehazing. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1674–1682

  3. Cai B, Xu X, Jia K, Qing C, Tao D (2016) Dehazenet: An end-to-end system for single image haze removal. IEEE Trans Image Process 25(11):5187–5198

    Article  MathSciNet  Google Scholar 

  4. Chen CF, Fan Q, Mallinar N, Sercu T, Feris R (2018) Big-little net: An efficient multi-scale feature representation for visual and speech recognition. arXiv:180703848

  5. Chen D, He M, Fan Q, Liao J, Zhang L, Hou D, Yuan L, Hua G (2019a) Gated context aggregation network for image dehazing and deraining. In: 2019 IEEE Winter conference on applications of computer vision (WACV). IEEE, pp 1375–1383

  6. Chen SZ, Ren ZG, Lian QS (2016) Single image dehazing algorithm based on improved dark channel prior and guided filter. Acta Automatica Sinica 42(3):455–465

    Google Scholar 

  7. Chen Y, Wang J, Chen X, Sangaiah AK, Yang K, Cao Z (2019b) Image super-resolution algorithm based on dual-channel convolutional neural networks. Appl Sci 9(11):2316

    Article  Google Scholar 

  8. Chen Y, Wang J, Chen X, Zhu M, Yang K, Wang Z, Xia R (2019c) Single-image super-resolution algorithm based on structural self-similarity and deformation block features. IEEE Access 7:58791–58801

    Article  Google Scholar 

  9. Chen Y, Wang J, Liu S, Chen X, Xiong J, Xie J, Yang K (2019d) Multiscale fast correlation filtering tracking algorithm based on a feature fusion model. Concurrency and Computation: Practice and Experience e5533

  10. Chen Y, Wang J, Xia R, Zhang Q, Cao Z, Yang K (2019e) The visual object tracking algorithm research based on adaptive combination kernel. J Ambient Intell Human Comput 10(12):4855–4867

    Article  Google Scholar 

  11. Chen Y, Xiong J, Xu W, Zuo J (2019f) A novel online incremental and decremental learning algorithm based on variable support vector machine. Clust Comput 22(3):7435–7445

    Article  Google Scholar 

  12. Chen Y, Xu W, Zuo J, Yang K (2019g) The fire recognition algorithm using dynamic feature fusion and iv-svm classifier. Clust Comput 22(3):7665–7675

    Article  Google Scholar 

  13. Chen Y, Tao J, Liu L, Xiong J, Xia R, Xie J, Zhang Q, Yang K (2020a) Research of improving semantic image segmentation based on a feature fusion model. Journal of Ambient Intelligence and Humanized Computing

  14. Chen Y, Tao J, Zhang Q, Yang K, Chen X, Xiong J, Xia R, Xie J (2020b) Saliency detection via the improved hierarchical principal component analysis method. Wirel Commun Mob Comput 2020

  15. Deng Z, Zhu L, Hu X, Fu CW, Xu X, Zhang Q, Qin J, Heng PA (2019) Deep multi-model fusion for single-image dehazing. In: Proceedings of the IEEE international conference on computer vision (ICCV), pp 2453–2462

  16. Dong Y, Liu Y, Zhang H, Chen S, Qiao Y (2020) Fd-gan: Generative adversarial networks with fusion-discriminator for single image dehazing. Proceedings of the Innovative Applications of Artificial Intelligence Conference (AAAI)

  17. Fan DP, Cheng MM, Liu JJ, Gao SH, Hou Q, Borji A (2018) Salient objects in clutter: Bringing salient object detection to the foreground. In: Proceedings of the european conference on computer vision (ECCV), pp 186–202

  18. Fan DP, Wang W, Cheng MM, Shen J (2019) Shifting more attention to video salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8554–8564

  19. Fattal R (2014) Dehazing using color-lines. ACM Trans Graphics (TOG) 34(1):1–14

    Article  Google Scholar 

  20. Fu K, Zhao Q, Gu IYH, Yang J (2019) Deepside: A general deep framework for salient object detection. Neurocomputing 356:69–82

    Article  Google Scholar 

  21. Galdran A (2018) Image dehazing by artificial multiple-exposure image fusion. Signal Process 149:135–147

    Article  Google Scholar 

  22. Gao S, Cheng MM, Zhao K, Zhang XY, Yang MH, Torr PH (2019) Res2net: A new multi-scale backbone architecture. IEEE Trans Pattern Anal Mach Intell

  23. Guan B, Yao J, Zhang G, Wang X (2019) Thigh fracture detection using deep learning method based on new dilated convolutional feature pyramid network. Pattern Recogn Lett 125:521–526

    Article  Google Scholar 

  24. He K, Sun J, Tang X (2010) Single image haze removal using dark channel prior. IEEE Trans Pattern Anal Mach Intell 33(12):2341–2353

    Google Scholar 

  25. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 770–778

  26. Hernandez-Beltran JE, Diaz-Ramirez VH, Trujillo L, Legrand P (2019) Design of estimators for restoration of images degraded by haze using genetic programming. Swarm Evolution Comput 44:49–63

    Article  Google Scholar 

  27. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 7132–7141

  28. Isola P, Zhu JY, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1125–1134

  29. Kim JH, Jang WD, Sim JY, Kim CS (2013) Optimized contrast enhancement for real-time image and video dehazing. J Vis Commun Image Represent 24(3):410–425

    Article  Google Scholar 

  30. Li B, Peng X, Wang Z, Xu J, Feng D (2017) Aod-net: All-in-one dehazing network. In: Proceedings of the IEEE international conference on computer vision (ICCV), pp 4770–4778

  31. Li B, Ren W, Fu D, Tao D, Feng D, Zeng W, Wang Z (2018a) Benchmarking single-image dehazing and beyond. IEEE Trans Image Process 28(1):492–505

    Article  MathSciNet  Google Scholar 

  32. Li J, Li G, Fan H (2018b) Image dehazing using residual-based deep cnn. IEEE Access 6:26831–26842

    Article  Google Scholar 

  33. Ling Z, Fan G, Gong J, Guo S (2019) Learning deep transmission network for efficient image dehazing. Multimed Tools Appl 78(1):213–236

    Article  Google Scholar 

  34. Liu R, Fan X, Hou M, Jiang Z, Luo Z, Zhang L (2018) Learning aggregated transmission propagation networks for haze removal and beyond. IEEE Trans Neural Netw Learn Syst 30(10):2973–2986

    Article  Google Scholar 

  35. Liu X, Ma Y, Shi Z, Chen J (2019a) Griddehazenet: Attention-based multi-scale network for image dehazing. In: Proceedings of the IEEE international conference on computer vision (ICCV), pp 7314–7323

  36. Liu Y, Pan J, Ren J, Su Z (2019b) Learning deep priors for image dehazing. In: Proceedings of the IEEE international conference on computer vision (ICCV), pp 2492–2500

  37. Liu Y, Shang J, Pan L, Wang A, Wang M (2019c) A unified variational model for single image dehazing. IEEE Access 7:15722–15736

    Article  Google Scholar 

  38. Luo Y, Qin J, Xiang X, Tan Y, Liu Q, Xiang L (2020) Coverless real-time image information hiding based on image block matching and dense convolutional network. J Real-Time Image Proc 17(1):125–135

    Article  Google Scholar 

  39. Qian R, Tan RT, Yang W, Su J, Liu J (2018) Attentive generative adversarial network for raindrop removal from a single image. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 2482–2491

  40. Qin X, Wang Z, Bai Y, Xie X, Jia H (2020) Ffa-net: Feature fusion attention network for single image dehazing. In: AAAI, pp 11908–11915

  41. Qu Y, Chen Y, Huang J, Xie Y (2019) Enhanced pix2pix dehazing network. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 8160–8168

  42. Ren W, Ma L, Zhang J, Pan J, Cao X, Liu W, Yang MH (2018) Gated fusion network for single image dehazing. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 3253–3261

  43. Tan RT (2008) Visibility in bad weather from a single image. In: 2008 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 1–8

  44. Tao X, Gao H, Shen X, Wang J, Jia J (2018) Scale-recurrent network for deep image deblurring. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 8174–8182

  45. Wang JB, He N, Zhang LL, Lu K (2015) Single image dehazing with a physical model and dark channel prior. Neurocomputing 149:718–728

    Article  Google Scholar 

  46. Wang W, Zhao S, Shen J, Hoi SC, Borji A (2019) Salient object detection with pyramid attention and salient edges. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1448–1457

  47. Xu H, Guo J, Liu Q, Ye L (2012) Fast image dehazing using improved dark channel prior. In: 2012 IEEE international conference on information science and technology. IEEE, pp 663–667

  48. Yang D, Sun J (2018) Proximal dehaze-net: A prior learning-based deep network for single image dehazing. In: Proceedings of the european conference on computer vision (ECCV), pp 702–717

  49. Yin S, Wang Y, Yang YH (2019) A novel residual dense pyramid network for image dehazing. Entropy 21(11):1123

    Article  MathSciNet  Google Scholar 

  50. Yin S, Wang Y, Yang Y H (2020) A novel image-dehazing network with a parallel attention block. Pattern Recogn 102:107255

    Article  Google Scholar 

  51. Yu F, Liu L, He B, Huang Y, Shi C, Cai S, Song Y, Du S, Wan Q (2019a) Analysis and fpga realization of a novel 5d hyperchaotic four-wing memristive system, active control synchronization, and secure communication application. Complexity 2019

  52. Yu F, Liu L, Xiao L, Li K, Cai S (2019b) A robust and fixed-time zeroing neural dynamics for computing time-variant nonlinear equation using a novel nonlinear activation function. Neurocomputing 350:108–116

    Article  Google Scholar 

  53. Yu T, Song K, Miao P, Yang G, Yang H, Chen C (2019c) Nighttime single image dehazing via pixel-wise alpha blending. IEEE Access 7:114619–114630

    Article  Google Scholar 

  54. Zhang H, Patel VM (2018) Densely connected pyramid dehazing network. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 3194–3203

  55. Zhang H, Sindagi V, Patel VM (2018) Multi-scale single image dehazing using perceptual pyramid deep network. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops (CVPRW), pp 902–911

  56. Zhang J, Xie Z, Sun J, Zou X, Wang J (2020) A cascaded r-cnn with multiscale attention and imbalanced samples for traffic sign detection. IEEE Access 8:29742–29754

    Article  Google Scholar 

  57. Zhao JX, Liu JJ, Fan DP, Cao Y, Yang J, Cheng MM (2019) Egnet: Edge guidance network for salient object detection. In: Proceedings of the IEEE international conference on computer vision, pp 8779–8788

  58. Zhu H, Peng X, Chandrasekhar V, Li L, Lim JH (2018) Dehazegan: When image dehazing meets differential programming. In: IJCAI, pp 1234–1240

  59. Zhu Q, Mai J, Shao L (2015) A fast single image haze removal algorithm using color attenuation prior. IEEE Trans Image Process 24(11):3522–3533

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This research was funded by National Natural Science Foundation of China: 61502396; the Education Department Foundation of Sichuan Province: 18ZB0484; and NSERC, Canada.

Author information

Author notes

  1. Yibin Wang, Shibai Yin & Anup Basu

    Present address: Department of Computing Science, University of Alberta, Edmonton, AB T6G 2E8, Canada

Authors and Affiliations

  1. Department of Engineering, Sichuan Normal University, Chengdu, 610066, China

    Yibin Wang

  2. Department of Economic Information Engineering, Southwestern University of Finance and Economics, Chengdu, 611130, China

    Shibai Yin

Authors
  1. Yibin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shibai Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anup Basu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All the authors conceived of the idea for the paper. Y.W. and S.Y. implemented the algorithm, performed the experiments and wrote the manuscript. A.B. provided technical guidance and revised the paper.

Corresponding author

Correspondence to Yibin Wang.

Ethics declarations

Conflict of Interests

The authors declare no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Y., Yin, S. & Basu, A. A multi-scale attentive recurrent network for image dehazing. Multimed Tools Appl 80, 32539–32565 (2021). https://doi.org/10.1007/s11042-021-11209-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11209-z

Keywords

Search

Navigation