Skip to main content
SpringerLink
Log in

Attention-based adaptive feature selection for multi-stage image dehazing

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Removing haze, especially non-homogeneous and in various concentrations, is quite challenging. Existing dehazing methods are usually used to deal with homogeneous haze or just to deal with non-homogeneous haze. Few methods can be used in dealing with differently distributed haze. To address this problem, we propose an attention-based adaptive feature selection for multi-stage image dehazing network (ASNet). Rich and effective detail features are extracted by adaptive selection and aggregation, which in turn leads to better recovery of haze images with different concentrations and non-uniform distribution. Specifically, our model first learns different scale contextual feature information using an encoder–decoder architecture and then fuses the original resolution imaged features with spatial information through convolution and cascade operations. We introduce a new adaptive feature selection module, which selects features at different stages through different attention mechanisms to extract more effective image features. For ASNet, the exchange of information between stages is crucial. Therefore, we designed the self-calibration attention module between the two stages to supervise the features under the guidance of ground-truth, which can effectively avoid the loss and redundancy of information while realizing the information transfer between the stages. This inter-stage modular design enhances the connectivity of the modules, reduces the network burden, and improves the processing power of the network. Extensive experiments have shown that the proposed ASNet outperforms advanced methods in both uniform and non-uniform haze datasets with excellent dehazing and satisfactory visual effects.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. McCartney, E.J.: Optics of the Atmosphere: Scattering by Molecules and Particles. Wiley, New York (1976)

  2. Narasimhan, S.G., Nayar, S.K.: Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No. PR00662), pp. 598–605. Publisher. IEEE (2000)

  3. He, K., Sun, J., Tang, X.: Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 33, 2341–2353 (2010)

    Google Scholar 

  4. Zhu, Q., Mai, J., Shao, L.: BMVC (2014)

  5. Ngo, D., Lee, S., Kang, B.: Robust single-image haze removal using optimal transmission map and adaptive atmospheric light. Remote Sens. 12, 2233 (2020)

    Article  Google Scholar 

  6. Wu, Q., Zhang, J., Ren, W., Zuo, W., Cao, X.: Accurate transmission estimation for removing haze and noise from a single image. IEEE Trans. Image Process. 29, 2583–2597 (2019)

    Article  MATH  Google Scholar 

  7. Chen, W.-T., Fang, H.-Y., Ding, J.-J., Kuo, S.-Y.: PMHLD: patch map-based hybrid learning DehazeNet for single image haze removal. IEEE Trans. Image Process. 29, 6773–6788 (2020)

    Article  MATH  Google Scholar 

  8. Yang, D., Sun, J.: Proceedings of the European Conference on Computer Vision (ECCV), pp. 702–717 (2018)

  9. Tangsakul, S., Wongthanavasu, S.: Article single image haze removal using deep cellular automata learning. IEEE Access 8, 103181–103199 (2020)

    Article  Google Scholar 

  10. Li, B., Zhao, J., Fu, H.: DLT-Net: deep learning transmittance network for single image haze removal. SIViP 14, 1245–1253 (2020)

    Article  Google Scholar 

  11. Cai, B., Xu, X., Jia, K., Qing, C., Tao, D.: Dehazenet: an end-to-end system for single image haze removal. IEEE Trans. Image Process. 25, 5187–5198 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  12. Wang, C., Li, Z., Wu, J., Fan, H., Xiao, G., Zhang, H.: Deep residual haze network for image dehazing and deraining. IEEE Access 8, 9488–9500 (2020)

    Article  Google Scholar 

  13. Xiao, J., Shen, M., Lei, J., Zhou, J., Klette, R., Sui, H.G.: Single image dehazing based on learning of haze layers. Neurocomputing 389, 108–122 (2020)

    Article  Google Scholar 

  14. Shao, Y., Li, L., Ren, W., Gao, C., Sang, N.: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2808–2817 (2020)

  15. Li, J., Li, G., Fan, H.: Image dehazing using residual-based deep CNN. IEEE Access 6, 26831–26842 (2018)

    Article  Google Scholar 

  16. Qin, X.,Wang, Z., Bai, Y., Xie, X., Jia, H.: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 11908–11915 (2020)

  17. Ma, Y.: GridDehazeNet: attention-based multi-scale network for image dehazing (2019)

  18. Chen, D., He, M., Fan, Q., Liao, J., Zhang, L.,Hou, D., Yuan, L., Hua, G.: 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1375–1383. IEEE (2019)

  19. Kar, A., Dhara, S.K., Sen, D., Biswas, P.K.: Transmission map, atmospheric light guided iterative updater network for single image dehazing (2020). arXiv preprint arXiv:2008.01701

  20. Hua, Z., Fan, G., Li, J.: Iterative residual network for image dehazing. IEEE Access 8, 167693–167710 (2020)

    Article  Google Scholar 

  21. Dong, H., Pan, J., Xiang, L., Hu, Z., Zhang, X., Wang, F., Yang, M.-H.: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2157–2167 (2020)

  22. Ren, W., Liu, S., Zhang, H., Pan, J., Cao, X., Yang, M.-H.: European Conference on Computer Vision, pp. 154–169. Springer (2016)

  23. Li, B., Peng, X., Wang, Z., Xu, J., Feng, D.: An all-in-one network for dehazing and beyond (2017). arXiv preprint arXiv:1707.06543

  24. Zhang, H., Patel, V.M.: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3194–3203 (2018)

  25. Li, R., Pan, J., Li, Z., Tang, J.: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8202–8211 (2018)

  26. Zhang, H., Sindagi, V., Patel, V.: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 902–911 (2018)

  27. Mei, K., Jiang, A., Li, J., Wang, M.: Progressive Feature Fusion Network for Realistic Image Dehazing, pp. 203–215. Springer, Berlin (2018)

  28. Yan, J., Li, C., Zheng, Y., Xu, S., Yan, X.: MMP-net: a multi-scale feature multiple parallel fusion network for single image haze removal. IEEE Access 8, 25431–25441 (2020)

    Article  Google Scholar 

  29. Dudhane, A., Murala, S.: RYF-Net: deep fusion network for single image haze removal. IEEE Trans. Image Process. 29, 628–640 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  30. Anvari, Z., Athitsos, V.: Dehazed-GLCGAN: unpaired single image de-hazing via adversarial training (2020). arXiv preprint arXiv:2008.06632

  31. Li, J., Feng, X., Hua, Z.: Low-light image enhancement via progressive-recursive network. In: IEEE Transactions on Circuits and Systems for Video Technology (2021)

  32. Jiang, K., Wang, Z., Yi, P., Chen, C., Huang, B., Luo, Y., Ma, J., Jiang, J.: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8346–8355 (2020)

  33. Cheng, B., Chen, L.-C., Wei, Y., Zhu, Y., Huang, Z., Xiong, J., Huang, T.S., Hwu, W.-M., Shi, H.: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5218–5228 (2019)

  34. Li, S.-J., AbuFarha, Y., Liu, Y., Cheng, M.-M., Gall, J.: MS-TCN++: multi-stage temporal convolutional network for action segmentation. IEEE Trans. Pattern Anal. Mach. Intell. (2020). https://doi.org/10.1109/TPAMI.2020.3021756

    Article  Google Scholar 

  35. Ghosh, P., Yao, Y., Davis, L., Divakaran, A.: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 576–585 (2020)

  36. Hu, D.: Proceedings of SAI Intelligent Systems Conference, pp. 432–448. Springer (2019)

  37. Yeh, C.-H., Huang, C.-H., Kang, L.-W.: Multi-scale deep residual learning-based single image haze removal via image decomposition, pp. 3153–3167. IEEE (2019)

  38. Huang, S., Li, D., Zhao, W., Liu, Y.: Haze removal algorithm for optical remote sensing image based on multi-scale model and histogram characteristic. IEEE Access 7, 104179–104196 (2019)

    Article  Google Scholar 

  39. Ronneberger, O., Fischer, P., Brox, T.: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234–241. Springer (2015)

  40. Huang, X., Zhang, B., Zhang, X., Tang, M., Miao, Q., Li, T., Jia, G.: Application of U-Net based multiparameter magnetic resonance image fusion in the diagnosis of prostate cancer. IEEE Access 9, 33756–33768 (2021)

  41. Shi, J., Wu, K., Yang, C., Deng, N.: A method of steel bar image segmentation based on multi-attention U-Net. IEEE Access 9, 13304–13313 (2021)

  42. Zhang, J., Zhu, H., Wang, P., Ling, X.: ATT squeeze U-Net: a lightweight network for forest fire detection and recognition. IEEE Access 9, 10858–10870 (2021)

    Article  Google Scholar 

  43. Chu, C., Zheng, J., Zhou, Y.: Ultrasonic thyroid nodule detection method based on U-Net network. Comput. Methods Progr. Biomed. 199, 105906 (2021)

    Article  Google Scholar 

  44. Hu, J., Shen, L., Sun, G.: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2018)

  45. Woo, S., Park, J., Lee, J.-Y., Kweon, I.S.: Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–19 (2018)

  46. Qilong, W., Banggu, W., Pengfei, Z., Peihua, L., Wangmeng, Z., Qinghua, H.: ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks (2020)

  47. Zhang, H., Dai, Y., Li, H., Koniusz, P.: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5978–5986 (2019)

  48. Suin, M., Purohit, K., Rajagopalan, A.N.: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3606–3615 (2020)

  49. Li, B., Ren, W., Fu, D., Tao, D., Feng, D., Zeng, W., Wang, Z.: Benchmarking single-image dehazing and beyond. IEEE Trans. Image Process. 28, 492–505 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  50. Zhang, Y., Song, S., Yumer, E., Savva, M., Lee, J.-Y., Jin, H., Funkhouser, T.: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5287–5295 (2017)

  51. Ancuti, C.O., Ancuti, C., Timofte, R., De Vleeschouwer, C.: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 754–762 (2018)

  52. Ancuti, C., Ancuti, C.O., Timofte, R., De Vleeschouwer, C.: International Conference on Advanced Concepts for Intelligent Vision Systems, pp. 620–631. Springer (2018)

  53. Ancuti, C.O., Ancuti, C., Timofte, R.: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 444–445 (2020)

  54. Ancuti, C.O., Ancuti, C., Vasluianu, F.-A., Timofte, R.: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 490–491 (2020)

  55. Berman, D., Avidan, S., et al.: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1674–1682 (2016)

  56. Das, S.D., Dutta, S.: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 482–483 (2020)

  57. Ancuti, C.O., Ancuti, C., Vasluianu, F.-A., Timofte, R.: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 490–491 (2020)

  58. Wu, H., Liu, J., Xie, Y., Qu, Y., Ma, L.: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 478–479 (2020)

  59. Yu, M., Cherukuri, V., Guo, T., Monga, V.: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 450–451 (2020)

Download references

Funding

The authors acknowledge the National Natural Science Foundation of China (Grant Nos. 61772319, 62002200, 61976125, 61976124, 12001327) and Shandong Natural Science Foundation of China (Grant No. ZR2020QF012).

Author information

Authors and Affiliations

  1. School of Information and Electronic Engineering, Shandong Technology and Business University, Institute of Network Technology, ICT(YANTAI), Yantai, China

    Xiaoling Li

  2. School of Information and electronic engineering, Shandong Technology and Business University, Yantai, China

    Zhen Hua

  3. School of Computer Science and Technology, Shandong Technology and Business University, Yantai, China

    Jinjiang Li

Authors
  1. Xiaoling Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhen Hua
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jinjiang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Ethics declarations

Conflicts of interest

The authors declare no conflict of interest. The funding sponsors had no role in the study.

Human and animals participants

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, X., Hua, Z. & Li, J. Attention-based adaptive feature selection for multi-stage image dehazing. Vis Comput 39, 663–678 (2023). https://doi.org/10.1007/s00371-021-02365-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-021-02365-2

Keywords

Search

Navigation