Skip to main content

Advertisement

Springer Nature Link
Log in

Edge-guided dynamic feature fusion network for object detection under foggy conditions

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Hazy images are often subject to blurring, low contrast and other visible quality degradation, making it challenging to solve object detection tasks. Most methods solve the domain shift problem by deep domain adaptive technology, ignoring the inaccurate object classification and localization caused by quality degradation. Different from common methods, we present an edge-guided dynamic feature fusion network (EDFFNet), which formulates the edge head as a guide to the localization task. Despite the edge head being straightforward, we demonstrate that it makes the model pay attention to the edge of object instances and improves the generalization and localization ability of the network. Considering the fuzzy details and the multi-scale problem of hazy images, we propose a dynamic fusion feature pyramid network (DF-FPN) to enhance the feature representation ability of the whole model. A unique advantage of DF-FPN is that the contribution to the fused feature map will dynamically adjust with the learning of the network. Extensive experiments verify that EDFFNet achieves 2.4\(\%\)AP and 3.6\(\%\)AP gains over the ATSS baseline on RTTS and Foggy Cityscapes, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data availability

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

Notes

  1. https://github.com/open-mmlab/mmdetection.

References

  1. Dong, B., Zhou, Y., Hu, C., et al.: BCNet: bidirectional collaboration network for edge-guided salient object detection. Neurocomputing 437, 58–71 (2021)

    Article  Google Scholar 

  2. Li, B., Ren, W., Fu, D., et al.: Benchmarking single-image dehazing and beyond. IEEE Trans. Image Process. 28(1), 492–505 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  3. Chen, C., Liu, M., Meng, X., et al.: RefineDetLite: a lightweight one-stage object detection framework for CPU-only devices. CVPR (2020). https://doi.org/10.1109/CVPRW50498.2020.00358

    Article  Google Scholar 

  4. Cheng, C., Yi, H., Yen, Y., Ming, H.: Every pixel matters: center-aware feature alignment for domain adaptive object detector. ECCV 12354, 733–748 (2020)

    Google Scholar 

  5. Feng, C., Zhong, Y., Gao, Y., et al.: Tood: task-aligned one-stage object detection. In: ICCV, pp. 3490–3499 (2021)

  6. Li, C., Guo, C., Guo, J., et al.: Pdr-net: perception-inspired single image dehazing network with refinement. IEEE Trans. Multim. 22(3), 704–716 (2020)

    Article  Google Scholar 

  7. Sakaridis, C., Dai, D., et al.: Semantic foggy scene understanding with synthetic data. Int. J. Comput. Vis. 126(9), 973–992 (2018)

    Article  Google Scholar 

  8. Bolya, D., Foley, S., Hays, J., et al.: TIDE: a general toolbox for identifying object detection errors. ECCV 12348, 558–573 (2020)

    Google Scholar 

  9. Kim, T., Jeong, M., Kim, S., et al.: Diversify and match: a domain adaptive representation learning paradigm for object detection. In: CVPR, pp. 12456–12465 (2019)

  10. Chuong, H., Thuy, C., Tuan, N., et al.: Improving object detection by label assignment distillation. In: IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1322–1331 (2022)

  11. Dong, H., Pan, J., Lei, X., et al.: Multi-scale boosted dehazing network with dense feature fusion. In: CVPR, pp. 2154–2164 (2020)

  12. Mazin, H., Hayder, R.: Multiscale domain adaptive YOLO for cross-domain object detection. In: 2021 IEEE International Conference on Image Processing (ICIP) (2021)

  13. Wu, H., Qu, Y., Lin, S., et al.: Contrastive learning for compact single image dehazing. In: CVPR, pp. 10551–10560 (2021)

  14. Deng, J., Li, W., Chen, Y., Duan, L.: Unbiased mean teacher for cross-domain object detection. In: CVPR, pp. 4091–4101 (2021)

  15. Hu, J., Li, S., Gang, S.: Squeeze-and-Excitation Networks. In: CVPR, pp. 7132–7141 (2018)

  16. Pang, J., Chen, K., Shi, J., et al.: Libra R-CNN: Towards balanced learning for object detection. In: CVPR, pp. 821–830 (2019)

  17. Weickert, J., Scharr, H.: A scheme for coherence-enhancing diffusion filtering with optimized rotation invariance. J. Visual Commun. Image Represent. 13(1–2), 103–118 (2002)

    Article  Google Scholar 

  18. Zhao, J., Liu, J., Fan, D., et al.: EGNet: edge guidance network for salient object detection. In: ICCV, pp. 8778–8787 (2019)

  19. Kim, K., Lee, H.: Probabilistic anchor assignment with iou prediction for object detection. ECCV 12370, 355–371 (2020)

    Google Scholar 

  20. He, Z., Zhang, L.: Multi-adversarial faster-rcnn for unrestricted object detection. In: ICCV, pp. 6667–6676 (2019)

  21. Cordts, M., Omran, M., Ramos, S., et al.: The cityscapes dataset for semantic urban scene understanding. In: CVPR, pp. 3213–3223 (2016)

  22. Tan, M., Pang, R., Quoc, V.: Efficientdet: scalable and efficient object detection. In: CVPR, pp. 10778–10787 (2020)

  23. Cai, Q., Pan, Y., Ngo, C., et al.: Exploring object relation in mean teacher for cross-domain detection. In: CVPR, pp. 11457–11466 (2019)

  24. Chen, Q., Wang, Y., Yang, T., et al.: You only look one-level feature. In: CVPR, pp. 13034–13043 (2021)

  25. Zhao, Q., Sheng, T., Wang, Y., et al.: M2det: a single-shot object detector based on multi-level feature pyramid network. In: AAAI, pp. 9259–9266 (2019)

  26. Li, C., Cong, R., Sam, K., et al.: Asif-net: attention steered interweave fusion network for RGB-D salient object detection. IEEE Trans. Cybern. 51(1), 88–100 (2021)

    Article  Google Scholar 

  27. Huang, S., Le, T., Jaw, D.: DSNet: joint semantic learning for object detection in inclement weather conditions. IEEE Trans. Pattern Anal. Mach. Intell. 43, 1–1 (2020)

    Article  Google Scholar 

  28. Liu, S., Qi, L., Qin, H., et al.: Path aggregation network for instance segmentation. In: CVPR, pp. 8759–8768 (2018)

  29. Ramprasaath, S., Michael, C., et al.: Grad-CAM: Visual explanations from deep networks via gradient-based localization. In: ICCV, pp. 618–626 (2017)

  30. Ren, S., He, K., Girshick, R., et al.: Faster r-cnn: towards real-time object detection with region proposal networks. Adv. Neural Inf. Process. Syst., pp. 91–99 (2015)

  31. Zhang, S., Chi, C., Yao, Y., et al.: Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection. In: CVPR, pp. 9759–9768 (2020)

  32. Zhang, S., Tuo, H., Hu, J., et al.: Domain adaptive YOLO for one-stage cross-domain detection. Asian Conf. Mach. Learn. ACML 157, 785–797 (2021)

    Google Scholar 

  33. Kong, T., Sun, F., Liu, H., et al.: FoveaBox: beyound anchor-based object detection. IEEE Trans. Image Process. 29, 7389–7398 (2020)

    Article  MATH  Google Scholar 

  34. Lin, T., Dollar, P., Girshick, R., et al.: Feature pyramid networks for object detection. In: CVPR (2017)

  35. Liu, W., Ren, G., Yu, R., et al.: Image-adaptive YOLO for object detection in adverse weather conditions. In: AAAI, pp. 1792–1800 (2022)

  36. Wang, W., Zhao, S., Shen, J., et al.: Salient object detection with pyramid attention and salient edges. In: CVPR, pp. 1448–1457 (2019)

  37. Chen, C., Zheng, Z., Ding, X., et al.: Harmonizing transferability and discriminability for adapting object detectors. In: CVPR, pp. 8866–8875 (2020)

  38. Li, X., Wang, W., Wu, L., et al.: Generalized focal loss: learning qualified and distributed bounding boxes for dense object detection. Adv. Neural Inf. Process. Syst. 33, 21002–12 (2020)

    Google Scholar 

  39. Ai, Y., Guo, J., Wang, Y.: Elunet: an efficient and lightweight u-shape network for real-time semantic segmentation. J. Electron. Imaging 31(2), 023019 (2022)

    Article  Google Scholar 

  40. Chen, Y., Li, W., et al.: Domain adaptive faster r-cnn for object detection in the wild. In: CVPR, pp. 3339–3348 (2018)

  41. Li, Y., Chen, Y., Wang, N., et al.: Scale-aware trident networks for object detection. In: ICCV, pp. 6053–6062 (2019)

  42. Pei, Y., Huang, Y., Zou, Q., et al.: Effects of image degradation and degradation removal to cnn-based image classification. IEEE Trans. Pattern Anal. Mach. Intell. 43(4), 1239–1253 (2021)

    Article  Google Scholar 

  43. Chen, Z., Wang, Y., Yang, Y., et al.: PSD: principled synthetic-to-real dehazing guided by physical priors. In: CVPR, pp. 7176–7185 (2021)

  44. Tian, Z., Shen, C., Chen, H., et al.: Fcos: Fully convolutional one-stage object detection. In: ICCV, pp. 9627–9636 (2019)

Download references

Funding

This work is supported by the National Natural Science Foundation of China (Grant No.62171315) and Tianjin Research Innovation Project for Postgraduate Students (No.2021YJSB153).

Author information

Authors and Affiliations

  1. School of Electrical and Information Engineering, Tianjin University, Tianjin, 300072, China

    Wanru He, Jichang Guo, Yudong Wang & Sida Zheng

Authors
  1. Wanru He
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Jichang Guo
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Yudong Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Sida Zheng
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

WH and YW provided the ideas, formulated the overall research goals and supported the implementation of the code. WH completed the experiment and wrote the first draft. JG oversaw and led the execution of research activities and contributed the required computational resources. SZ: maintained required equipment and proofread tables and figures. All authors reviewed and revised the manuscript.

Corresponding author

Correspondence to Jichang Guo.

Ethics declarations

Conflict of interest

I declare that the authors have no competing interests as defined by Springer, or other interests that might be perceived to influence the results and/or discussion reported in this paper.

Ethical approval

This declaration is “not applicable.”

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

He, W., Guo, J., Wang, Y. et al. Edge-guided dynamic feature fusion network for object detection under foggy conditions. SIViP 17, 1975–1983 (2023). https://doi.org/10.1007/s11760-022-02410-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-022-02410-0

Keywords