Skip to main content
SpringerLink
Log in

A localization method for stagnant water in city road traffic image

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Stagnant water on roads has always been a major cause of traffic jams and accidents. Traditional urban waterlogging monitoring and warning system is mainly based on a large amount of historical data and predictive network, which has low accuracy and weak generalization ability. Considering the deep neural network algorithms have demonstrated strong capabilities in computer vision tasks such as object detection, we aim to apply them to road stagnant water detection. In this paper, a novel automatic stagnant water localization method under weak supervision based on visual image is proposed. First, the template matching method is applied to extract road information from the traffic image. Then, due to the complexity of data annotation, we locate stagnant water in image based on Class Activation Maps (CAM) mechanism, which is a weakly supervised method. The detection model consists of the ResNet-18 and the Grad-CAM++ mechanism. Finally, based on the heat map and template, we set a suitable threshold to segment stagnant water area in image. In the experiments, the precision and recall for road stagnant water classification by the proposed model are 99.39% and 99.60%, while the Intersection over Union (IoU) for stagnant water area segmentation is up to 63%. These show that our method is effective for road stagnant water localization.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Cao C, Liu X, Yang Y et al (2015) Look and think twice: capturing top-down visual attention with feedback convolutional neural networks. In: IEEE international conference on computer vision (ICCV). Santiago, Chile, pp 2956–2964

  2. Chattopadhyay A, Sarkar A, Howlader P et al (2018) Grad-CAM++: Generalized gradient-based visual explanations for deep convolutional networks. In: IEEE winter conference on applications of computer vision (WACV). Lake Tahoe, NV, USA, pp 839–847

  3. Choudhury AD, Agrawal A, Sinha P et al (2012) A methodology for GPS-based waterlogging prediction and smart route generation. In: International conference on intelligent systems design and applications. Kochi, India, pp 274–278

  4. Fan Q, Zhuo W, Tang CK et al (2020) Few-shot object detection with attention-RPN and multi-relation detector. In: IEEE/CVF conference on computer vision and pattern recognition (CVPR). Seattle, WA, USA, pp 4012–4021

  5. Gupta A, Bansal A, Gupta R et al (2017) Urban waterlogging detection and severity prediction using artificial neural networks. In: IEEE 19th International conference on high performance computing and communications; IEEE 15th international conference on smart city; IEEE 3rd international conference on data science and systems (HPCC/SmartCity/DSS). Bangkok, Thailand, pp 42–49

  6. He K, Zhang X, Ren S et al (2016) Deep residual learning for image recognition. In: IEEE conference on computer vision and pattern recognition (CVPR). Las Vegas, NV, USA, pp 770–778

  7. Huang G, Liu Z, Van Der Maaten L et al (2017) Densely connected convolutional networks. In: IEEE conference on computer vision and pattern recognition (CVPR). Honolulu, HI, USA, pp 2261–2269

  8. Huang K, Meng F, Li H et al (2019) Class activation map generation by multiple level class grouping and orthogonal constraint. In: Perth WA (ed) Digital image computing: techniques and applications (DICTA). Australia, pp 1–6

  9. Janos V, Kriz M (2018) Smart urban transport. In: Smart city symposium prague (SCSP). Czech Republic, Prague, pp 1–5

  10. Jarrett K, Kavukcuoglu K, Ranzato M et al (2009) What is the best multi-stage architecture for object recognition. In: IEEE international conference on computer vision (ICCV). Kyoto, Japan, pp 2146–2153

  11. Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. In: Conference and workshop on neural information processing systems (NIPS). Lake Tahoe, Nevada, USA, pp 1106–1114

  12. Sandler M, Howard A, Zhu M et al (2018) Inverted residuals and linear bottlenecks: mobile networks for classification, detection and segmentation. [Online]. Available: arxiv: 1801.04381

  13. Selvaraju RR, Cogswell M, Das A et al (2020) Grad-CAM: Visual explanations from deep networks via gradient-based localization. International Journal of Computer Vision 128(2):336–359

    Article  Google Scholar 

  14. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: International conference on learning representations. San Diego, CA, USA

  15. Szegedy C, Liu W, Jia Y et al (2015) Going deeper with convolutions. In: IEEE conference on computer vision and pattern recognition (CVPR). MA, USA, Boston, pp 1–9

  16. Szegedy C, Vanhoucke V, Ioffe S et al (2016) Rethinking the inception architecture for computer vision. In: IEEE conference on computer vision and pattern recognition (CVPR). Las Vegas, NV, USA, pp 2818–2826

  17. Tagaris T, Sdraka M, Stafylopatis A (2019) High-resolution class activation mapping. In: IEEE international conference on image processing (ICIP). Taipei, Taiwan, pp 4514–4518

  18. Vaswani A, Shazeer N, Parmar N et al (2017) Attention is all you need. In: Conference and workshop on neural information processing systems (NIPS). Long Beach, CA, USA, pp 5998–6008

  19. Wang Y, Li J, Zhang H (2016) Study on city rainstorm waterlogging warning system based on historical data. In: 13th International computer conference on wavelet active media technology and information processing (ICCWAMTIP). Chengdu, China, pp 171–174

  20. Wang H, Wang Z, Du M et al (2020) Score-CAM: Score-weighted visual explanations for convolutional neural networks. In: IEEE/CVF conference on computer vision and pattern recognition workshops (CVPRW). Seattle, WA, USA, pp 111–119

  21. Xia J, Cao H, Yang Y et al (2019) Detection of waterlogging stress based on hyperspectral images of oilseed rape leaves. Computers and Electronics in Agriculture 159:59–68

    Article  Google Scholar 

  22. Xu M, Wang H, Ni B et al (2020) Cross-domain detection via graph-induced prototype alignment. IEEE/CVF conference on computer vision and pattern recognition (CVPR). Seattle, WA, USA, pp 12352–12361

  23. Yang J, Zhao Z, Zhang H et al (2019) Data augmentation for X-Ray prohibited item images using generative adversarial networks. IEEE Access 7:28894–28902

    Article  Google Scholar 

  24. Zhang J, Bargal SA, Lin Z et al (2018) Top-down neural attention by excitation backprop. International Journal of Computer Vision 126(10):1084–1102

    Article  Google Scholar 

  25. Zhang T, Lin G, Cai J et al (2019) Decoupled spatial neural attention for weakly supervised semantic segmentation. IEEE Transactions on Multimedia 21(11):2930–2941

    Article  Google Scholar 

  26. Zhang Z, Li D, Zeng F et al (2018) A dynamic risk assessment method of waterlogging points by coupling hydrology model with deep neural network. In: 26th International conference on geoinformatics. pp 1–6

  27. Zhang W, Wang X, Liu Y et al (2016) Simulation of rainstorm waterlogging based on SWMM and visualization module research. In: International conference on smart city and systems engineering (ICSCSE). Hunan, China, pp 394–397

  28. Zhou B, Khosla A, Lapedriza A et al (2016) Learning deep features for discriminative localization. In: IEEE conference on computer vision and pattern recognition (CVPR). Las Vegas, NV, USA, pp 2921–2929

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China (Grant No. 61806208, 62076166), in part by the General Higher Education Project of Guangdong Provincial Education Department (Grant No. 2020ZDZX3082), in part by the Guangdong Provincial Rural Science and Technology Specialists Project (Grant No. KPT20200220), in part by the Shenzhen Science and Technology Program (Grand No. RCBS20200714114940262).

Author information

Authors and Affiliations

  1. Institute of Applied Artificial Intelligence of the Guangdong-Hong Kong-Macao Greater Bay Area, Shenzhen Polytechnic, Shenzhen, Guangdong, China

    Zihao Zhao & Haigang Zhang

Authors
  1. Zihao Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haigang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Haigang Zhang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, Z., Zhang, H. A localization method for stagnant water in city road traffic image. Multimed Tools Appl 81, 2453–2466 (2022). https://doi.org/10.1007/s11042-021-11638-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11638-w

Keywords

Search

Navigation