Skip to main content
SpringerLink
Log in

Detection and localization of potholes in thermal images using deep neural networks

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

A pothole is a depression caused on roads due to seepage of water into soil structure or weight of continuously moving traffic. This not only damages the suspension of the vehicles but is also a prime reason for road accidents worldwide. This necessitates the need to develop an efficient automatic pothole detection system which can assist concerned authorities for timely repair and maintenance of the roads. This paper proposes a novel approach of bounding box based pothole localization from thermal images using deep neural networks. The modified ResNet34-single shot multibox detector gives an average precision of 74.53% whereas modified ResNet50-RetinaNet model provides 91.15% precision. The results obtained by the proposed modified ResNet50-RetinaNet model are the state-of-the-art results for localization of potholes using thermal images. In real-world scenarios such a system can assist relevant authorities to judge the severity of road damage and take appropriate effective measures accordingly.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. An KE, Lee SW, Ryu SK, Seo D (2018) Detecting a pothole using deep convolutional neural network models for an adaptive shock observing in a vehicle driving. IEEE Int Conf Consum Electron ICCE. https://doi.org/10.1109/ICCE.2018.8326142

  2. Aparna BY, Rai R, Gupta V, Aggarwal N, Akula A (2019) Convolutional neural networks based potholes detection using thermal imaging. J King Saud Univ-Comput Inf Sci 2019:1–11. https://doi.org/10.1016/j.jksuci.2019.02.004

    Article  Google Scholar 

  3. Azhar K, Murtaza F, Yousaf MH, Habib HA (2016) Computer vision based detection and localization of potholes in asphalt pavement images. IEEE Can Conf Electr Comput Eng IEEE 2016:1–5

  4. Bhatt U, Mani S, Xi E, Kolter Z (2017) Intelligent pothole detection and road condition assessment. Data Good Exch 2017:1–7

    Google Scholar 

  5. Dhiman A, Chien HJ, Klette R (2018) A multi-frame stereo vision-based road profiling technique for distress analysis. 15th Int Symp Pervasive Syst Algorithms networks, I-SPAN. IEEE 2018: 7–14. https://doi.org/10.1109/I-SPAN.2018.00012.

  6. Eriksson J, Girod L, Hull B, Newton R, Madden S, Balakrishnan H (2008) The pothole patrol: using a Mobile sensor network for road surface monitoring. Proc 6th Int Conf Mob Syst Appl Serv

  7. Fan R, Liu M (2019) Road damage detection based on unsupervised disparity map segmentation. IEEE Trans Intell Transp Syst 2019:1–6. https://doi.org/10.1109/tits.2019.2947206

    Article  Google Scholar 

  8. Fan R, Ozgunalp U, Hosking B, Liu M, Pitas I (2020) Pothole detection based on disparity transformation and road surface modeling. IEEE Trans Image Process 29:897–908. https://doi.org/10.1109/TIP.2019.2933750

    Article  MathSciNet  Google Scholar 

  9. Fan D-P, Ji G-P, Sun G, Cheng M-M, Shen J, Shao L (2020) Camouflaged object detection. Conf Comput Vis Pattern Recognit (CVPR)

  10. Gayathri S, Menita P, Mamatha RG, Manasa B, Sanjana BM (2019) Automatic pothole detection system. Int J Eng Res Technol 7:1–5

    Google Scholar 

  11. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. IEEE Conf Comput Vis Pattern Recog 2016:770–778. https://doi.org/10.1246/cl.2003.428

    Article  Google Scholar 

  12. Hou Z, Wang KCP, Gong W (2007) Experimentation of 3D pavement imaging through stereovision. Int Conf Transp Eng 2007:376–381. https://doi.org/10.1061/40932(246)62

    Article  Google Scholar 

  13. Jo Y, Ryu S (2015) Pothole detection system using a black-box camera. Sensors 15:29316–29331. https://doi.org/10.3390/s151129316

    Article  Google Scholar 

  14. Kim T, Ryu SK (2014) Review and analysis of pothole detection methods. J Emerg Trends Comput Inf Sci 5:603–608

    Google Scholar 

  15. Koch C, Brilakis I (2011) Pothole detection in asphalt pavement images. Adv Eng Inform 25:507–515. https://doi.org/10.1016/j.aei.2011.01.002

    Article  Google Scholar 

  16. Kotha M, Chadalavada M, Karuturi SH, Venkataraman H (2020) PotSense - pothole detection on Indian roads using smartphone sensors. ACM Int Conf Proc Ser https://doi.org/10.1145/3377283.3377286.

  17. Le TN, Nguyen TV, Nie Z, Tran MT, Sugimoto A (2019) Anabranch network for camouflaged object segmentation. Comput Vis Image Underst 184:45–56. https://doi.org/10.1016/j.cviu.2019.04.006

    Article  Google Scholar 

  18. Li Q, Yao M, Yao X, Xu B (2009) A real-time 3D scanning system for pavement distortion inspection. Meas Sci Technol 21:15702–15709. https://doi.org/10.1088/0957-0233/21/1/015702

    Article  Google Scholar 

  19. Lin T-Y, Goyal P, Girshick R, He K, Dollar P (2017) Focal loss for dense object detection. Proc IEEE Int Conf Comput Vis 2017:2999–3007. https://doi.org/10.1109/ICCV.2017.324

    Article  Google Scholar 

  20. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg AC (2016) SSD: single shot multibox detector. ArXiv: 151202325v5[CsCV] 2016. https://doi.org/10.1007/978-3-319-46448-0_2.

  21. Mednis A, Strazdins G, Zviedris R, Kanonirs G (2011) Real time pothole detection using android smartphones with accelerometers. 2011 Int Conf Distrib Comput Sens Syst Work DCOSS’11

  22. Moazzam I, Kamal K, Mathavan S, Usman S, Rahman M (2013) Metrology and visualization of potholes using the microsoft kinect sensor. IEEE Conf Intell Transp Syst Proc, ITSC 2013:1284–1291. https://doi.org/10.1109/ITSC.2013.6728408

    Article  Google Scholar 

  23. Redmon J, Farhadi A (2017) YOLO9000: better, faster, stronger. 30th IEEE Conf Comput Vis Pattern Recog CVPR 2017: 6517–25. https://doi.org/10.1109/CVPR.2017.690.

  24. Sharma SK, Sharma RC (2019) Pothole detection and warning system for Indian roads. Adv Interdiscip Eng Springer Singapore 2019:15–26. https://doi.org/10.1007/978-981-13-6577-5.

  25. Suong LK, Jangwoo K (2018) Detection of potholes using a deep convolutional neural network. J Univ Comput Sci 24:1244–1257

    Google Scholar 

  26. Times of India Report (2018) Potholes killed 3,597 across India in 2017, terror 803. https://timesofindia.indiatimes.com/india/potholes-killed-3597-across-india-in-2017-terror-803/articleshow/64992956.cms. Accessed 15 May 2020

  27. Turkowski K (1990) Filters for common resampling tasks. Graph Gems 1990:147–165. https://doi.org/10.1016/B978-0-08-050753-8.50042-5

    Article  Google Scholar 

  28. Wang HW, Chen CH, Cheng DY, Lin CH, Lo CC (2015) A real-time pothole detection approach for intelligent transportation system. Math Probl Eng 2015:1–7. https://doi.org/10.1155/2015/869627

    Article  Google Scholar 

  29. Wilhelm B, Burge MJ (2009) Principles of digital image processing: Core algorithms. Springer, London

    MATH  Google Scholar 

Download references

Acknowledgements

We are thankful to the Design Innovation Centre, Panjab University Chandigarh (INDIA) for providing us with the dataset for the proposed research work.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Chandigarh College of Engineering and Technology (Degree Wing), Chandigarh, India

    Saksham Gupta, Paras Sharma, Dakshraj Sharma & Varun Gupta

  2. Department of Computer Science and Engineering, Punjab Engineering College (Deemed to be University), Chandigarh, India

    Nitigya Sambyal

Authors
  1. Saksham Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paras Sharma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dakshraj Sharma
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Varun Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nitigya Sambyal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Varun Gupta.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gupta, S., Sharma, P., Sharma, D. et al. Detection and localization of potholes in thermal images using deep neural networks. Multimed Tools Appl 79, 26265–26284 (2020). https://doi.org/10.1007/s11042-020-09293-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09293-8

Keywords

Search

Navigation