Abstract
Potholes in the road are major causes of car accidents, affecting road conditions, driver safety, and vehicle fuel consumption. The timely and effective detection of potholes is a critical step in developing appropriate strategies for driving maintenance and safety. Detecting potholes on Indian road surfaces under diverse climatic conditions, such as potholes covered by tree shadow and filled with water, remains a challenging task. As a result, this research presents a novel method for detecting potholes on Indian roads. This work is being done in two stages. To begin, the Haar feature cascade classifier ensemble is equipped with a sequence of 9-adaptive boosting (Ada-Boost) layers to locate the potholes. Following that, an 11-layer convolutional neural network (CNN) is designed to predict the potholes. Finally, an instance segmentation algorithm is used to mask and quantify the pothole region. The proposed method is trained and tested on Kaggle, CCSAD, and custom-based pothole datasets. For performance assessment of the proposed method, the evaluation metrics, namely sensitivity, precision, recall, F1 score, and mAP, were adopted. It is found that the proposed model detects potholes with an accuracy of 98.65%. The segmentation of 50 test images showed that the Mean Intersection over Unit (MIoU) pothole rate could be 89.52%.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data Availability
Enquiries about data availability should be directed to the authors.
References
Akagic A, Buza E, Omanovic S (2017) Pothole detection: an efficient vision based method using RGB color space image segmentation. In: 40th international convention on information and communication technology, electronics and microelectronics (MIPRO), (pp. 1104–1109) IEEE
Angulo A, Vega-Fernández JA, Aguilar-Lobo LM, Natraj S, Ochoa-Ruiz G (2019) Road damage detection acquisition system based on deep neural networks for physical asset management. In: Martínez-Villaseñor L, Batyrshin I, Marín-Hernández A (eds) Advances in soft computing. Lecture notes in computer science. Springer, Cham
Bhatia Y, Rai R, Gupta V, Aggarwal N, Akula A (2019) Convolutional neural networks based potholes detection using thermal imaging. J King Saud Univ Comput Inf Sci. https://doi.org/10.1016/j.jksuci.2019.02.004
Cao MT et al (2021) Image processing-based automatic detection of asphalt pavement rutting using a novel metaheuristic optimized machine learning approach. Soft Comput 25(20):12839–12855
Chen H, Yao M, Qinlong Gu (2020) Pothole detection using location-aware convolutional neural networks. Int J Mach Learn Cybern 11(4):899–911
Dhiman A, Klette R (2019) Pothole detection using computer vision and learning. IEEE Trans Intell Transp Syst 21(8):3536–3550
Dutta A and A Zisserman. (2019) The VIA annotation software for images, audio and video. In: proceedings of the 27th ACM international conference on multimedia (MM ’19), October 21–25, 2019, Nice, France. ACM, New York, NY, USA, 4 pages
Guzmán R, JB Hayet, and R Klette (2015) Towards ubiquitous autonomous driving: the CCSAD dataset. In: international conference on computer analysis of images and patterns. Springer, Cham
Mazurek P, and T Hachaj (2019) Robustness of Haar feature-based cascade classifier for face detection under presence of image distortions. In: international conference on image processing and communications. Springer, Cham
Fan R, Ozgunalp U, Hosking B, Liu M, Pitas I (2020) Pothole detection based on disparity transformation and road surface modeling. IEEE Trans Image Process 29:897–908. https://doi.org/10.1109/TIP.2019.2933750
Gangopadhyay I, Chatterjee A, Das I (2019) Face detection and expression recognition using Haar cascade classifier and fisherface algorithm. In: Bhattacharyya S, Pal S, Pan I, Das A (eds) Recent trends in signal and image processing. Advances in intelligent systems and computing. Springer, Singapore
He K et al. (2017) Mask r-cnn. In: proceedings of the IEEE international conference on computer vision
Koch C, Jog GM, Brilakis I (2013) Automated pothole distress assessment using asphalt pavement video data. J Comput Civ Eng 27(4):370–378
Koch C, Georgieva K, Kasireddy V, Akinci B, Fieguth P (2015) A review on computer vision-base defect detection and condition assessment of concrete and asphalt civil infrastructure. Adv Eng Inform 29(2):196–210
Maeda H, Sekimoto Y, Seto T, Kashiyama T, and Omata H (2018) Road damage detection using deep neural networks with images captured through a smartphone. arXiv preprint arXiv:1801.09454
Nienaber S, Marthinus JB, and RS Kroon (2015) Detecting potholes using simple image processing techniques and real-world footage
Oliveira H, Correia PL (2009) Supervised crack detection and classification in images of road pavement flexible surfaces. In: Zaher AA (ed) Recent advances in signal processing. InTech, Rijeka, Croatia
Oliveira H, Correia PL (2013) Automatic road crack detection and characterization. IEEE Trans Intell Transp Syst 14(1):155–168
Patra S, Middya AI, Roy S (2021) PotSpot: participatory sensing based monitoring system for pothole detection using deep learning. Multimed Tools Appl 80(16):25171–25195
Ryu SK, Kim T, Kim YR (2015) Image-based pothole detection system for its service and road management system. Math Probl Eng. https://doi.org/10.1155/2015/968361
Suong LK, Kwon J (2018) Detection of potholes using deep convolutional neural network. J Univ Comput Sci 24(9):1244–1257
Tedeschi A, Benedetto F (2017) A real-time automatic pavement crack and pothole recognition system for mobile android-based devices. Adv Eng Inform 32:11–25
Ukhwah EN, Yuniarno EM and Suprapto YK, (2019) Asphalt pavement pothole detection using deep learning method based on YOLO neural network. In: 2019 international seminar on intelligent technology and its applications (ISITIA), Surabaya, Indonesia, pp. 35-40
Varona B, Monteserin A, Teyseyre A (2019) A deep learning approach to automatic road surface monitoring and pothole detection. Pers Ubiquit Comput. https://doi.org/10.1007/s00779-019-01234-z
Funding
The authors have not disclosed any funding.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors have not disclosed any competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Satti, S.K., Devi, K.S., Dhar, P. et al. Detecting potholes on Indian roads using Haar feature-based cascade classifier, convolutional neural network, and instance segmentation. Soft Comput 26, 9141–9153 (2022). https://doi.org/10.1007/s00500-022-07265-8
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-022-07265-8