Abstract
In this research, we explore the problems of pothole detection, segmentation, and area estimation using deep neural networks and unmanned aerial vehicles (drones). We start by compiling two datasets, one that contains ground-level and aerial images of potholes, and another that only contains ground-level images, and we train a total of six deep neural network models for pothole detection; we do this to determine whether aerial images are necessary for training UAV-based object detection models. We then determine which pothole detection model is the most accurate and we also determine which combinations of camera angle and UAV altitude are best for detecting potholes. Furthermore, we take the strongest pothole segmentation model and apply it to area estimation using a combination of homography, the intrinsic and extrinsic parameters of the UAV camera, and novel methods. Our method for pothole area estimation using YOLOv8 has an average area estimation error of 9.71%.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abbas, S.A., Zisserman, A.: A geometric approach to obtain a bird’s eye view from an image. In: 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), pp. 4095–4104 (2019). https://doi.org/10.1109/ICCVW.2019.00504
Asad, M.H., Khaliq, S., Yousaf, M.H., Ullah, M.O., Ahmad, A.: Pothole detection using deep learning: a real-time and AI-on-the-edge perspective. Adv. Civil Eng. 2022, 1–13 (2022)
SB, B.K., Guhan, S., Kishore, M., Santhosh, R.: Real-time pothole detection using YOLOv5 algorithm: a feasible approach for intelligent transportation systems. In: 2023 Second International Conference on Electronics and Renewable Systems (ICEARS), pp. 1678–1683 (2023)
De Zoysa, K., Keppitiyagama, C., Seneviratne, G.P., Shihan, W.: A public transport system based sensor network for road surface condition monitoring. In: Proceedings of the 2007 Workshop on Networked Systems for Developing Regions, pp. 1–6 (2007)
Diamantas, S., Alexis, K.: Modeling pixel intensities with log-normal distributions for background subtraction. In: IEEE International Conference on Imaging Systems and Techniques, Beijing, China, pp. 1–6 (2017)
Diamantas, S., Alexis, K.: Optical flow based background subtraction with a moving camera: application to autonomous driving. In: Bebis, G., et al. (eds.) ISVC 2020. LNCS, vol. 12510, pp. 398–409. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-64559-5_31
Eriksson, J., Girod, L., Hull, B., Newton, R., Madden, S., Balakrishnan, H.: The pothole patrol: using a mobile sensor network for road surface monitoring. In: Proceedings of the 6th International Conference on Mobile Systems, Applications, and Services, pp. 29–39 (2008)
Eriksson, L.H..S.: An Investigation of detecting potholes with UAV LiDAR and UAV Photogrammetry. Ph.D. thesis, University of Gavle (2021)
Ge, Z., Liu, S., Wang, F., Li, Z., Sun, J.: Yolox: Exceeding yolo series in 2021 (2021)
He, K., Gkioxari, G., Dollar, P., Girshick, R.: Mask R-CNN. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2980–2988 (2017). https://doi.org/10.1109/ICCV.2017.322
Jocher, G., Chaurasia, A., Qiu, J.: YOLO by Ultralytics (2023). https://github.com/ultralytics/ultralytics
Kang, B.H., Choi, S.i.: Pothole detection system using 2d lidar and camera. In: 2017 Ninth International Conference on Ubiquitous and Future Networks (ICUFN), pp. 744–746 (2017)
Kharel, S., Ahmed, K.R.: Potholes detection using deep learning and area estimation using image processing. In: Intelligent Systems with Applications (2021)
Lin, T., et al.: Microsoft COCO: common objects in context. CoRR abs/1405.0312 (2014). http://arxiv.org/abs/1405.0312
Peralta-Lopez, J.E., et al.: Speed bump and pothole detection using deep neural network with images captured through zed camera. Appl. Sci. 8349, 1–17 (2023)
Prasad, V., Kumari, S.: Pothole detection using lidar. Adv. Automob. Eng. 10, 1–3 (2021)
Shaghouri, A.A., Alkhatib, R., Berjaoui, S.: Real-time pothole detection using deep learning, pp. 1–10 (2021). https://arxiv.org/abs/2107.06356
Silvister, S., et al.: Deep learning approach to detect potholes in real-time using smartphone. In: 2019 IEEE Pune Section International Conference (PuneCon), pp. 1–4 (2019)
Welborn, E.A.: Detecting Potholes Using Deep Neural Networks with Unmanned Aerial Vehicles. Ph.D. thesis, Tarleton State University (2023)
Wu, C., et al.: An automated machine-learning approach for road pothole detection using smartphone sensor data. Sensors 20, 1–23 (2020)
Yik, Y.K., Alias, N.E., Yusof, Y., Isaak, S.: A real-time pothole detection based on deep learning approach. J. Phys.: Conf. Ser., 1–7 (2021)
Yu, B.X., Yu, X.: Vibration-based system for pavement condition evaluation. In: Applications of Advanced Technology in Transportation, pp. 183–189 (2006)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Welborn, E., Diamantas, S. (2023). Pothole Segmentation and Area Estimation with Deep Neural Networks and Unmanned Aerial Vehicles. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2023. Lecture Notes in Computer Science, vol 14362. Springer, Cham. https://doi.org/10.1007/978-3-031-47966-3_29
Download citation
DOI: https://doi.org/10.1007/978-3-031-47966-3_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-47965-6
Online ISBN: 978-3-031-47966-3
eBook Packages: Computer ScienceComputer Science (R0)