Skip to main content
SpringerLink
Log in

MED-YOLOv8s: a new real-time road crack, pothole, and patch detection model

  • Research
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Real-time road damage detection and assessment is crucial to ensure road safety. Traditional road damage detection methods mostly rely on manual labor, which is not only inefficient, but it is also difficult to guarantee its reliability. In this study, a road damage detection model, MED-YOLOv8s, based on YOLOv8s is proposed. MobileNetv3 is adopted as the backbone of the detection algorithm, which reduces the number of parameters and the number of computations in the process of feature extraction, enabling the model to achieve a good balance between the detection speed and the detection accuracy. The introduction of the ultralightweight attention mechanism, ECA, adapts the optimization of the correlation of channels to improve the model generalization performance. In addition, replacing the standard convolution with the DW convolution in the 21st layer of the network not only eliminates part of the redundant feature maps but also better extracts the correlation information between the feature maps. In this study, we also discuss the influence of the mix-up data augmentation weight parameter on the detection effect of the model. The experimental results show that the mAP@0.5 of the MED-YOLOv8s model proposed in this study is 95.2%, which is 1.1% higher than that of the original model, and at the same time, the calculation amount of the model is reduced by 46.2%. This method not only improves the detection accuracy but also greatly reduces the model complexity, providing a reference for subsequent model migration.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Data availability

Data will be made available on request.

References

  1. Hou, Y., Li, Q., Zhang, C., et al.: The state-of-the-art review on applications of intrusive sensing, image processing techniques, and machine learning methods in pavement monitoring and analysis. Engineering 7(6), 845–856 (2021)

    Article  Google Scholar 

  2. Pais, J.C., Amorim, S.I.R., Minhoto, M.J.C.: Impact of traffic overload on road pavement performance. J. Transp. Eng. 139(9), 873–879 (2013)

    Article  Google Scholar 

  3. Madli, R., Hebbar, S., Pattar, P., et al.: Automatic detection and notification of potholes and humps on roads to aid drivers. IEEE Sens. J. 15(8), 4313–4318 (2015)

    Article  Google Scholar 

  4. Gao, Y., Cao, H., CAI, W., et al.: Pixel-level road crack detection in UAV remote sensing images based on ARD-Unet. Measurement 219 (2023)

  5. Rojo, M., Gonzalo-Orden, H., Linares, A., et al.: Impact of a lower conservation budget on road safety indices. J. Adv. Transp. 2018, 1–9 (2018)

    Article  Google Scholar 

  6. Pan, Y., Zhang, X., Tian, J., et al.: Mapping asphalt pavement aging and condition using multiple endmember spectral mixture analysis in Beijing, China. J. Appl. Remote Sens. 11(1) (2017)

  7. Zalama, E., Gómez-García-Bermejo, J., Medina, R., et al.: Road crack detection using visual features extracted by Gabor filters. Comput.-Aid. Civ. Infrastruct. Eng. 29(5), 342–358 (2014)

    Article  Google Scholar 

  8. Laurent, J., Hébert, J.F., Lefebvre, D., et al.: Using 3D laser profiling sensors for the automated measurement of road surface conditions. Rilem Bookser. 4, 157–167 (2012)

    Article  Google Scholar 

  9. Gopalakrishnan, K.: Deep learning in data-driven pavement image analysis and automated distress detection: a review. Data 3(3) (2018)

  10. Quan, Y., Sun, J., Zhang, Y. et al.: The method of the road surface crack detection by the improved Otsu threshold. In: Proceedings of the 2019 IEEE International Conference on Mechatronics and Automation (ICMA), 2019

  11. Dan, D., Dan, Q.: Automatic recognition of surface cracks in bridges based on 2D-APES and mobile machine vision. Measurement 168 (2021)

  12. Wang, W., Li, L., Han, Y.: Crack detection in shadowed images on gray level deviations in a moving window and distance deviations between connected components. Constr. Build. Mater. 271 (2021)

  13. Zhao, H., Qin, G., Wang, X.: Improvement of canny algorithm based on pavement edge detection. In: Proceedings of the 2010 3rd International Congress on Image and Signal Processing, 2010. IEEE

  14. Hanzaei, S.H., Afshar, A., Barazandeh, F.: Automatic detection and classification of the ceramic tiles’ surface defects. Pattern Recognit. 66, 174–189 (2017)

    Article  Google Scholar 

  15. Li, P., Xia, H., Zhou, B., et al.: A method to improve the accuracy of pavement crack identification by combining a semantic segmentation and edge detection model. Appl. Sci. 12(9) (2022)

  16. Prasad, A., Kumar, M., Choudhury, D.R.: Color image encoding using fractional Fourier transformation associated with wavelet transformation. Opt. Commun.Commun. 285(6), 1005–1009 (2012)

    Article  Google Scholar 

  17. Sharma, K.K., Sharma, M.: Image fusion based on image decomposition using self-fractional Fourier functions. SIViP 8(7), 1335–1344 (2012)

    Article  Google Scholar 

  18. Yae, S., Ikehara, M.: Inverted residual Fourier transformation for lightweight single image deblurring. IEEE Access. 11, 29175–29182 (2023)

    Article  Google Scholar 

  19. Lecun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)

    Article  Google Scholar 

  20. Zhao, Z.Q., Zheng, P., Xu, S.T., et al. Object detection with deep learning: a review. IEEE Trans. Neural Netw. Learn. Syst. 3212–3232 (2019)

  21. Girshick, R., Donahue, J., Darrell, T., et al.: Rich feature hierarchies for accurate object detection and semantic segmentation. IEEE Conf. Comput. Vis. Pattern Recognit. 2014, 580–587 (2014)

    Google Scholar 

  22. Girshick, R.: Fast R-CNN. In: Proceedings of the International Conference on Computer Vision, 2015

  23. Ren, S., He, K., Girshick, R., et al.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell.Intell. 39(6), 1137–1149 (2017)

    Article  Google Scholar 

  24. Lin, T.-Y., Dollar, P., Girshick, R., et al. Feature pyramid networks for object detection. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 936–944 (2017)

  25. He, K., Gkioxari, G., Dollár, P., et al.: Mask R-CNN. IEEE Trans. Pattern Anal. Mach. Intell. (2017)

  26. Redmon, J., Divvala, S., Girshick, R., et al.: You only look once: unified, real-time object detection. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) 2016, 779–788 (2016)

    Google Scholar 

  27. Redmon, J., Farhadi, A.: YOLO9000: better, faster, stronger. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) 2017, 6517–6525 (2017)

    Google Scholar 

  28. Redmon, J., Farhadi, A.: YOLOv3: an incremental improvement. arXiv e-prints (2018)

  29. Bochkovskiy, A., Wang, C.Y., Liao, H.Y.M.: YOLOv4: optimal speed and accuracy of object detection. (2020)

  30. Li, C., Li, L., Jiang, H., et al.: YOLOv6: a single-stage object detection framework for industrial applications. arXiv preprint http://arxiv.org/abs/220902976 (2022)

  31. Wang, C.-Y., Bochkovskiy, A., Liao, H.-Y.M.: YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. In: Proceedings of the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023

  32. Roy, A.M., Bhaduri, J.: DenseSPH-YOLOv5: an automated damage detection model based on DenseNet and Swin-Transformer prediction head-enabled YOLOv5 with attention mechanism. Adv. Eng. Inform. 56 (2023)

  33. Wang, W., Wu, B., Yang, S., et al.: Road damage detection and classification with faster R-CNN. In: Proceedings of the 2018 IEEE International Conference on Big Data (Big data), 2018. IEEE

  34. Chen, Q., Gan, X., Huang, W., et al.: Road damage detection and classification using mask R-CNN with DenseNet backbone. Comput. Mater. Continua 65(3), 2201–2215 (2020)

    Article  Google Scholar 

  35. Haciefendioğlu, K., Başağa, H.B.: Concrete road crack detection using deep learning-based faster R-CNN method. Iran. J. Sci. Technol. Trans. Civ. Eng. 1–13 (2022)

  36. Liu, Z., Yeoh, J.K.W., Gu, X., et al.: Automatic pixel-level detection of vertical cracks in asphalt pavement based on GPR investigation and improved mask R-CNN. Autom. Constr. 146 (2023)

  37. Shen, T., Nie, M.: Pavement damage detection based on cascade R-CNN. In: Proceedings of the Proceedings of the 4th International Conference on Computer Science and Application Engineering, 2020

  38. Li, S., Huang, Y.: Damage detection algorithm based on faster-RCNN. In: Proceedings of the 2023 5th International Conference on Electronics and Communication, Network and Computer Technology (ECNCT), 2023. IEEE

  39. Ding, W., Zhao, X., Zhu, B., et al.: An ensemble of one-stage and two-stage detectors approach for road damage detection. In: Proceedings of the 2022 IEEE International Conference on Big Data (Big Data), 2022 [C]. IEEE

  40. Tran, T.S., Nguyen, S.D., Lee, H.J., et al.: Advanced crack detection and segmentation on bridge decks using deep learning. Constr. Build. Mater. 400, 132839 (2023)

    Article  Google Scholar 

  41. Sami, A.A., Sakib, S., Deb, K., et al.: Improved YOLOv5-based real-time road pavement damage detection in road infrastructure management. Algorithms 16(9), 452 (2023)

    Article  Google Scholar 

  42. Wang, X., Gao, H., Jia, Z., et al.: BL-YOLOv8: an improved road defect detection model based on YOLOv8. Sensors 23(20), 8361 (2023)

    Article  Google Scholar 

  43. Alfarrarjeh, A., Trivedi, D., Kim, S.H., et al.: A deep learning approach for road damage detection from smartphone images. In: Proceedings of the 2018 IEEE International Conference on Big Data (Big Data), 2018. IEEE

  44. Guo, G., Zhang, Z.: Road damage detection algorithm for improved YOLOv5. Sci. Rep. 12(1), 15523 (2022)

    Article  Google Scholar 

  45. Inam, H., Islam, N.U., Akram, M.U., et al.: Smart and automated infrastructure management: a deep learning approach for crack detection in bridge images. Sustainability 15(3) (2023)

  46. Ren, M., Zhang, X., Chen, X., et al.: YOLOv5s-M: a deep learning network model for road pavement damage detection from urban street-view imagery. Int. J. Appl. Earth Observ. Geoinf. 120 (2023)

  47. Du, Y., Zhong, S., Fang, H., et al.: Modeling automatic pavement crack object detection and pixel-level segmentation. Autom. Constr. 150 (2023)

  48. Arya, D., Maeda, H., Ghosh, S.K., et al.: Crowdsensing-based road damage detection challenge (CRDDC’2022). In: Proceedings of the 2022 IEEE International Conference on Big Data (Big Data), 2022. IEEE

  49. Zhang, H., Cisse, M., Dauphin, Y.N., et al.: mixup: beyond empirical risk minimization. arXiv preprint http://arxiv.org/abs/171009412 (2017)

  50. Terven, J., Cordova-Esparza, D.: A comprehensive review of YOLO: from YOLOv1 and beyond. arXiv 2023. arXiv preprint http://arxiv.org/abs/230400501

  51. Koonce, B., Koonce, B.: MobileNetV3. In: Convolutional Neural Networks with Swift for Tensorflow: Image Recognition and Dataset Categorization, pp. 125–44 (2021)

  52. Tan, M., Le, Q.: Efficientnetv2: smaller models and faster training. In: Proceedings of the International Conference on Machine Learning, 2021. PMLR

  53. Koonce, B., Koonce, B.: EfficientNet. In: Convolutional Neural Networks with Swift for Tensorflow: Image Recognition and Dataset Categorization, pp. 109–23 (2021)

  54. Howard, A.G., Zhu, M., Chen, B., et al.: Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv preprint http://arxiv.org/abs/170404861 (2017)

  55. Sandler, M., Howard, A., Zhu, M., et al.: Mobilenetv2: inverted residuals and linear bottlenecks. In: Proceedings of the Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018

  56. Chollet, F.: Xception: deep learning with depthwise separable convolutions. In: Proceedings of the Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017

  57. Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018

  58. Wang, Q., Wu, B., Zhu, P., et al.: ECA-Net: efficient channel attention for deep convolutional neural networks. In: Proceedings of the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020

  59. Sun, P., Zhang, R., Jiang, Y., et al.: Sparse R-CNN: end-to-end object detection with learnable proposals. In: Proceedings of the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021

  60. Wang, G., Chen, Y., An, P., et al.: UAV-YOLOv8: a small-object-detection model based on improved YOLOv8 for UAV aerial photography scenarios. Sensors 23(16), 7190 (2023)

    Article  Google Scholar 

  61. Zheng, X., Qian, S., Wei, S., et al.: The combination of transformer and you only look once for automatic concrete pavement crack detection. Appl. Sci. 13(16), 9211 (2023)

    Article  Google Scholar 

  62. Wu, Y., Han, Q., Jin, Q., et al.: LCA-YOLOv8-Seg: an improved lightweight YOLOv8-Seg for real-time pixel-level crack detection of dams and bridges. Appl. Sci. 13(19), 10583 (2023)

    Article  Google Scholar 

  63. Yang, L., Yan, J., Li, H., et al.: Real-time classification of invasive plant seeds based on improved YOLOv5 with attention mechanism. Diversity 14(4), 254 (2022)

    Article  Google Scholar 

  64. Huang, Y., He, J., Liu, G., et al.: YOLO-EP: a detection algorithm to detect eggs of Pomacea canaliculata in rice fields. Ecol. Inform. 77, 102211 (2023)

    Article  Google Scholar 

Download references

Acknowledgements

This study was partly supported by the XIAN Youth Talent Support Program (Grant no. 959202313010). The authors are responsible for all views and opinions expressed in this paper

Author information

Authors and Affiliations

  1. School of Science, Xi’an Polytechnic University, Xi’an, 710048, China

    Minghu Zhao, Yaoheng Su, Jiuxin Wang, Kaihang Wang, Zishen Liu, Man Liu & Zhou Guo

  2. School of Electronic Information, Xi’an Polytechnic University, Xi’an, 710048, China

    Xinru Liu

Authors
  1. Minghu Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yaoheng Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiuxin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xinru Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kaihang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zishen Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Man Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Zhou Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

MZ: Methodology, Software, Formal analysis, Investigation. YS: Supervision, Writing – original draft, Writing – review & editing, Funding acquisition. JW: Conceptualization, Methodology, Validation, Supervision. XL: Conceptualization, Methodology, Validation, Supervision. KW: Methodology, Validation, Supervision. ZL: Methodology, Validation, Supervision. ML: Validation, Resources, Supervision, Writing – review & editing. ZG: Validation, Resources, Supervision, Writing review & editing.

Corresponding author

Correspondence to Yaoheng Su.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, M., Su, Y., Wang, J. et al. MED-YOLOv8s: a new real-time road crack, pothole, and patch detection model. J Real-Time Image Proc 21, 26 (2024). https://doi.org/10.1007/s11554-023-01405-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11554-023-01405-5

Keywords

Search

Navigation