Skip to main content
Springer Nature Link
Log in

Yolo and RetinaNet Ensemble Transfer Learning Detector: Application in Pavement Distress

  • Conference paper
  • First Online:
Computing, Communication and Learning (CoCoLe 2023)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1892))

Included in the following conference series:

  • 155 Accesses

Abstract

The significance of using roads for transportation has always been important and it plays a major role in the economy of a country. However, with the increase in population and urbanization, the number of vehicles on the roads as well as the length of roads have significantly increased. This has led to issues such as cracks and potholes caused by heavy rainfall and road construction materials which pose a serious risk to road users. Therefore, it is crucial to detect and maintain these defects. To address this issue, we developed a new method that uses ensemble transfer learning to identify road damages automatically. To improve the dataset, we increased the number of images and balanced the classes by augmenting the dataset resulting in 263,360 images across eight different categories. In addition, we improved the picture quality using various image processing techniques such as sharpening, histogram equalization, grey scaling, and smoothening. We trained and validated two pre-trained deep learning models over the dataset and combined them using an ensemble approach to create a final model. Our proposed model achieved an F1 score of 0.927, which suggests that it could serve as a benchmark for road damage detection.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Road Accidents in India, Ministry of Road Transport and Highways, Transport Research Wing, Govt. of India. https://morth.nic.in/sites/default/files/Road_Accidednt.pdf (2018)

  2. Hatmoko, J., Setiadji, B., Wibowo, M.: Investigating causal factors of road damage: a case study. MATEC Web Conf. 258, 02007 (2019). https://doi.org/10.1051/matecconf/201925802007

    Article  Google Scholar 

  3. Shin, H., Roth, H.R., Gao, M., Lu, L., Xu, Z., Nogues, I., et al.: Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans. Med. Imaging 35, 1285–1289 (2016)

    Article  Google Scholar 

  4. Kermany, D.S., et al.: Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell 172(5), 1122–1131.e9 (2018)

    Article  Google Scholar 

  5. Hegadi, R.: Image Processing: Research Opportunities and Challenges (2010)

    Google Scholar 

  6. Gonzalez, Rafael Digital image processing. New York, NY: Pearson. ISBN 978–0–13–335672–4. OCLC 966609831(2018)

    Google Scholar 

  7. Yuchuan, D., Pan, N., Zihao, X., Fuwen Deng, Y., Shen, H.K.: Pavement distress detection and classification based on YOLO network. Int. J. Pavement Eng. 22(13), 1659–1672 (2020). https://doi.org/10.1080/10298436.2020.1714047

    Article  Google Scholar 

  8. Majidifard, H., Jin, P., Adu-Gyamfi, Y., Buttlar, W.G.: Pavement image datasets: a new benchmark dataset to classify and densify pavement distresses. Transp. Res. Rec. 2674, 328–339 (2020). https://doi.org/10.1177/0361198120907283

    Article  Google Scholar 

  9. Patra, S., Middya, A.I., Roy, S.: PotSpot: participatory sensing based monitoring system for pothole detection using deep learning. Multimedia Tools Appl. 80(16), 25171–25195 (2021). https://doi.org/10.1007/s11042-021-10874-4

    Article  Google Scholar 

  10. Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning, MIT press (2016). http://www.deeplearningbook.org

  11. Maeda, H., Sekimoto, Y., Seto, T., Kashiyama, T., Omata, H.: Road damage detection and classification using deep neural networks with smartphone images. Comput. Aided Civ. Infrastruct. Eng. 33, 1127–1141 (2018). https://doi.org/10.1111/mice.12387

    Article  Google Scholar 

  12. Alfarrarjeh, A., Trivedi, D., Kim, S.H., Shahabi, C.: A deep learning approach for road damage detection from smartphone images, In: 2018 IEEE International Conference on Big Data (Big Data), IEEE, pp. 5201–5204 (2018). https://doi.org/10.1109/BigData.2018.8621899

  13. Kluger, F., et al.: Region-based cycle-consistent data augmentation for object detection, In: 2018 IEEE International Conference on Big Data (Big Data), IEEE, pp. 5205–5211 (2018). https://doi.org/10.1109/BigData.2018.8622318

  14. Wang, Y.J., Ding, M., Kan, S., Zhang, S., Lu, C.: Deep proposal and detection networks for road damage detection and classification. In: 2018 IEEE International Conference on Big Data (Big Data), IEEE, pp. 5224–5227 (2018). https://doi.org/10.1109/BigData.2018.8622599

  15. Wang, W., Wu, B., Yang, S., Wang, Z.: Road damage detection and classification with faster R-CNN. In: 2018 IEEE International Conference on Big Data (Big Data), IEEE, pp. 5220–5223 (2018). https://doi.org/10.1109/BigData.2018.8622354

  16. Angulo, A., Vega-Fernández, J.A., Aguilar-Lobo, L.M., Natraj, S., Ochoa-Ruiz, G.: Road damage detection acquisition system based on deep neural networks for physical asset management. In: Martínez-Villaseñor, L., Batyrshin, I., Marín-Hernández, A. (eds.) MICAI 2019. LNCS (LNAI), vol. 11835, pp. 3–14. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33749-0_1

    Chapter  Google Scholar 

  17. Roberts, R., Giancontieri, G., Inzerillo, L., Di Mino, G.: Towards low-cost pavement condition health monitoring and analysis using deep learning. Appl. Sci. 10, 319 (2020). https://doi.org/10.3390/app10010319

    Article  Google Scholar 

  18. Biçici, S., Zeybek, M.: An approach for the automated extraction of road surface distress from a UAV-derived point cloud. Autom. Constr. 122, 103475 (2021). https://doi.org/10.1016/j.autcon.2020.103475

    Article  Google Scholar 

  19. Zhang, A., et al.: Automated pixel-level pavement crack detection on 3D asphalt surfaces using a deep-learning network. Comput. Aided Civ. Infrastruct. Eng. 32, 805–819 (2017). https://doi.org/10.1111/mice.12297

    Article  Google Scholar 

  20. Zhang, L., Yang, F., Zhang, Y.D., Zhu, Y.J.: Road crack detection using deep convolutional neural network. In: 2016 IEEE International Conference on Image Processing (ICIP). IEEE, pp. 3708–3712 (2016). https://doi.org/10.1109/ICIP.2016.7533052

  21. Silva, W.R.L.d., Lucena, D.S.d.: Concrete cracks detection based on deep learning image classification. In: Multidisciplinary Digital Publishing Institute Proceedings, vol. 2, p. 489 (2018). https://doi.org/10.3390/ICEM18-05387

  22. Anand, S., Gupta, S., Darbari, V., Kohli, S.: Crack-pot: autonomous road crack and pothole detection. In: 2018 Digital Image Computing: Techniques and Applications (DICTA), IEEE, pp. 1–6 (2018).https://doi.org/10.1109/DICTA.2018.8615819

  23. Fan, Z., Wu, Y., Lu, J., Li, W.: Automatic pavement crack detection based on structured prediction with the convolutional neural network, arXiv preprint arXiv:1802.02208 (2018)

  24. Zhu, J., Zhong, J., Ma, T, Huang, X., Zhang, W., Zhou, Y.: Pavement distress detection using convolutional neural networks with images captured via UAV, Autom. Constr. 133, 103991 (2022). https://doi.org/10.1016/j.autcon.2021.103991

  25. Zhang, C., Nateghinia, E., Miranda-Moreno, L.F., Sun, L.: Pavement distress detection using convolutional neural network : a case study in Montreal, Canada, Int. J. Transportation Sci. Technol. 11(2), 298–309 (2022)https://doi.org/10.1016/j.ijtst.2021.04.008

  26. Guerrieri, M., Parla, G.: Flexible and stone pavements distress detection and measurement by deep learning and low-cost detection devices. Eng. Fail. Anal. 141, 106714 (2022). https://doi.org/10.1016/j.engfailanal.2022.106714

    Article  Google Scholar 

  27. Wen, T., et al.: Automated pavement distress segmentation on asphalt surfaces using a deep learning network. Int. J. Pavement Eng. 24(2), 2027414 (2022). https://doi.org/10.1080/10298436.2022.2027414

    Article  Google Scholar 

  28. Oliveira, H., Correia, P.L.: CrackIT—an image processing toolbox for crack detection and characterization. In: 2014 IEEE International Conference on Image Processing (ICIP), IEEE, pp. 798–802 (2014). https://doi.org/10.1109/ICIP.2014.7025160

  29. Zou, Q., Cao, Y., Li, Q., Mao, Q., Wang, S.: CrackTree: automatic crack detection from pavement images. Pattern Recog. Lett. 33(3), 227–238 (2012). https://doi.org/10.1016/j.patrec.2011.11.004

    Article  Google Scholar 

  30. Dorafshan, S., Thomas, R.J., Maguire, M.: SDNET2018: an annotated image dataset for non-contact concrete crack detection using deep convolutional neural networks. Data Brief 21, 1664–1668 (2018). https://doi.org/10.1016/j.dib.2018.11.015

    Article  Google Scholar 

  31. Yang, F., Zhang, L., Yu, S., Prokhorov, D., Mei, X., Ling, H.: Feature pyramid and hierarchical boosting network for pavement crack detection. IEEE Trans. Intell. Transp. Syst. 21, 1525–1535 (2020). https://doi.org/10.1109/TITS.2019.2910595

    Article  Google Scholar 

  32. Eisenbach, M., et al.: How to get pavement distress detection ready for deep learning? a systematic approach. In: International Joint Conference on Neural Networks (IJCNN), pp. 2039–2047 (2017). https://doi.org/10.1109/IJCNN.2017.7966101

  33. Stricker, R., Eisenbach, M., Sesselmann, M., Debes, K., Gross, H.M.: Improving visual road condition assessment by extensive experiments on the extended gaps dataset. In: International Joint Conference on Neural Networks (IJCNN), pp. 1–8 (2019). https://doi.org/10.1109/IJCNN.2019.8852257

  34. Arya, D. Maeda, H., Ghosh, S.K., Toshniwal, D., Mraz, A., Sekimoto, Y.: Deep learning-based road damage detection and classification for multiple countries, Autom. Construct. 132, 103935 (2021). ISSN 0926–5805.https://doi.org/10.1016/j.autcon.2021.103935

  35. Roboflow: Give Your Software the Power to See Objects in Images and Video. https://roboflow.com

  36. Wang, C.Y., Bochkovskiy, A., Liao, H.Y.: YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. https://doi.org/10.48550/arXiv.2207.02696 (2022)

  37. Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal Loss for Dense Object Detection. In: IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, 2, pp. 318–327 (2020). https://doi.org/10.1109/TPAMI.2018.2858826

  38. https://www.who.int/data/gho/data/themes/topics/topic-details/GHO/road-traffic-mortality#:~:text=Road%20traffic%20injuries%20are%20currently,safety%20in%20a%20holistic%20manner

  39. Mei, Q., Gül, M.: A cost effective solution for pavement crack inspection using cameras and deep neural networks. Constr. Build. Mater. 256, 119397 (2020). https://doi.org/10.1016/j.conbuildmat.2020.119397

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dr B R Ambedkar National Institute of Technology Jalandhar, Barnala, Punjab, India

    Ravi Khatri

  2. National Institute of Technology Kurukshetra, Thanesar, India

    Kuldeep Kumar

Authors
  1. Ravi Khatri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kuldeep Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ravi Khatri .

Editor information

Editors and Affiliations

  1. National Institute of Technology, Warangal, India

    Sanjaya Kumar Panda

  2. National Institute of Technology, Warangal, India

    Rashmi Ranjan Rout

  3. National Institute of Technology, Warangal, India

    Manjubala Bisi

  4. National Institute of Technology, Warangal, India

    Ravi Chandra Sadam

  5. Providence University, Taichung City, Taiwan

    Kuan-Ching Li

  6. University of Milan, Milan, Italy

    Vincenzo Piuri

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Khatri, R., Kumar, K. (2024). Yolo and RetinaNet Ensemble Transfer Learning Detector: Application in Pavement Distress. In: Panda, S.K., Rout, R.R., Bisi, M., Sadam, R.C., Li, KC., Piuri, V. (eds) Computing, Communication and Learning. CoCoLe 2023. Communications in Computer and Information Science, vol 1892. Springer, Cham. https://doi.org/10.1007/978-3-031-56998-2_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-56998-2_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-56997-5

  • Online ISBN: 978-3-031-56998-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics