Skip to main content
Springer Nature Link
Log in

Deep learning-based visual crack detection using Google Street View images

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In this study, the utility of using Google Street View (GSV) for evaluating the quality of pavement is investigated. A convolutional neural network (CNN) is developed to perform image classification on GSV pavement images. Pavement images are extracted from GSV and then divided into smaller image patches to form data sets. Each image patch is visually classified into different categories of pavement cracks based on the standard practice. A comparative study of pavement quality assessment is conducted between the results of the CNN classified image patches obtained from GSV and those from a sophisticated commercial visual inspection company. The result of the comparison indicates the feasibility and effectiveness of using GSV images for pavement evaluation. The trained network is then tested on a new data set. This study shows that the designed CNN helps classify the pavement images into different defined crack categories.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

6. References

  1. Roads. American Society Civil Engineers (ASCE) (2017) Infrastruct Rep Card n.d. https://www.infrastructurereportcard.org/cat-item/roads/. Accessed 3 Aug 2018

  2. Phares BM, Rolander DD, Graybeal BA, Washer GA (2001) Reliability of visual bridge inspection turner-fairbank highway research center. Federal Highway Administration 64(5):22–29

    Google Scholar 

  3. Lenz H, Weichers B (2008) Applications of specialized visual inspection techniques on nuclear components

  4. Ye XW, Dong CZ, Liu T (2016) A review of machine vision-based structural health monitoring: methodologies and applications. J Sensors 2016:7103039. https://doi.org/10.1155/2016/7103039

    Article  Google Scholar 

  5. Xu Y, Brownjohn JM (2018) Review of machine-vision based methodologies for displacement measurement in civil structures. J Civ Struct Heal Monit 8(1):91–110

    Article  Google Scholar 

  6. Jahanshahi MR, Masri SF, Sukhatme GS (2011) Multi-image stitching and scene reconstruction for evaluating defect evolution in structures. Struct Health Monit 10(6):643–657

    Article  Google Scholar 

  7. Lee BJ, Shin DH, Seo JW, Jung JD, Lee JY (2011, June) Intelligent bridge inspection using remote controlled robot and image processing technique. In: International Symposium on Automation and Robotics in Construction (ISARC), Seoul, Korea, pp 1426–1431

  8. Lattanzi D, Miller G (2017) Review of robotic infrastructure inspection systems. J Infrastruct Syst 23(3):04017004

    Article  Google Scholar 

  9. Dorafshan S, Maguire M (2018) Bridge inspection: Human performance, unmanned aerial systems and automation. J Civ Struct Heal Monit 8(3):443–476

    Article  Google Scholar 

  10. Cafiso S, Graziano AD, Battiato S (2006, October) Evaluation of pavement surface distress using digital image collection and analysis. In: Seventh international congress on advances in civil engineering, pp 1–10

  11. Gavilán M, Balcones D, Marcos O, Llorca DF, Sotelo MA, Parra I, Ocaña M, Aliseda YP, Amírola A (2011) Adaptive road crack detection system by pavement classification. Sensors 11(10):9628–9657

    Article  Google Scholar 

  12. Lopes G, Ribeiro AF, Sillero N, Gonçalves-Seco L, Silva C, Franch M, Trigueiros P (2016) High resolution trichromatic road surface scanning with a line scan camera and light emitting diode lighting for road-kill detection. Sensors 16(4):558

    Article  Google Scholar 

  13. Zhang L, Yang F, Zhang YD, Zhu YJ (2016, September) Road crack detection using deep convolutional neural network. In: 2016 IEEE international conference on image processing (ICIP), pp 3708–3712

  14. Maeda H, Sekimoto Y, Seto T, Kashiyama T, Omata H (2018) Road damage detection using deep neural networks with images captured through a smartphone. arXiv preprint arXiv:1801.09454

  15. Varadharajan S, Jose S, Sharma K, Wander L, Mertz C (2014, March) Vision for road inspection. In: IEEE winter conference on applications of computer vision, pp 115–122

  16. Lee S, Ha J, Zokhirova M, Moon H, Lee J (2018) Background information of deep learning for structural engineering. Archives of Computational Methods in Engineering 25(1):121–129

    Article  MathSciNet  Google Scholar 

  17. Liu R, Yang B, Zio E, Chen X (2018) Artificial intelligence for fault diagnosis of rotating machinery: a review. Mech Syst Signal Process 108:33–47

    Article  Google Scholar 

  18. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  Google Scholar 

  19. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

  20. Deng J, Dong W, Socher R, Li L, Li K, Fei-Fei L (2009, June) Imagenet: a large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition, pp 248–255

  21. Cha YJ, Choi W, Büyüköztürk O (2017) Deep learning-based crack damage detection using convolutional neural networks. Computer-Aided Civil and Infrastructure Engineering 32(5):361–378

    Article  Google Scholar 

  22. Cha YJ, Choi W, Suh G, Mahmoudkhani S, Büyüköztürk O (2018) Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types. Computer-Aided Civil and Infrastructure Engineering 33(9):731–747

    Article  Google Scholar 

  23. Huang HW, Li QT, Zhang DM (2018) Deep learning based image recognition for crack and leakage defects of metro shield tunnel. Tunneling and Underground Space Technology 77:166–176

    Article  Google Scholar 

  24. Chen FC, Jahanshahi MR (2017) NB-CNN: Deep learning-based crack detection using convolutional neural network and Naïve Bayes data fusion. IEEE Trans Industr Electron 65(5):4392–4400

    Article  Google Scholar 

  25. Zhang A, Wang KC, Li B, Yang E, Dai X, Peng Y, Fei Y, Liu Y, Li JQ, Chen C (2017) Automated pixel-level pavement crack detection on 3D asphalt surfaces using a deep-learning network. Computer-Aided Civil and Infrastructure Engineering 32(10):805–819

    Article  Google Scholar 

  26. Zhang L, Yang F, Zhang YD, Zhu YJ (2016) Road crack detection using deep convolutional neural network. In: 2016 IEEE international conference on image processing (ICIP), pp 3708–3712

  27. Eisenbach M, Stricker R, Debes K, Gross HM (2017) Crack detection with an interactive and adaptive video inspection system. Arbeitsgruppentagung Infrastrukturmanagement, pp 94–103

  28. Pauly L, Hogg D, Fuentes R, Peel H (2017, July) Deeper networks for pavement crack detection. In: Proceedings of the 34th ISARC, pp 479–485

  29. PEER Hub ImageNet n.d. http://apps.peer.berkeley.edu/spo/.

  30. Chacra DBA, Zelek JS (2017) Fully automated road defect detection using street view images. In: 14th Conference on Computer and Robot Vision (CRV), Edmonton, AB, Canada, pp 353–360. https://doi.org/10.1109/CRV.2017.50

  31. Zhang M, Liu Y, Luo S, Gao S (2020) Research on baidu street view road crack information extraction based on deep learning method. J Phys: Conf Ser 1616:12086. https://doi.org/10.1088/1742-6596/1616/1/012086

    Article  Google Scholar 

  32. Lei X, Liu X, Li L, Wang G (2020) Automated pavement distress detection and deterioration analysis using street view map. IEEE Access. https://doi.org/10.1109/ACCESS.2020.2989028

    Article  Google Scholar 

  33. Shapiro A (2018) Street-level: google street view’s abstraction by datafication. New Media Soc 20(3):1201–1219

    Article  Google Scholar 

  34. Rundle AG, Bader MD, Richards CA, Neckerman KM, Teitler JO (2011) Using google street view to audit neighborhood environments. Am J Prev Med 40(1):94–100

    Article  Google Scholar 

  35. Torii A, Havlena M, Pajdla T (2009, September) From google street view to 3d city models. In: 2009 IEEE 12th international conference on computer vision workshops, ICCV Workshops, pp 2188–2195

  36. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  37. LeCun Y, Boser BE, Denker JS, Henderson D, Howard RE, Hubbard WE, Jackel LD (1990) Handwritten digit recognition with a back-propagation network. In: Advances in neural information processing systems, pp 396–404

  38. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv:1412.6980

  39. Reddi SJ, Kale S, Kumar S (2019) On the convergence of adam and beyond. arXiv preprint arXiv:1904.09237. Bengio Y. practical recommendations for gradient-based training of deep architectures. Neural Netw. Tricks Trade. Springer 2012:437–478

    Google Scholar 

  40. Masters D and Luschi C (2018) Revisiting small batch training for deep neural networks. arXiv:1804.07612

  41. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958. http://jmlr.org/papers/v15/srivastava14a.html

    MathSciNet  MATH  Google Scholar 

  42. Canziani A, Paszke A, Culurciello E (2016) An analysis of deep neural network models for practical applications. arXiv:1605.07678

  43. Xiao J, Quan L (2009, October) Multiple view semantic segmentation for street view images. In: 2009 IEEE 12th international conference on computer vision, pp 686–693

  44. Jae Lee Y, Efros AA, Hebert M (2013) Style-aware mid-level representation for discovering visual connections in space and time. In: Proceedings of the IEEE international conference on computer vision, pp 1857–1864

  45. Goodfellow IJ, Bulatov Y, Ibarz J, Arnoud S, Shet V (2013) Multi-digit number recognition from street view imagery using deep convolutional neural networks. arXiv:1312.6082

  46. Zamir AR, Shah M (2010) Accurate image localization based on google maps street view. In: European conference on computer vision. Springer, Berlin, Heidelberg

  47. ARRB Group Inc. Road Survey Equipment. ARRB Group n.d. http://arrbgroup.net/. Accessed 24 May 2019

  48. He H, Garcia EA (2009) Learning from imbalanced data. IEEE Trans Knowl Data Eng 21(9):1263–1284

    Article  Google Scholar 

  49. Wu R, Yan S, Shan Y, Dang Q, Sun G (2015) Deep image: scaling up image recognition. arXiv:1501.02876

  50. Inoue H (2018) Data augmentation by pairing samples for image classification. arXiv:1801.02929

  51. Mikołajczyk A, Grochowski M (2018, May) Data augmentation for improving deep learning in image classification problem. In: 2018 international interdisciplinary Ph.D. workshop (IIPhDW), pp 117–122

  52. Abdi H, Williams LJ (2010) Principal component analysis. Wiley interdisciplinary reviews: computational statistics 2(4):433–459

    Article  Google Scholar 

  53. Glossary of terms journal of machine learning n.d. http://ai.stanford.edu/~ronnyk/glossary.html. Accessed 28 May 2019

Download references

Funding

The authors confirm that there is no source of funding for this study.

Author information

Authors and Affiliations

  1. Department of Civil Engineering, University of Memphis, Memphis, TN, 38152, USA

    Mohsen Maniat, Charles V. Camp & Ali R. Kashani

Authors
  1. Mohsen Maniat
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Charles V. Camp
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Ali R. Kashani
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Charles V. Camp.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Maniat, M., Camp, C.V. & Kashani, A.R. Deep learning-based visual crack detection using Google Street View images. Neural Comput & Applic 33, 14565–14582 (2021). https://doi.org/10.1007/s00521-021-06098-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06098-0

Keywords