Skip to main content
SpringerLink
Log in

Engineering-oriented bridge multiple-damage detection with damage integrity using modified faster region-based convolutional neural network

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

A bridge damage detector with preserving integrity based on modified Faster region-based convolutional neural network (R-CNN) is proposed for multiple damage types. The methodologies of dataset collection, damage annotation, and anchors generation are modified. The performance for bridge multiple-damage detectors with ResNet50 or ResNet101 as feature extraction network are compared. The results show that, with the modified Faster R-CNN, the mean average precision reaches 84.56% (76.43%) at the intersection-over-union metrics of 0.5 (0.75). We further demonstrate that the localization offset for Faster R-CNN is lower than that of YOLOv3. The modified bridge damage detector enables better detecting performance, and can preserve the damage integrity.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Adeli H (2001) Neural networks in civil engineering: 1989–2000. Comput-Aided Civ Infrastruct Eng 16(2):126–142

    Article  Google Scholar 

  2. Adhikari RS, Moselhi O, Bagchi A (2014) Image-based retrieval of concrete crack properties for bridge inspection. Autom Constr 39(39):180–194

    Article  Google Scholar 

  3. Butcher J, Day C, Austin J, Haycock P, Verstraeten D, Schrauwen B (2014) Defect detection in reinforced concrete using random neural architectures. Comput-Aided Civ Infrastruct Eng 29(3):191–207

    Article  Google Scholar 

  4. Cai W, Wei Z (2020) Remote sensing image classification based on a cross-attention mechanism and graph convolution. IEEE Geosci Remote Sens Lett 19:1–5. https://doi.org/10.1109/LGRS.2020.3026587

    Article  Google Scholar 

  5. Cai W, Liu B, Wei Z, Li M, Kan J (2021) Tardb-net: triple-attention guided residual dense and bilstm networks for hyperspectral image classification. Multimed Tools Appl 80(2):11291–11312

    Article  Google Scholar 

  6. Cha Y-J, Choi W, Büyüköztürk O (2017) Deep learning-based crack damage detection using convolutional neural networks. Comput-Aided Civ Infrastruct Eng 32(5):361–378

    Article  Google Scholar 

  7. Cha Y-J, Choi W, Suh G, Mahmoudkhani S, Büyüköztürk O (2018) Autonomous structural visual inspection using region based deep learning for detecting multiple damage types. Comput-Aided Civ Infrastruct Eng 33(9):731–747

    Article  Google Scholar 

  8. Chen F-C, Jahanshahi MR (2018) NB-CNN: deep learning-based crack detection using convolutional neural network and Naïve Bayes data fusion. IEEE Trans Ind Electron 65(5):4392–4400

    Article  Google Scholar 

  9. Dawood T, Zhu Z, Zayed T (2017) Machine vision-based model for spalling detection and quantification in subway networks. Autom Constr 81:149–160

    Article  Google Scholar 

  10. Deng JH, Lu Y, Lee VC-S (2019) Concrete crack detection with handwriting script interferences using faster region-based convolutional neural network. Comput-Aided Civ Infrastruct Eng 33(9):731–747

    Google Scholar 

  11. Dorafshan S, Maguire M, Thomas R (2018) SDNET2018: an annotated image dataset for non-contact concrete crack detection using deep convolutional neural networks. Data Br 21:1664–1668

    Article  Google Scholar 

  12. Dung CV, Anh LD (2019) Autonomous concrete crack detection using deep fully convolutional neural network. Autom Constr 99(3):52–58

    Article  Google Scholar 

  13. Everingham M, Gool LI, Williams CKI, Winn J, Zisserman A (2010) The pascal visual object classes (voc) challenge. Int J Comput Vis 88(2):303–338

    Article  Google Scholar 

  14. Fan Q, Brown L, Smith J (2016) A closer look at faster R-CNN for vehicle detection. In: Proceedings of 2016 IEEE intelligent vehicles symposium (IV), Gothenburg, Sweden, 19–22 June 2016, pp 124–129

  15. Gao Y, Mosalam K (2018) Deep transfer learning for image-based structural damage recognition. Comput-Aided Civ Infrastruct Eng 33(9):748–768

    Article  Google Scholar 

  16. German S, Brilakis I, DesRoches R (2012) Rapid entropy-based detection and properties measurement of concrete spalling with machine vision for post-earthquake safety assessments. Adv Eng Inform 26(4):846–858

    Article  Google Scholar 

  17. Ghosh-Dastidar S, Adeli H (2009) A new supervised learning algorithm for multiple spiking neural networks with application in epilepsy and seizure detection. Neural Netw 22(10):1419–1431

    Article  Google Scholar 

  18. Girshick R (2015) Fast R-CNN. In: Proceedings of the IEEE international conference on computer vision, Santiago, Chile, 07–13 December 2015, pp 1440–1148

  19. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE conference on computer vision and pattern recognition, Columbus, OH, pp 580–587

  20. Gou HY, Yang LC, Mo ZX, Guo W, Shi XY, Bao Y (2019) Effect of long-term bridge deformations on safe operation of high-speed railway and vibration of vehicle-bridge coupled system. Int J Struct Stab Dyn 19(9):1950111

    Article  MathSciNet  Google Scholar 

  21. Hoang ND, Nguyen QL, Tran VD (2018) Automatic recognition of asphalt pavement cracks using metaheuristic optimized edge detection algorithms and convolution neural network. Autom Constr 94(10):203–213

    Google Scholar 

  22. Huynh TC, Park JH, Jung HJ, Kim, & J. T. (2019) Quasi-autonomous bolt-loosening detection method using vision-based deep learning and image processing. Autom Constr 105(9):1–18

    Google Scholar 

  23. Isailović D, Vladeta S, Matthias T, Rico R, Rade H, Jürgen D (2020) Bridge damage: detection, IFC-based semantic enrichment and visualization. Autom Constr 112(4):1–22

    Google Scholar 

  24. Jiao L, Zhang F, Liu F, Yang S, Li L, Feng Z, Qu R (2019) A survey of deep learning-based object detection. IEEE access, pp 128837–128868

  25. Ju HY, Li W, Tighe S, Zhai JZ, Xu ZC, Chen Y (2019) Detection of sealed and unsealed cracks with complex backgrounds using deep convolutional neural network. Autom Constr 107(11):1–14

    Google Scholar 

  26. Kanungo T, Mount DM, Netanyahu NS, Piatko CD, Silverman R, Wu AY (2002) An efficient k-means clustering algorithm: analysis and implementation. IEEE Trans Pattern Anal Mach Intell 24(7):881–892

    Article  MATH  Google Scholar 

  27. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. Retrieved from https://arxiv.org/abs/1412.6980/

  28. Kohavi R, Provost F (1998) Glossary of terms. Mach Learn 30:271–274

    Article  Google Scholar 

  29. Koziarski M, Cyganek B (2017) Image recognition with deep neural networks in presence of noise—dealing with and taking advantage of distortions. Integr Comput Aided Eng 24(4):337–350

    Article  Google Scholar 

  30. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  Google Scholar 

  31. Li G, He S, Ju Y, Du K (2014) Long-distance precision inspection method for bridge cracks with image processing. Autom Constr 41:83–95

    Article  Google Scholar 

  32. Li H, Ou J, Zhang X, Pei M, Li N (2015) Research and practice of health monitoring for long-span bridges in the mainland of China. Smart Struct Syst 15(3):555–576

    Article  Google Scholar 

  33. Li R, Yuan Y, Zhang W, Yuan Y (2018) Unified vision based methodology for simultaneous concrete defect detection and geolocalization. Comput-Aided Civ Infrastruct Eng 33(7):527–544

    Article  Google Scholar 

  34. Li S, Zhao X, Zhou G (2019) Automatic pixel-level multiple damage detection of concrete structure using fully convolutional network. Comput-Aided Civ Infrastruct Eng 34(7):616–634

    Article  Google Scholar 

  35. Li MW, Wang YT, Geng J, Hong WC (2021) Chaos cloud quantum bat hybrid optimization algorithm. Nonlinear Dyn 103(4):1167–1193

    Article  Google Scholar 

  36. Liang X (2019) Image-based post-disaster inspection of reinforced concrete bridge systems using deep learning with Bayesian optimization. Comput-Aided Civ Infrastruct Eng 34(5):415–430

    Article  Google Scholar 

  37. Liao K-W, Lee Y-T (2016) Detection of rust defects on steel bridge coatings via digital image recognition. Autom Constr 71(2):294–306

    Article  Google Scholar 

  38. Lin Y, Nie Z, Ma H (2017) Structural damage detection with automatic feature-extraction through deep learning. Comput-Aided Civ Infrastruct Eng 32(12):1025–1046

    Article  Google Scholar 

  39. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016) SSD: single shot multibox detector. Proceedings of European conference on computer vision. Springer, Berlin, pp 21–37

  40. Oksuz K, Cam BC, Akbas E, Kalkan S (2018) Localization recall precision (LRP): a new performance metric for object detection. Retrieved from https://arxiv.org/abs/1807.01696

  41. Qin SQ, Gao ZY (2017) Developments and prospects of long-span high-speed railway bridge technologies in China. Engineering 003(006):787–794

    Article  Google Scholar 

  42. Qin SQ, Zhou Y, Kang J (2019) Footbridge serviceability analysis: from system identification to tuned mass damper implementation. KSCE J Civ Eng 23(2):754–762

    Article  Google Scholar 

  43. Rafiei MH, Adeli H (2017) A novel machine learning-based algorithm to detect damage in high-rise building structures. Struct Design Tall Spec Build 26(18):e1400

    Article  Google Scholar 

  44. Rawat W, Wang Z (2017) Deep convolutional neural networks for image classification: a comprehensive review. Neural Comput 29(9):2352–2449

    Article  MathSciNet  MATH  Google Scholar 

  45. Redmon J, Farhadi A (2018) YOLOv3: an incremental improvement. Retrieved from https://arxiv.org/abs/1804.02767/

  46. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. Proceedings of the IEEE conference on computer vision and pattern recognition, Las Vegas, NV, pp 779–788

  47. Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: towards real-time object detection with region proposal networks. Proceedings of the advances in neural information processing systems, Montreal, pp 91–99

  48. Tang MC (2007) Evolution of bridge technology. Iabse Symposium Report 93(31), 38–48(11)

  49. Tang MC (2016) Conceptual design of bridges. Struct Infrastruct Eng 13(4):1–10

    Google Scholar 

  50. Torres JF, Galicia A, Troncoso A, Martínez-Álvarez F (2018) A scalable approach based on deep learning for big data time series forecasting. Integr Comput Aided Eng 25(4):335–348

    Article  Google Scholar 

  51. Wang NN, Zhao XF, Zhao P, Zhang Y, Ou JP (2019) Automatic damage detection of historic masonry buildings based on mobile deep learning. Autom Constr 103(7):53–66

    Article  Google Scholar 

  52. Xiang H, Ge Y (2007) State-of-the-art of long-span bridge engineering in China. Front Archit Civ Eng China 1(4):379–388

    Article  Google Scholar 

  53. Xiang SY, Zhang Y, Gong J, Guo X, Lin L, Hao Y (2019) STDP-based unsupervised spike pattern learning in a photonic spiking neural network with VCSELs and VCSOAs. IEEE J Sel Top Quantum Electron 25(6):1700109

    Article  Google Scholar 

  54. Xiang SY, Ren Z, Zhang Y, Song Z, Hao Y (2020) All-optical neuromorphic XOR operation with inhibitory dynamics of a single photonic spiking neuron based on VCSEL-SA. Opt Lett 45(5):1104–1107

    Article  Google Scholar 

  55. Yang Z, Zhang S, Hu Y, Hu Z, Huang Y (2020) Vae-stega: linguistic steganography based on variational auto-encoder. IEEE Trans Inf Forensics Secur 16:880–895

    Article  Google Scholar 

  56. Yeum CM, Dyke SJ (2015) Vision-based automated crack detection for bridge inspection. Comput-Aided Civ Infrastruct Eng 30(10):759–770

    Article  Google Scholar 

  57. Zhang ZC, Hong WC (2021) Application of variational mode decomposition and chaotic grey wolf optimizer with support vector regression for forecasting electric loads. Knowl-Based Syst 228:107297

    Article  Google Scholar 

  58. Zhang A, Wang KCP, Li B, Yang E, Dai X, Peng Y, Fei Y, Liu Y, Li JQ, Chen C (2017) Automated pixel-level pavement crack detection on 3d asphalt surfaces using a deep-learning network. Comput-Aided Civ Infrastruct Eng 32(10):805–819

    Article  Google Scholar 

  59. Zhang JD, Deng WQ, Hashino T, Liu D (2018) The world's first extradosed bridge with corrugated steel webs. Bridge Eng 171(4):237–245

    Google Scholar 

  60. Zhang X, Rajan D, Story B (2019) Concrete crack detection using context-aware deep semantic segmentation network. Comput-Aided Civ Infrastruct Eng 34(11):951–971

    Article  Google Scholar 

  61. Zhang CB, Chang C-C, Jamshidi M (2020) Concrete bridge surface damage detection using a single-stage detector. Comput-Aided Civ Infrastruct Eng 35(4):389–409

    Article  Google Scholar 

  62. Zou Z, Shi Z, Guo Y, Ye J (2019) Object detection in 20 years: a survey. arXiv: computer vision and pattern recognition

Download references

Acknowledgements

This work was supported by the National Key Research and Development Program of China (2021YFB2801900, 2021YFB2801901, 2021YFB2801902, 2021YFB2801904); by the National Natural Science Foundation of China (61674119, 61974177); by the Key Science and Technology Project in Transportation Industry of China (2020-MS1-060).

Author information

Authors and Affiliations

  1. School of Highway, Chang’an University, Xi’an, 710064, China

    Licun Yu & Shuanhai He

  2. CCCC First Highway Consultants Co., Ltd, Xi’an, 710075, China

    Licun Yu & Ming Ma

  3. State Key Laboratory of Integrated Service Networks, Xidian University, Xi’an, 710071, China

    Xiaosong Liu & Shuiying Xiang

Authors
  1. Licun Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shuanhai He
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaosong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ming Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shuiying Xiang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Licun Yu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yu, L., He, S., Liu, X. et al. Engineering-oriented bridge multiple-damage detection with damage integrity using modified faster region-based convolutional neural network. Multimed Tools Appl 81, 18279–18304 (2022). https://doi.org/10.1007/s11042-022-12703-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12703-8

Keywords

Search

Navigation