Abstract
Recently, the construction industry has been digitizing its production processes, the so-called Construction 4.0, in allusion to the paradigm of the fourth industrial revolution. The application of Deep Learning in computer vision systems has been highlighted in Construction 4.0. Thus, the main contribution of this work is to present a systematic review of Deep Learning for vision systems under Construction 4.0, considering the most cited and most recent journal articles between 2017 and 2021 from Scopus database. For this, a research method selected and analyzed 76 published papers. Six main points were evaluated in the proposed methodology: study area, computer vision applications, Deep Learning methods, hyperparameter tuning, data augmentation, and future work. The following topics stand out as relevant perspectives and directions for continued advancement in this field of research: improving Deep Learning models, increasing the quality of databases, investigating the generality techniques and optimizing processing capacity.
Similar content being viewed by others
Data availability
The datasets used or analyzed during the current study are available from the corresponding author on reasonable request.
References
Forcael, E., Ferrari, I., Opazo-Vega, A., Pulido-Arcas, J.A.: Construction 4.0: a literature review. Sustainability 12(22), 66 (2020)
Nagy, O., Papp, I., Szabó, R.Z.: Construction 4.0 organisational level challenges and solutions. Sustainability 13(21), 1–18 (2021)
Perrier, N., Bled, A., Bourgault, M., Cousin, N., Danjou, C., Pellerin, R., Roland, T.: Construction 4.0: a survey of research trends. J. Inf. Technol. Constr. 25, 416–437 (2020)
Schönbeck, P., Löfsjögård, M., Ansell, A.: Quantitative review of Construction 4.0 technology presence in construction project research. Buildings 10(10), 66 (2020)
Sawhney, A., Riley, M., Irizarry, J.: Construction 4.0: An Innovation Platform for the Built Environment. Routledge, London (2020)
Rey, R.O., de Melo, R.R.S., Costa, D.B.: Design and implementation of a computerized safety inspection system for construction sites using UAS and digital checklists-smart inspecs. Saf. Sci. 143, 105430 (2021)
Ottoni, A.L.C., Novo, M.S., Costa, D.B.: Hyperparameter tuning of convolutional neural networks for building construction image classification. Vis. Comput. 66, 1–15 (2022)
Pang, J., Zhang, H., Zhao, H., Li, L.: Dcsnet: a real-time deep network for crack segmentation. Signal Image Video Process. 16(4), 911–919 (2022)
Cha, Y.-J., Choi, W., Büyüköztürk, O.: Deep learning-based crack damage detection using convolutional neural networks. Comput. Aided Civ. Infrastruct. Eng. 32(5), 361–378 (2017)
Gopalakrishnan, K., Khaitan, S.K., Choudhary, A., Agrawal, A.: Deep convolutional neural networks with transfer learning for computer vision-based data-driven pavement distress detection. Constr. Build. Mater. 157, 322–330 (2017)
Kim, H., Kim, H., Hong, Y.W., Byun, H.: Detecting construction equipment using a region-based fully convolutional network and transfer learning. J. Comput. Civ. Eng. 32(2), 04017082 (2018)
Dung, C.V., Anh, L.D.: Autonomous concrete crack detection using deep fully convolutional neural network. Autom. Constr. 99, 52–58 (2019)
Li, S., Zhao, X.: Image-based concrete crack detection using convolutional neural network and exhaustive search technique. Adv. Civ. Eng. 6, 66 (2019)
Lecun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)
Schmidhuber, J.: Deep learning in neural networks: an overview. Neural Netw. 61, 85–117 (2015)
Litjens, G., Kooi, T., Bejnordi, B.E., Setio, A.A.A., Ciompi, F., Ghafoorian, M., Van Der Laak, J.A., Van Ginneken, B., Sánchez, C.I.: A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60–88 (2017)
Moen, E., Bannon, D., Kudo, T., Graf, W., Covert, M., Van Valen, D.: Deep learning for cellular image analysis. Nat. Methods 16(12), 1233–1246 (2019)
Tian, C., Fei, L., Zheng, W., Xu, Y., Zuo, W., Lin, C.-W.: Deep Learning on image denoising: an overview. Neural Netw. 6, 66 (2020)
Elgendy, M.: Deep Learning for Vision Systems. Manning Publications (2020)
Kc, K., Yin, Z., Wu, M., Wu, Z.: Evaluation of deep learning-based approaches for Covid-19 classification based on chest X-ray images. Signal Image Video Process. 15(5), 959–966 (2021)
Bolhasani, H., Mohseni, M., Rahmani, A.M.: Deep learning applications for iot in health care: a systematic review. Inform. Med. Unlocked 23, 100550 (2021)
Safayari, A., Bolhasani, H.: Depression diagnosis by deep learning using EEG signals: a systematic review. Med. Nov. Technol. Devices 12, 100102 (2021)
Dorafshan, S., Thomas, R.J., Maguire, M.: Comparison of deep convolutional neural networks and edge detectors for image-based crack detection in concrete. Constr. Build. Mater. 186, 1031–1045 (2018)
Yang, Z., He, B., Liu, Y., Wang, D., Zhu, G.: Classification of rock fragments produced by tunnel boring machine using convolutional neural networks. Autom. Constr. 125, 103612 (2021)
Jang, Y., Ahn, Y., Kim, H.Y.: Estimating compressive strength of concrete using deep convolutional neural networks with digital microscope images. J. Comput. Civ. Eng. 33(3), 04019018 (2019)
Guo, J., Wang, Q., Li, Y.: Semi-supervised learning based on convolutional neural network and uncertainty filter for façade defects classification. Comput. Aided Civ. Infrastruct. Eng. 36(3), 302–317 (2021)
Majidifard, H., Adu-Gyamfi, Y., Buttlar, W.G.: Deep machine learning approach to develop a new asphalt pavement condition index. Constr. Build. Mater. 247, 118513 (2020)
Ren, Y., Huang, J., Hong, Z., Lu, W., Yin, J., Zou, L., Shen, X.: Image-based concrete crack detection in tunnels using deep fully convolutional networks. Constr. Build. Mater. 234, 117367 (2020)
Alipour, M., Harris, D.K., Miller, G.R.: Robust pixel-level crack detection using deep fully convolutional neural networks. J. Comput. Civ. Eng. 33(6), 04019040 (2019)
Chen, J., Kira, Z., Cho, Y.K.: Deep learning approach to point cloud scene understanding for automated scan to 3d reconstruction. J. Comput. Civ. Eng. 33(4), 04019027 (2019)
Crawford, P.S., Al-Zarrad, M.A., Graettinger, A.J., Hainen, A.M., Back, E., Powell, L.: Rapid disaster data dissemination and vulnerability assessment through synthesis of a web-based extreme event viewer and deep learning. Adv. Civ. Eng. 6, 66 (2018)
Dais, D., Bal, I.E., Smyrou, E., Sarhosis, V.: Automatic crack classification and segmentation on masonry surfaces using convolutional neural networks and transfer learning. Autom. Constr. 125, 103606 (2021)
Kumar, S.S., Abraham, D.M., Jahanshahi, M.R., Iseley, T., Starr, J.: Automated defect classification in sewer closed circuit television inspections using deep convolutional neural networks. Autom. Constr. 91, 273–283 (2018)
Park, S., Bang, S., Kim, H., Kim, H.: Patch-based crack detection in black box images using convolutional neural networks. J. Comput. Civ. Eng. 33(3), 04019017 (2019)
Xue, Y., Li, Y.: A fast detection method via region-based fully convolutional neural networks for shield tunnel lining defects. Comput. Aided Civ. Infrastruct. Eng. 33(8), 638–654 (2018)
Qin, X., Cui, S., Liu, L., Wang, P., Wang, M., Xin, J.: Prediction of mechanical strength based on deep learning using the scanning electron image of microscopic cemented paste backfill. Adv. Civ. Eng. 6, 66 (2018)
Tong, Z., Gao, J., Zhang, H.: Innovative method for recognizing subgrade defects based on a convolutional neural network. Constr. Build. Mater. 169, 69–82 (2018)
Wang, X., Zhu, Z.: Vision-based hand signal recognition in construction: a feasibility study. Autom. Constr. 125, 103625 (2021)
Yang, Y., Yang, L., Wu, B., Yao, G., Li, H., Robert, S.: Safety prediction using vehicle safety evaluation model passing on long-span bridge with fully connected neural network. Adv. Civ. Eng. 6, 66 (2019)
Ali, R., Cha, Y.-J.: Subsurface damage detection of a steel bridge using deep learning and uncooled micro-bolometer. Constr. Build. Mater. 226, 376–387 (2019)
Bang, S., Park, S., Kim, H., Kim, H.: Encoder-decoder network for pixel-level road crack detection in black-box images. Comput. Aided Civ. Infrastruct. Eng. 34(8), 713–727 (2019)
Bianchi, E., Abbott, A.L., Tokekar, P., Hebdon, M.: Coco-bridge: structural detail data set for bridge inspections. J. Comput. Civ. Eng. 35(3), 04021003 (2021)
Cha, Y.-J., Choi, W., Suh, G., Mahmoudkhani, S., Büyüköztürk, O.: Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types. Comput. Aided Civ. Infrastruct. Eng. 33(9), 731–747 (2018)
Chen, S., Demachi, K.: Towards on-site hazards identification of improper use of personal protective equipment using deep learning-based geometric relationships and hierarchical scene graph. Autom. Constr. 125, 103619 (2021)
Cheng, J.C., Wang, M.: Automated detection of sewer pipe defects in closed-circuit television images using deep learning techniques. Autom. Constr. 95, 155–171 (2018)
Deng, G., Zhou, Z., Chu, X., Shao, S.: Identification of behavioral features of bridge structure based on static image sequences. Adv. Civ. Eng. 6, 66 (2020)
Ding, L., Fang, W., Luo, H., Love, P.E., Zhong, B., Ouyang, X.: A deep hybrid learning model to detect unsafe behavior: integrating convolution neural networks and long short-term memory. Autom. Constr. 86, 118–124 (2018)
Fang, Q., Li, H., Luo, X., Ding, L., Luo, H., Rose, T.M., An, W.: Detecting non-hardhat-use by a deep learning method from far-field surveillance videos. Autom. Constr. 85, 1–9 (2018)
Fang, W., Ding, L., Luo, H., Love, P.E.: Falls from heights: a computer vision-based approach for safety harness detection. Autom. Constr. 91, 53–61 (2018)
Gao, Y., Mosalam, K.M.: Deep transfer learning for image-based structural damage recognition. Comput. Aided Civ. Infrastruct. Eng. 33(9), 748–768 (2018)
Gulgec, N.S., Takáč, M., Pakzad, S.N.: Convolutional neural network approach for robust structural damage detection and localization. J. Comput. Civ. Eng. 33(3), 04019005 (2019)
Guo, F., Qian, Y., Shi, Y.: Real-time railroad track components inspection based on the improved yolov4 framework. Autom. Constr. 125, 103596 (2021)
Guo, F., Qian, Y., Wu, Y., Leng, Z., Yu, H.: Automatic railroad track components inspection using real-time instance segmentation. Comput. Aided Civ. Infrastruct. Eng. 36(3), 362–377 (2021)
Nhat-Duc, H., Nguyen, Q.-L., Tran, V.-D.: Automatic recognition of asphalt pavement cracks using metaheuristic optimized edge detection algorithms and convolution neural network. Autom. Constr. 94, 203–213 (2018)
Hoang, N.-D., Nguyen, Q.-L.: A novel approach for automatic detection of concrete surface voids using image texture analysis and history-based adaptive differential evolution optimized support vector machine. Adv. Civ. Eng. 6, 66 (2020)
Kang, D., Cha, Y.-J.: Autonomous uavs for structural health monitoring using deep learning and an ultrasonic beacon system with geo-tagging. Comput. Aided Civ. Infrastruct. Eng. 33(10), 885–902 (2018)
Kim, D., Liu, M., Lee, S., Kamat, V.R.: Remote proximity monitoring between mobile construction resources using camera-mounted uavs. Autom. Constr. 99, 168–182 (2019)
Kolar, Z., Chen, H., Luo, X.: Transfer learning and deep convolutional neural networks for safety guardrail detection in 2d images. Autom. Constr. 89, 58–70 (2018)
Li, Y., Zhang, H., Wang, S., Wang, H., Li, J.: Image-based underwater inspection system for abrasion of stilling basin slabs of dam. Adv. Civ. Eng. 6, 66 (2019)
Li, Y., Wei, H., Han, Z., Huang, J., Wang, W.: Deep learning-based safety helmet detection in engineering management based on convolutional neural networks. Adv. Civ. Eng. 6, 66 (2020)
Li, S., Gu, X., Xu, X., Xu, D., Zhang, T., Liu, Z., Dong, Q.: Detection of concealed cracks from ground penetrating radar images based on deep learning algorithm. Constr. Build. Mater. 273, 121949 (2021)
Lin, J.J., Ibrahim, A., Sarwade, S., Golparvar-Fard, M.: Bridge inspection with aerial robots: automating the entire pipeline of visual data capture, 3d mapping, defect detection, analysis, and reporting. J. Comput. Civ. Eng. 35(2), 04020064 (2021)
Martinez, P., Barkokebas, B., Hamzeh, F., Al-Hussein, M., Ahmad, R.: A vision-based approach for automatic progress tracking of floor paneling in offsite construction facilities. Autom. Constr. 125, 103620 (2021)
Pan, Z., Yang, J., Wang, X.-E., Wang, F., Azim, I., Wang, C.: Image-based surface scratch detection on architectural glass panels using deep learning approach. Constr. Build. Mater. 282, 122717 (2021)
Park, J.A., Yeum, C.M., Hrynyk, T.D.: Learning-based image scale estimation using surface textures for quantitative visual inspection of regions-of-interest. Comput. Aided Civ. Infrastruct. Eng. 36(2), 227–241 (2021)
Park, S., Baek, F., Sohn, J., Kim, H.: Computer vision-based estimation of flood depth in flooded-vehicle images. J. Comput. Civ. Eng. 35(2), 04020072 (2021)
Peng, X., Zhong, X., Zhao, C., Chen, Y.F., Zhang, T.: The feasibility assessment study of bridge crack width recognition in images based on special inspection UAV. Adv. Civ. Eng. 6, 66 (2020)
Maeda, H., Sekimoto, Y., Seto, T., Kashiyama, T., Omata, H.: Road damage detection and classification using deep neural networks with smartphone images. Comput. Aided Civ. Infrastruct. Eng. 33(12), 1127–1141 (2018)
Tong, Z., Gao, J., Zhang, H.: Recognition, location, measurement, and 3d reconstruction of concealed cracks using convolutional neural networks. Constr. Build. Mater. 146, 775–787 (2017)
Wei, W., Ding, L., Luo, H., Li, C., Li, G.: Automated bughole detection and quality performance assessment of concrete using image processing and deep convolutional neural networks. Constr. Build. Mater. 281, 122576 (2021)
Wijnands, J.S., Zhao, H., Nice, K.A., Thompson, J., Scully, K., Guo, J., Stevenson, M.: Identifying safe intersection design through unsupervised feature extraction from satellite imagery. Comput. Aided Civ. Infrastruct. Eng. 36(3), 346–361 (2021)
Xiao, B., Kang, S.-C.: Development of an image data set of construction machines for deep learning object detection. J. Comput. Civ. Eng. 35(2), 05020005 (2021)
Xiao, B., Kang, S.-C.: Vision-based method integrating deep learning detection for tracking multiple construction machines. J. Comput. Civ. Eng. 35(2), 04020071 (2021)
Xu, Y., Shen, X., Lim, S.: Cordet: corner-aware 3d object detection networks for automated scan-to-bim. J. Comput. Civ. Eng. 35(3), 04021002 (2021)
Yao, G., Wei, F., Yang, Y., Sun, Y.: Deep-learning-based bughole detection for concrete surface image. Adv. Civ. Eng. 6, 66 (2019)
Luo, X., Li, H., Cao, D., Dai, F., Seo, J., Lee, S., et al.: Recognizing diverse construction activities in site images via relevance networks of construction-related objects detected by convolutional neural networks. J. Comput. Civ. Eng. 32(3), 04018012 (2018)
Yin, X., Ma, T., Bouferguene, A., Al-Hussein, M.: Automation for sewer pipe assessment: Cctv video interpretation algorithm and sewer pipe video assessment (spva) system development. Autom. Constr. 125, 103622 (2021)
Yu, Y., Li, H., Umer, W., Dong, C., Yang, X., Skitmore, M., Wong, A.Y.: Automatic biomechanical workload estimation for construction workers by computer vision and smart insoles. J. Comput. Civ. Eng. 33(3), 04019010 (2019)
Zhang, A., Wang, K.C., Li, B., Yang, E., Dai, X., Peng, Y., Fei, Y., Liu, Y., Li, J.Q., Chen, C.: Automated pixel-level pavement crack detection on 3d asphalt surfaces using a deep-learning network. Comput. Aided Civ. Infrastruct. Eng. 32(10), 805–819 (2017)
Zhang, A., Wang, K.C., Fei, Y., Liu, Y., Tao, S., Chen, C., Li, J.Q., Li, B.: Deep learning-based fully automated pavement crack detection on 3d asphalt surfaces with an improved cracknet. J. Comput. Civ. Eng. 32(5), 04018041 (2018)
Zhang, J., Zi, L., Hou, Y., Wang, M., Jiang, W., Deng, D.: A deep learning-based approach to enable action recognition for construction equipment. Adv. Civ. Eng. 6, 66 (2020)
Khilji, T.N., Lopes Amaral Loures, L., Rezazadeh Azar, E.: Distress recognition in unpaved roads using unmanned aerial systems and deep learning segmentation. J. Comput. Civ. Eng. 35(2), 04020061 (2021)
Liu, Z., Cao, Y., Wang, Y., Wang, W.: Computer vision-based concrete crack detection using u-net fully convolutional networks. Autom. Constr. 104, 129–139 (2019)
Park, G., Lee, M., Jang, H., Kim, C.: Thermal anomaly detection in walls via cnn-based segmentation. Autom. Constr. 125, 103627 (2021)
Pi, Y., Nath, N.D., Behzadan, A.H.: Detection and semantic segmentation of disaster damage in uav footage. J. Comput. Civ. Eng. 35(2), 04020063 (2021)
Yang, X., Li, H., Yu, Y., Luo, X., Huang, T., Yang, X.: Automatic pixel-level crack detection and measurement using fully convolutional network. Comput. Aided Civ. Infrastruct. Eng. 33(12), 1090–1109 (2018)
Zhou, S., Song, W.: Crack segmentation through deep convolutional neural networks and heterogeneous image fusion. Autom. Constr. 125, 103605 (2021)
Hutter, F., Hoos, H., Leyton-Brown, K.: An efficient approach for assessing hyperparameter importance. In: Proceedings of International Conference on Machine Learning 2014 (ICML 2014), pp. 754–762 (2014)
Hutter, F., Kotthoff, L., Vanschoren, J. (eds.): Automated Machine Learning: Methods, Systems, Challenges. Springer, Berlin (2018)
Sajedi, S.O., Liang, X.: Uncertainty-assisted deep vision structural health monitoring. Comput. Aided Civ. Infrastruct. Eng. 36(2), 126–142 (2021)
Valikhani, A., Jaberi Jahromi, A., Pouyanfar, S., Mantawy, I.M., Azizinamini, A.: Machine learning and image processing approaches for estimating concrete surface roughness using basic cameras. Comput. Aided Civ. Infrastruct. Eng. 36(2), 213–226 (2021)
Ottoni, A.L.C., Amorim, R.M., Novo, M.S., Costa, D.B.: Tuning of data augmentation hyperparameters in Deep Learning to building construction image classification with small datasets. Int. J. Mach. Learn. Cybernet. 66, 1–16 (2022)
Chollet, F., Allaire, J.J.: Deep Learning With R. Manning Publications (2018)
Tong, Z., Wang, Z., Wang, X., Ma, Y., Guo, H., Liu, C.: Characterization of hydration and dry shrinkage behavior of cement emulsified asphalt composites using deep learning. Constr. Build. Mater. 274, 121898 (2021)
Bianchi, E., Abbott, A.L., Tokekar, P., Hebdon, M.: Coco-bridge: Common Objects in Context Dataset for Structural Detail Detection of Bridges (2021)
Shen, J., Xiong, X., Li, Y., He, W., Li, P., Zheng, X.: Detecting safety helmet wearing on construction sites with bounding-box regression and deep transfer learning. Comput. Aided Civ. Infrastruct. Eng. 36(2), 180–196 (2021)
Xiao, B., Kang, S.-C.: Acid—alberta construction image dataset. https://www.acidb.ca/ (2021)
Acknowledgements
This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brazil (CAPES) - Finance Code 001 and Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq) - Chamada CNPq/MCTI/FNDCT \(\hbox {N}^\circ \) 18/2021 - UNIVERSAL.
Author information
Authors and Affiliations
Contributions
AO: conceptualization, methodology, writing, review and editing. MN: conceptualization, writing, review, editing and supervision. DC: conceptualization, review and supervision. All authors contributed to the final manuscript.
Corresponding author
Ethics declarations
Conflict of interest
All the authors declare that they have no conflict of interest.
Ethical approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Ottoni, A.L.C., Novo, M.S. & Costa, D.B. Deep Learning for vision systems in Construction 4.0: a systematic review. SIViP 17, 1821–1829 (2023). https://doi.org/10.1007/s11760-022-02393-y
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11760-022-02393-y