Abstract
Cultural heritage buildings damage detection is of a great significance for planning restoration operations. However, the buildings analysis is generally performed by experts through on-site qualitative visual assessments. A highly time-consuming task, hardly possible at the scale of large historical buildings.
This paper proposes a new neural network architecture for automatic detection of spalling zones in limestone walls with color images. This architecture consists of the latest YOLO network, enhanced with layers of transformers encoder providing more comprehensive features. The performances of the proposed network improve significantly those of the YOLO core network on our dataset of over 1000 high resolution images from the Renaissance style Château de Chaumont in the Loire Valley (France).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
ICOMOS-ISCS: Illustrated glossary on stone deterioration patterns = icomos-iscs: Glossaire illustré sur les formes d’altération de la pierre (2008)
Dosovitskiy, A., et al.: An image is worth 16x16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)
He, K., Zhang, X., Ren, S., Sun, J.: Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 37(9), 1904–1916 (2015)
Hosang, J., Benenson, R., Schiele, B.: Learning non-maximum suppression. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017
Janvier-Badosa, S., Beck, K., Brunetaud, X., Guirimand-Dufour, A., Al-Mukhtar, M.: Gypsum and spalling decay mechanism of Tuffeau limestone. Environ. Earth Sci. 74(3), 2209–2221 (2015). https://doi.org/10.1007/s12665-015-4212-2
Jocher, G., Nishimura, K., Mineeva, T., Vilariño, R.: Yolov5. Code repository (2020) https://github.com/ultralytics/yolov5
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 25, 1097–1105 (2012)
Kwon, D., Yu, J.: Automatic damage detection of stone cultural property based on deep learning algorithm. Int. Arch. Photogram. Remote Sens. Spatial Inf. Sci. 42(2/W15) (2019)
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48
Manferdini, A.M., Baroncini, V., Corsi, C.: An integrated and automated segmentation approach to deteriorated regions recognition on 3d reality-based models of cultural heritage artifacts. J. Cult. Herit. 13(4), 371–378 (2012)
Pierrot, D.M.: Producing orthomosaic with a free open source software (micmac), application to the archeological survey of meremptah’s tomb. In: Workshop Digital Specimen, Berlin, pp. 8–12, September 2014
Rezatofighi, H., Tsoi, N., Gwak, J., Sadeghian, A., Reid, I., Savarese, S.: Generalized intersection over union: A metric and a loss for bounding box regression. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 658–666 (2019)
Smith, L.N.: A disciplined approach to neural network hyper-parameters: Part 1-learning rate, batch size, momentum, and weight decay. arXiv preprint arXiv:1803.09820 (2018)
Szegedy, C., et al.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision And Pattern Recognition, pp. 1–9 (2015)
Valero, E., Forster, A., Bosché, F., Hyslop, E., Wilson, L., Turmel, A.: Automated defect detection and classification in ashlar masonry walls using machine learning. Autom. Construct. 106, 102846 (2019)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
Wang, C.Y., Liao, H.Y.M., Wu, Y.H., Chen, P.Y., Hsieh, J.W., Yeh, I.H.: CspNet: a new backbone that can enhance learning capability of CNN. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 390–391 (2020)
Wang, N., Zhao, Q., Li, S., Zhao, X., Zhao, P.: Damage classification for masonry historic structures using convolutional neural networks based on still images. Comput. Aid. Civil Infrastruct. Eng. 33(12), 1073–1089, (2018)
Wang, N., Zhao, X., Zhao, P., Zhang, Y., Zou, Z., Ou, J.: Automatic damage detection of historic masonry buildings based on mobile deep learning. Autom. Construct. 103, 53–66 (2019)
Zhang, Z., Lu, X., Cao, G., Yang, Y., Jiao, L., Liu, F.: ViT-YOLO: transformer-based yolo for object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2799–2808 (2021)
Zheng, Z., Wang, P., Liu, W., Li, J., Ye, R., Ren, D.: Distance-IoU loss: faster and better learning for bounding box regression. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 12993–13000 (2020)
Acknowledgment
The authors benefited from the use of the cluster at the Centre de Calcul Scientifique en région Centre-Val de Loire.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Idjaton, K., Desquesnes, X., Treuillet, S., Brunetaud, X. (2022). Transformers with YOLO Network for Damage Detection in Limestone Wall Images. In: Mazzeo, P.L., Frontoni, E., Sclaroff, S., Distante, C. (eds) Image Analysis and Processing. ICIAP 2022 Workshops. ICIAP 2022. Lecture Notes in Computer Science, vol 13374. Springer, Cham. https://doi.org/10.1007/978-3-031-13324-4_26
Download citation
DOI: https://doi.org/10.1007/978-3-031-13324-4_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-13323-7
Online ISBN: 978-3-031-13324-4
eBook Packages: Computer ScienceComputer Science (R0)