Skip to main content
SpringerLink
Log in

Stone-by-Stone Segmentation for Monitoring Large Historical Monuments Using Deep Neural Networks

  • Conference paper
  • First Online:
Pattern Recognition. ICPR International Workshops and Challenges (ICPR 2021)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12667))

Included in the following conference series:

Abstract

Monitoring and restoration of cultural heritage buildings require the definition of an accurate health record. A critical step is the labeling of the exhaustive constitutive elements of the building. Stone-by-stone segmentation is a major part. Traditionally it is done by visual inspection and manual drawing on a 2D orthomosaic. This is an increasingly complex, time-consuming and resource-intensive task.

In this paper, algorithms to perform stone-by-stone segmentation automatically on large cultural heritage building are presented. Two advanced convolutional neural networks are tested and compared to conventional edge detection or thresholding methods on image dataset from Loire Valley’s châteaux: Château de Chambord and Château de Chaumont-sur-Loire, two castles of Renaissance style. The results show the applicability of the methods to the historical buildings of the Renaissance style.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017). https://doi.org/10.1109/TPAMI.2016.2644615

    Article  Google Scholar 

  2. Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. PAMI–8, 679–698 (1986). https://doi.org/10.1109/TPAMI.1986.4767851

    Article  Google Scholar 

  3. Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2018). https://doi.org/10.1109/tpami.2017.2699184

    Article  Google Scholar 

  4. Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European conference on computer vision (ECCV), pp. 801–818 (2018)

    Google Scholar 

  5. Garcia-Garcia, A., Orts-Escolano, S., Oprea, S., Villena-Martinez, V., Garcia-Rodriguez, J.: A review on deep learning techniques applied to semantic segmentation. arXiv preprint arXiv:1704.06857 (2017)

  6. Girshick, R.: Fast R-CNN. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 1440–1448, December 2015. https://doi.org/10.1109/ICCV.2015.169

  7. Grilli, E., Dininno, D., Petrucci, G., Remondino, F.: From 2D to 3D supervised segmentation and classification for cultural heritage applications. Int. Arch. Photogr. Remote Sens. Spat. Inf. Sci. 42(2) (2018)

    Google Scholar 

  8. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016

    Google Scholar 

  9. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  10. Hess, M.R., Petrovic, V., Kuester, F.: Interactive classification of construction materials: feedback driven framework for annotation and analysis of 3D point clouds. ISPRS - Int. Arch. Photogr. Remote Sens. Spat. Inf. Sci. XLII–2/W5, 343–347 (2017). https://doi.org/10.5194/isprs-archives-XLII-2-W5-343-2017

    Article  Google Scholar 

  11. Idjaton, K., Desquesnes, X., Treuillet, S., Brunetaud, X.: Segmentation automatique d’images pour le diagnostic de monuments historiques. In: Colloque GRETSI (Groupement de Recherche en Traitement du Signal et des Images) (2019)

    Google Scholar 

  12. Janvier-Badosa, S., Beck, K., Brunetaud, X., Guirimand-Dufour, A., Al-Mukhtar, M.: Gypsum and spalling decay mechanism of tuffeau limestone. Environ. Earth Sci. 74(3), 2209–2221 (2015). https://doi.org/10.1007/s12665-015-4212-2

    Article  Google Scholar 

  13. Janvier-Badosa, S., Brunetaud, X., Beck, K., Al-Mukhtar, M.: Kinetics of stone degradation of the castle of chambord in France. Int. J. Archit. Heritage 10(1), 96–105 (2016)

    Article  Google Scholar 

  14. Kapsalas, P., Zervakis, M., Maravelaki-Kalaitzaki, P., Delegou, E., Moropoulou, A.: Machine vision schemes towards detecting and estimating the state of corrosion. In: Pattern Recognition and Signal Processing in Archaeometry: Mathematical and Computational Solutions for Archaeology, pp. 146–165. IGI Global (2012)

    Google Scholar 

  15. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3431–3440 (2015). https://doi.org/10.1109/CVPR.2015.7298965

  16. Manferdini, A.M., Baroncini, V., Corsi, C.: An integrated and automated segmentation approach to deteriorated regions recognition on 3D reality-based models of cultural heritage artifacts. J. Cult. Heritage (2012). https://doi.org/10.1016/j.culher.2012.01.014

    Article  Google Scholar 

  17. Marson, C., Sammartano, G., Spanò, A., Valluzzi, M.R.: Lidar data analyses for assessing the conservation status of the so-called baths-church in Hierapolis of Phrygia (TR). ISPRS - Int. Arch. Photogr. Remote Sens. Spat. Inf. Sci. XLII–2/W11, 823–830 (2019). https://doi.org/10.5194/isprs-archives-XLII-2-W11-823-2019

    Article  Google Scholar 

  18. Murtiyoso, A., Grussenmeyer, P.: Point cloud segmentation and semantic annotation aided by GIS data for heritage complexes. In: ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. 42, pp. 523–528. Copernicus GmbH (2019). https://doi.org/10.5194/isprs-archives-XLII-2-W9-523-2019

  19. Oses, N., Dornaika, F.: Image-based delineation of built heritage masonry for automatic classification. In: Kamel, M., Campilho, A. (eds.) ICIAR 2013. LNCS, vol. 7950, pp. 782–789. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39094-4_90

    Chapter  Google Scholar 

  20. Otsu, N.: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. 9(1), 62–66 (1979)

    Article  Google Scholar 

  21. Pierrot, D.M.: Producing orthomosaic with a free open source software (micmac), application to the archeological survey of meremptah’s tomb (2014)

    Google Scholar 

  22. Pinte, A., Héno, R., Pierrot-Deseilligny, M., Brunetaud, X., Janvier-Badosa, S., Janvier, R.: Orthoimages of the outer walls and towers of the château de Chambord. ISPRS Ann. Photogr. Remote Sens. Spat. Inf. Sci. II–5/W3, 243–250 (2015). https://doi.org/10.5194/isprsannals-II-5-W3-243-2015

    Article  Google Scholar 

  23. Remondino, F.: Heritage recording and 3D modeling with photogrammetry and 3D scanning. Remote Sens. 3(6), 1104–1138 (2011). https://doi.org/10.3390/rs3061104

    Article  Google Scholar 

  24. Riveiro, B., Conde-Carnero, B., González-Jorge, H., Arias, P., Caamaño, J.: Automatic creation of structural models from point cloud data: the case of masonry structures. ISPRS Ann. Photogr. Remote Sens. Spat. Inf. Sci. 2 (2015)

    Google Scholar 

  25. Riveiro, B., Lourenço, P.B., Oliveira, D.V., González-Jorge, H., Arias, P.: Automatic morphologic analysis of quasi-periodic masonry walls from LiDAR. Comput. Aided Civ. Infrastr. Eng. 31(4), 305–319 (2016). https://doi.org/10.1111/mice.12145

    Article  Google Scholar 

  26. Sánchez-Aparicio, L.J., Del Pozo, S., Ramos, L.F., Arce, A., Fernandes, F.M.: Heritage site preservation with combined radiometric and geometric analysis of TLS data. Autom. Constr. 85, 24–39 (2018). https://doi.org/10.1016/j.autcon.2017.09.023

    Article  Google Scholar 

  27. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  28. Sobel, I.: An Isotropic 3\(\times \) 3 Gradient Operator, Machine Vision for Three-dimensional Scenes, p. 376379. Freeman, H., Academic Press (1990)

    Google Scholar 

  29. Valero, E., Bosché, F., Forster, A.: Automatic segmentation of 3D point clouds of rubble masonry walls, and its application to building surveying, repair and maintenance. Autom. Constr. (2018). https://doi.org/10.1016/j.autcon.2018.08.018

  30. Valero, E., Forster, A., Bosché, F., Hyslop, E., Wilson, L., Turmel, A.: Automated defect detection and classification in ashlar masonry walls using machine learning. Autom. Constr. (2019). https://doi.org/10.1016/j.autcon.2019.102846

  31. Valero, E., Forster, A., Bosché, F., Renier, C., Hyslop, E., Wilson, L.: High level-of-detail BIM and machine learning for automated masonry wall defect surveying (2018). https://doi.org/10.22260/ISARC2018/0101

  32. Yuan, L., Guo, J., Wang, Q.: Automatic classification of common building materials from 3D terrestrial laser scan data. Autom. Constr. (2020). https://doi.org/10.1016/j.autcon.2019.103017

Download references

Author information

Authors and Affiliations

  1. Université d’Orléans, INSA CVL, PRISME EA 4229, 45072, Orléans, France

    Koubouratou Idjaton, Xavier Desquesnes & Sylvie Treuillet

  2. Université d’Orléans, INSA CVL, LaMé, Orléans, France

    Xavier Brunetaud

Authors
  1. Koubouratou Idjaton
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xavier Desquesnes
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sylvie Treuillet
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xavier Brunetaud
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Koubouratou Idjaton .

Editor information

Editors and Affiliations

  1. Dipartimento di Ingegneria dell’Informazione, University of Firenze, Firenze, Italy

    Alberto Del Bimbo

  2. Dipartimento di Ingegneria “Enzo Ferrari”, Università di Modena e Reggio Emilia, Modena, Italy

    Rita Cucchiara

  3. Department of Computer Science, Boston University, Boston, MA, USA

    Stan Sclaroff

  4. Dipartimento di Matematica e Informatica, University of Catania, Catania, Italy

    Giovanni Maria Farinella

  5. Cloud & AI, JD.COM, Beijing, China

    Tao Mei

  6. Dipartimento di Ingegneria dell’Informazione, University of Firenze, Firenze, Italy

    Marco Bertini

  7. Computational Sciences Department, National Institute of Astrophysics, Optics and Electronics (INAOE), Tonantzintla, Puebla, Mexico

    Hugo Jair Escalante

  8. Dipartimento di Ingegneria “Enzo Ferrari”, Università di Modena e Reggio Emilia, Modena, Italy

    Roberto Vezzani

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Idjaton, K., Desquesnes, X., Treuillet, S., Brunetaud, X. (2021). Stone-by-Stone Segmentation for Monitoring Large Historical Monuments Using Deep Neural Networks. In: Del Bimbo, A., et al. Pattern Recognition. ICPR International Workshops and Challenges. ICPR 2021. Lecture Notes in Computer Science(), vol 12667. Springer, Cham. https://doi.org/10.1007/978-3-030-68787-8_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-68787-8_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-68786-1

  • Online ISBN: 978-3-030-68787-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation