Abstract
The reconstruction of destroyed frescoes is a complex task: very small fragments, irregular shapes, color alterations and missing pieces are only some of the possible problems that we have to deal with. Surely, an important preliminary step involves the separation of mixed fragments. In fact, in a real scenario, such as a church destroyed by an earthquake, it is likely that pieces of different frescoes, which were close on the same wall, end up mixed together, making their reconstruction more complex. Their separation may be especially difficult if there are many of them and if there are no (or very old) reference images of the original frescoes. A possible way to separate the fragments is to treat this problem as a stylistic classification task, in which we have only parts of an artwork instead of a complete one. In this work, we tested various machine and deep learning solutions on the DAFNE dataset (to date the largest open access collection of artificially fragmented fresco images). The experiments showed promising results, with good performances in both binary and multi-class classification.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abate, D.: FRAGMENTS: a fully automatic photogrammetric fragments recomposition workflow. J. Cult. Herit. 47, 155–165 (2021). https://doi.org/10.1016/j.culher.2020.09.015
Arora, R.S., Elgammal, A.: Towards automated classification of fine-art painting style: a comparative study. In: Proceedings of the 21st International Conference on Pattern Recognition (ICPR 2012), pp. 3541–3544 (2012)
Barra, P., Barra, S., Nappi, M., Narducci, F.: SAFFO: a SIFT based approach for digital anastylosis for fresco reconstruction. Pattern Recogn. Lett. 138, 123–129 (2020). https://doi.org/10.1016/j.patrec.2020.07.008
Cetinic, E., Lipic, T., Grgic, S.: Fine-tuning convolutional neural networks for fine art classification. Expert Syst. Appl. 114, 107–118 (2018). https://doi.org/10.1016/j.eswa.2018.07.026
Cetinic, E., Lipic, T., Grgic, S.: Learning the principles of art history with convolutional neural networks. Pattern Recogn. Lett. 129, 56–62 (2020). https://doi.org/10.1016/j.patrec.2019.11.008
Del Chiaro, R., Bagdanov, A.D., Del Bimbo, A.: NoisyArt: a dataset for webly-supervised artwork recognition. In: VISIGRAPP (4: VISAPP), pp. 467–475 (2019)
Derech, N., Tal, A., Shimshoni, I.: Solving archaeological puzzles. Pattern Recogn. 119, 108065 (2021). https://doi.org/10.1016/j.patcog.2021.108065
Dondi, P., Lombardi, L., Malagodi, M., Licchelli, M.: Stylistic classification of historical violins: a deep learning approach. In: Del Bimbo, A., et al. (eds.) ICPR 2021. LNCS, vol. 12667, pp. 112–125. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-68787-8_8
Dondi, P., Lombardi, L., Setti, A.: DAFNE: a dataset of fresco fragments for digital anastlylosis. Pattern Recogn. Lett. 138, 631–637 (2020). https://doi.org/10.1016/j.patrec.2020.09.015
Elgammal, A., Liu, B., Kim, D., Elhoseiny, M., Mazzone, M.: The shape of art history in the eyes of the machine. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)
Europeana: Europeana digital library. https://www.europeana.eu/en. Accessed 02 Feb 2022
Fiorucci, M., Khoroshiltseva, M., Pontil, M., Traviglia, A., Del Bue, A., James, S.: Machine learning for cultural heritage: a survey. Pattern Recogn. Lett. 133, 102–108 (2020). https://doi.org/10.1016/j.patrec.2020.02.017
Fornasier, M., Toniolo, D.: Fast, robust and efficient 2D pattern recognition for re-assembling fragmented images. Pattern Recogn. 38(11), 2074–2087 (2005). https://doi.org/10.1016/j.patcog.2005.03.014
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016). https://doi.org/10.1109/CVPR.2016.90
Karayev, S., et al.: Recognizing image style. In: Proceedings of the British Machine Vision Conference. BMVA Press (2014). https://doi.org/10.5244/C.28.122
Koller, D., Trimble, J., Najbjerg, T., Gelfand, N., Levoy, M.: Fragments of the city: Stanford’s digital Forma Urbis Romae project. In: Proceedings of the Third Williams Symposium on Classical Architecture, vol. 61, pp. 237–252 (2006)
Lermé, N., Hégarat-Mascle, S.L., Zhang, B., Aldea, E.: Fast and efficient reconstruction of digitized frescoes. Pattern Recogn. Lett. 138, 417–423 (2020). https://doi.org/10.1016/j.patrec.2020.08.006
Li, J., Wang, J.Z.: Studying digital imagery of ancient paintings by mixtures of stochastic models. IEEE Trans. Image Process. 13(3), 340–353 (2004). https://doi.org/10.1109/TIP.2003.821349
Lombardi, T.E.: The classification of style in fine-art painting. Pace University (2005)
Papaodysseus, C., Panagopoulos, T., Exarhos, M., Triantafillou, C., Fragoulis, D., Doumas, C.: Contour-shape based reconstruction of fragmented, 1600 BC wall paintings. IEEE Trans. Signal Process. 50(6), 1277–1288 (2002). https://doi.org/10.1109/TSP.2002.1003053
Pouyanfar, S., et al.: A survey on deep learning: algorithms, techniques, and applications. ACM Comput. Surv. 51(5) (2018). https://doi.org/10.1145/3234150
Ragusa, F., Furnari, A., Battiato, S., Signorello, G., Farinella, G.M.: EGO-CH: dataset and fundamental tasks for visitors behavioral understanding using egocentric vision. Pattern Recogn. Lett. 131, 150–157 (2020)
Sandoval, C., Pirogova, E., Lech, M.: Two-stage deep learning approach to the classification of fine-art paintings. IEEE Access 7, 41770–41781 (2019). https://doi.org/10.1109/ACCESS.2019.2907986
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Tan, W.R., Chan, C.S., Aguirre, H.E., Tanaka, K.: Ceci n’est pas une pipe: a deep convolutional network for fine-art paintings classification. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 3703–3707 (2016). https://doi.org/10.1109/ICIP.2016.7533051
Voulodimos, A., Doulamis, N., Doulamis, A., Protopapadakis, E.: Deep learning for computer vision: a brief review. Comput. Intell. Neurosci. 2018(7068349) (2018). https://doi.org/10.1155/2018/7068349
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Cascone, L., Dondi, P., Lombardi, L., Narducci, F. (2022). Automatic Classification of Fresco Fragments: A Machine and Deep Learning Study. In: Sclaroff, S., Distante, C., Leo, M., Farinella, G.M., Tombari, F. (eds) Image Analysis and Processing – ICIAP 2022. ICIAP 2022. Lecture Notes in Computer Science, vol 13231. Springer, Cham. https://doi.org/10.1007/978-3-031-06427-2_58
Download citation
DOI: https://doi.org/10.1007/978-3-031-06427-2_58
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-06426-5
Online ISBN: 978-3-031-06427-2
eBook Packages: Computer ScienceComputer Science (R0)