Skip to main content
SpringerLink
Log in

The AIRES-CH Project: Artificial Intelligence for Digital REStoration of Cultural Heritages Using Nuclear Imaging and Multidimensional Adversarial Neural Networks

  • Conference paper
  • First Online:
Image Analysis and Processing – ICIAP 2022 (ICIAP 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13231))

Included in the following conference series:

Abstract

Artificial Intelligence for digital REStoration of Cultural Heritage (AIRES-CH) aims at building a web-based app for the digital restoration of pictorial artworks through Computer Vision technologies applied to physical imaging raw data. Physical imaging techniques, such as XRF, PIXE, PIGE, and FTIR, are capable of exploring a wide range of wavelengths providing spectra that are used to infer the chemical composition of the pigments. A multidimensional neural network, specifically designed to automatically restore damaged or hidden pictorial work, will be deployed on the INFN-CHNet Cloud as a web service, freely available to authenticated researchers. In this contribution, we report the status of the project, its current results, the development plans as well as future prospects.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    For other Machine learning approaches in Cultural Heritage, see [7], and references therein.

  2. 2.

    For a review on Image Quality measures, their issues and prospects in the fields, see [6].

  3. 3.

    Actually, the ADC of our XRF detector has \(2^{14} = 16384\) channels; we rebinned the histogram down to 500; the number was chosen by trial and error to be the smallest useful for the Deep learning, and the biggest tolerable for RAM memory consumption while training the algorithm.

References

  1. Ahmetovic, M.: Multi-analytical approach for the study of a XVII century Florentine painting: complementarity and data-crossing of the results of non-invasive diagnostics aimed at attribution and conservation. Master’s thesis, University of Florence (2020)

    Google Scholar 

  2. Albertin, F., et al.: “Ecce Homo’’ by Antonello da Messina, from non-invasive investigations to data fusion and dissemination. Sci. Rep. 11(1), 15868 (2021). https://doi.org/10.1038/s41598-021-95212-2

    Article  Google Scholar 

  3. Alfeld, M.: MA-XRF for historical paintings: state of the art and perspective. Microsc. Microanal. 26(S2), 72–75 (2020)

    Article  Google Scholar 

  4. Bochicchio, L., et al.: Chapter 7 “Art is not science”: a study of materials and techniques in five of Enrico Baj’s nuclear paintings. In: Sgamellotti, A. (ed.) Science and Art: The Contemporary Painted Surface, pp. 139–168. The Royal Society of Chemistry (2020). https://doi.org/10.1039/9781788016384-00139

  5. Bombini, A., et al.: CHNet cloud: an EOSC-based cloud for physical technologies applied to cultural heritages. In: GARR (ed.) Conferenza GARR 2021 - Sostenibile/Digitale. Dati e tecnologie per il futuro, Selected Papers. Associazione Consortium GARR (2021). https://doi.org/10.26314/GARR-Conf21-proceedings-09

  6. Chandler, D.M.: Seven challenges in image quality assessment: past, present, and future research. ISRN Signal Process. 2013 (2013). https://doi.org/10.1155/2013/905685

  7. Fiorucci, M., Khoroshiltseva, M., Pontil, M., Traviglia, A., Del Bue, A., James, S.: Machine learning for cultural heritage: a survey. Pattern Recogn. Lett. 133, 102–108 (2020). https://doi.org/10.1016/j.patrec.2020.02.017

    Article  Google Scholar 

  8. Gagliani, L.: Multi-technique investigations on a XIX century painting for the non-invasive characterization of visible and hidden materials and pictorial layers. Master’s thesis, University of Florence (2020)

    Google Scholar 

  9. Giuntini, L., et al.: Detectors and cultural heritage: the INFN-CHNet experience. Appl. Sci. 11(8) (2021). https://doi.org/10.3390/app11083462

  10. Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press (2016). http://www.deeplearningbook.org

  11. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. CoRR abs/1512.03385 (2015)

    Google Scholar 

  12. Iizuka, S., Simo-Serra, E., Ishikawa, H.: Globally and locally consistent image completion. ACM Trans. Graph. 36(4) (2017). https://doi.org/10.1145/3072959.3073659

  13. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization (2017)

    Google Scholar 

  14. Kleynhans, T., Schmidt Patterson, C.M., Dooley, K.A., Messinger, D.W., Delaney, J.K.: An alternative approach to mapping pigments in paintings with hyperspectral reflectance image cubes using artificial intelligence. Heritage Sci. 8(1), 1–16 (2020). https://doi.org/10.1186/s40494-020-00427-7

    Article  Google Scholar 

  15. Knoll, G.F.: Radiation Detection and Measurement, 4th edn. Wiley, Hoboken (2010)

    Google Scholar 

  16. Kogou, S., Lee, L., Shahtahmassebi, G., Liang, H.: A new approach to the interpretation of XRF spectral imaging data using neural networks. X-Ray Spectrometry 50(4) (2020). https://doi.org/10.1002/xrs.3188

  17. Larsson, G., Maire, M., Shakhnarovich, G.: FractalNet: ultra-deep neural networks without residuals. CoRR abs/1605.07648 (2016)

    Google Scholar 

  18. Licciardi, G.A., Del Frate, F.: Pixel unmixing in hyperspectral data by means of neural networks. IEEE Trans. Geosci. Remote Sens. 49(11), 4163–4172 (2011). https://doi.org/10.1109/TGRS.2011.2160950

    Article  Google Scholar 

  19. Mandò, P.A., Przybyłowicz, W.J.: Particle-Induced X-Ray Emission (PIXE), pp. 1–48. American Cancer Society (2016). https://doi.org/10.1002/9780470027318.a6210.pub3. https://onlinelibrary.wiley.com/doi/abs/10.1002/9780470027318.a6210.pub3

  20. Mazzinghi, A., et al.: MA-XRF for the characterisation of the painting materials and technique of the entombment of Christ by Rogier van der Weyden. Appl. Sci. 11(13) (2021). https://doi.org/10.3390/app11136151

  21. van den Oord, A., et al.: WaveNet: a generative model for raw audio. CoRR abs/1609.03499 (2016)

    Google Scholar 

  22. Pandey, S.K., Shekhawat, H., Prasanna, S.: Emotion recognition from raw speech using wavenet (2019). https://doi.org/10.1109/TENCON.2019.8929257

  23. Ricciardi, P., Mazzinghi, A., Legnaioli, S., Ruberto, C., Castelli, L.: The Choir Books of San Giorgio Maggiore in Venice: results of in depth non-invasive analyses. Heritage 2(2), 1684–1701 (2019). https://doi.org/10.3390/heritage2020103

    Article  Google Scholar 

  24. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. CoRR abs/1505.04597 (2015)

    Google Scholar 

  25. Ruberto, C., et al.: La rete CHNet a servizio di Ottavio Leoni: la diagnostica per la comprensione dei materiali da disegno. In: Leo, S., Olschki editore, F. (eds.) Accademia toscana di scienze e lettere la colombaria. atti e memorie, vol. LXXXV (2020)

    Google Scholar 

  26. Ruberto, C., et al.: Imaging study of Raffaello’s La Muta by a portable XRF spectrometer. Microchem. J. 126, 63–69 (2016). https://doi.org/10.1016/j.microc.2015.11.037

    Article  Google Scholar 

  27. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition (2015)

    Google Scholar 

  28. Szegedy, C., et al.: Going deeper with convolutions. CoRR abs/1409.4842 (2014). http://arxiv.org/abs/1409.4842

  29. Taccetti, F., et al.: A multipurpose X-ray fluorescence scanner developed for in situ analysis. Rendiconti Lincei. Scienze Fisiche e Naturali 30(2), 307–322 (2019). https://doi.org/10.1007/s12210-018-0756-x

    Article  Google Scholar 

  30. Wang, M., Zhao, M., Chen, J., Rahardja, S.: Nonlinear unmixing of hyperspectral data via deep autoencoder networks. IEEE Geosci. Remote Sens. Lett. 16(9), 1467–1471 (2019). https://doi.org/10.1109/LGRS.2019.2900733

    Article  Google Scholar 

  31. Wang, Z., Simoncelli, E., Bovik, A.: Multiscale structural similarity for image quality assessment (2003). https://doi.org/10.1109/ACSSC.2003.1292216

  32. Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004). https://doi.org/10.1109/TIP.2003.819861

    Article  Google Scholar 

  33. Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions (2016)

    Google Scholar 

  34. Zabihi, M., Rad, A.B., Kiranyaz, S., Särkkä, S., Gabbouj, M.: 1D convolutional neural network models for sleep arousal detection (2019)

    Google Scholar 

  35. Zhang, X., Sun, Y., Zhang, J., Wu, P., Jiao, L.: Hyperspectral unmixing via deep convolutional neural networks. IEEE Geosci. Remote Sens. Lett. 15(11), 1755–1759 (2018). https://doi.org/10.1109/LGRS.2018.2857804

    Article  Google Scholar 

  36. Zhao, H., Gallo, O., Frosio, I., Kautz, J.: Loss functions for neural networks for image processing. CoRR abs/1511.08861 (2015)

    Google Scholar 

Download references

Acknowledgements

This research is part of the project AIRES-CH - Artificial Intelligence for digital REStoration of Cultural Heritage (CUP I95F21001120008) jointly funded by Tuscany Region (Progetto Giovani Sì) and INFN.

Author information

Authors and Affiliations

  1. INFN, Florence Section, Via Bruno Rossi 1, 50019, Sesto Fiorentino, FI, Italy

    Alessandro Bombini, Lucio Anderlini, Chiara Ruberto & Francesco Taccetti

  2. INFN CNAF, Viale Carlo Berti Pichat 6, 40127, Bologna, BO, Italy

    Luca dell’Agnello & Francesco Giaocmini

Authors
  1. Alessandro Bombini
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lucio Anderlini
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Luca dell’Agnello
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Francesco Giaocmini
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chiara Ruberto
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Francesco Taccetti
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alessandro Bombini .

Editor information

Editors and Affiliations

  1. Boston University, Boston, MA, USA

    Stan Sclaroff

  2. National Research Council, Lecce, Italy

    Cosimo Distante

  3. National Research Council, Lecce, Italy

    Marco Leo

  4. University of Catania, Catania, Italy

    Giovanni M. Farinella

  5. Technische Universität München, Garching, Germany

    Federico Tombari

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bombini, A., Anderlini, L., dell’Agnello, L., Giaocmini, F., Ruberto, C., Taccetti, F. (2022). The AIRES-CH Project: Artificial Intelligence for Digital REStoration of Cultural Heritages Using Nuclear Imaging and Multidimensional Adversarial Neural Networks. In: Sclaroff, S., Distante, C., Leo, M., Farinella, G.M., Tombari, F. (eds) Image Analysis and Processing – ICIAP 2022. ICIAP 2022. Lecture Notes in Computer Science, vol 13231. Springer, Cham. https://doi.org/10.1007/978-3-031-06427-2_57

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-06427-2_57

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-06426-5

  • Online ISBN: 978-3-031-06427-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation