Abstract
Artificial Intelligence for digital REStoration of Cultural Heritage (AIRES-CH) aims at building a web-based app for the digital restoration of pictorial artworks through Computer Vision technologies applied to physical imaging raw data.
In previous work [7], it was shown that it is possible to develop a multidimensional deep neural network capable of inferring the RGB image from an X-Ray Fluorescence raw data.
The developed network comprises two branches: a one-dimensional branch, which works pixel-by-pixel, and a two-dimensional branch, capable of performing image segmentation.
In this project, we report the results of the hyperparameter optimisation of both branches.
This research is part of the project AIRES-CH - Artificial Intelligence for digital REStoration of Cultural Heritage (CUP I95F21001120008) jointly funded by Tuscany Region (Progetto Giovani Sì) and INFN.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
For other Machine learning approaches in Cultural Heritage, see [9], and references therein.
- 2.
The tensor \(\mathcal I_{i,j}\) contains, for each pixel (i, j), the histogram of the counts in bins of the X photon energies. It is customary to convert the raw data into elemental maps by integrating the per-pixel energy spectra around the characteristic energy peak associated to each element [15, 19].
- 3.
Which means each energy bin is \((38.5 - 0.5)\) KeV/500 = 0.076 KeV = 76 eV wide.
- 4.
For more theoretical details, see [21]. Link to paper.
- 5.
i.e., if \(\mathbb {W}\) indicates the individual layer weight matrix, \(\rho \) is the density of the eigenvalues of
$$ \mathbb {X} = \frac{1}{N} \, \mathbb {W}^T \mathbb {W}. $$.
- 6.
References
Ahmetovic, M.: Multi-analytical approach for the study of a XVII century Florentine painting: complementarity and data-crossing of the results of non-invasive diagnostics aimed at attribution and conservation. Master’s thesis, University of Florence (2020)
Akiba, T., Sano, S., Yanase, T., Ohta, T., Koyama, M.: Optuna: a next-generation hyperparameter optimization framework (2019). https://doi.org/10.48550/ARXIV.1907.10902, https://arxiv.org/abs/1907.10902
Akiba, T., Sano, S., Yanase, T., Ohta, T., Koyama, M.: Optuna: a next-generation hyperparameter optimization framework. In: Proceedings of the 25rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2019)
Albertin, F., et al.: “Ecce Homo” by Antonello da Messina, from non-invasive investigations to data fusion and dissemination. Sci. Rep. 11(1), 15868 (2021). https://doi.org/10.1038/s41598-021-95212-2
Alfeld, M.: MA-XRF for historical paintings: state of the art and perspective. Microscopy Microanalysis 26(S2), 72–75 (2020)
Bochicchio, L., et al.: Chapter 7 “Art is Not Science”: a study of materials and techniques in five of Enrico Baj’s nuclear paintings. In: Sgamellotti, A. (ed.) Science and Art: The Contemporary Painted Surface, pp. 139–168. The Royal Society of Chemistry (2020). https://doi.org/10.1039/9781788016384-00139
Bombini, A., Anderlini, L., dell’Agnello, L., Giacomini, F., Ruberto, C., Taccetti, F.: The AIRES-CH project: artificial Intelligence for digital REStoration of Cultural Heritages using physical imaging and multidimensional adversarial neural networks. Accepted for Publication on the ICIAP2021 Conference Proceedings (2021)
Bombini, A., et al.: CHNet cloud: an EOSC-based cloud for physical technologies applied to cultural heritages. In: GARR (ed.) Conferenza GARR 2021 - Sostenibile/Digitale. Dati e tecnologie per il futuro, vol. Selected Papers. Associazione Consortium GARR (2021). https://doi.org/10.26314/GARR-Conf21-proceedings-09
Fiorucci, M., Khoroshiltseva, M., Pontil, M., Traviglia, A., Del Bue, A., James, S.: Machine learning for cultural heritage: a survey. Pattern Recogn. Lett. 133, 102–108 (2020). https://doi.org/10.1016/j.patrec.2020.02.017
Gagliani, L.: Multi-technique investigations on a XIX century painting for the non-invasive characterization of visible and hidden materials and pictorial layers. Master’s thesis, University of Florence (2020)
Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016). http://www.deeplearningbook.org
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. CoRR abs/1512.03385 (2015)
Iizuka, S., Simo-Serra, E., Ishikawa, H.: Globally and locally consistent image completion. ACM Trans. Graph. 36(4) (2017). https://doi.org/10.1145/3072959.3073659
Kleynhans, T., Schmidt Patterson, C.M., Dooley, K.A., Messinger, D.W., Delaney, J.K.: An alternative approach to mapping pigments in paintings with hyperspectral reflectance image cubes using artificial intelligence. Heritage Sci. 8(1), 1–16 (2020). https://doi.org/10.1186/s40494-020-00427-7
Knoll, G.F.: Radiation Detection and Measurement, 4 edn. Wiley, Hoboken (2010)
Kogou, S., Lee, L., Shahtahmassebi, G., Liang, H.: A new approach to the interpretation of XRF spectral imaging data using neural networks. X-Ray Spectrometry 50(4) (2020). https://doi.org/10.1002/xrs.3188
Larsson, G., Maire, M., Shakhnarovich, G.: FractalNet: ultra-deep neural networks without residuals. CoRR abs/1605.07648 (2016)
Licciardi, G.A., Del Frate, F.: Pixel unmixing in hyperspectral data by means of neural networks. IEEE Trans. Geosci. Remote Sens. 49(11), 4163–4172 (2011). https://doi.org/10.1109/TGRS.2011.2160950
Mandò, P.A., Przybyłowicz, W.J.: Particle-Induced X-Ray Emission (PIXE), pp. 1–48. American Cancer Society (2016). https://doi.org/10.1002/9780470027318.a6210.pub3, https://onlinelibrary.wiley.com/doi/abs/10.1002/9780470027318.a6210.pub3
Martin, C.H., Mahoney, M.W.: Heavy-tailed universality predicts trends in test accuracies for very large pre-trained deep neural networks (2019). https://doi.org/10.48550/ARXIV.1901.08278, https://arxiv.org/abs/1901.08278
Martin, C.H., Mahoney, M.W.: Universality and Capacity Metrics in Deep Neural Networks (2019)
Martin, C.H., Mahoney, M.W.: Implicit self-regularization in deep neural networks: evidence from random matrix theory and implications for learning. J. Mach. Learn. Res. 22, 165:1–165:73 (2021)
Martin, C.H., Peng, T.S., Mahoney, M.W.: Predicting trends in the quality of state-of-the-art neural networks without access to training or testing data. Nat. Commun. 12(1), 4122 (2021)
Mazzinghi, A., et al.: MA-XRF for the characterisation of the painting materials and technique of the entombment of christ by Rogier van der Weyden. Appl. Sci. 11(13) (2021). https://doi.org/10.3390/app11136151
van den Oord, A., et al.: WaveNet: a generative model for raw audio. CoRR abs/1609.03499 (2016)
Ricciardi, P., Mazzinghi, A., Legnaioli, S., Ruberto, C., Castelli, L.: The choir books of San Giorgio Maggiore in Venice: results of in depth non-invasive analyses. Heritage 2(2), 1684–1701 (2019). https://doi.org/10.3390/heritage2020103
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. CoRR abs/1505.04597 (2015)
Ruberto, C., et al.: La rete CHNet a servizio di Ottavio Leoni: la diagnostica per la comprensione dei materiali da disegno. In: Leo S. Olschki editore, F. (ed.) Accademia toscana di scienze e lettere la colombaria. atti e memorie, vol. LXXXV (2020)
Ruberto, C., et al.: Imaging study of Raffaello’s La Muta by a portable XRF spectrometer. Microchem. J. 126, 63–69 (2016). https://doi.org/10.1016/j.microc.2015.11.037
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition (2015)
Szegedy, C., et al.: Going deeper with convolutions. CoRR abs/1409.4842 (2014), http://arxiv.org/abs/1409.4842
Wang, M., Zhao, M., Chen, J., Rahardja, S.: Nonlinear unmixing of hyperspectral data via deep autoencoder networks. IEEE Geosci. Remote Sens. Lett. 16(9), 1467–1471 (2019). https://doi.org/10.1109/LGRS.2019.2900733
Wang, Z., Simoncelli, E., Bovik, A.: Multiscale structural similarity for image quality assessment (2003). https://doi.org/10.1109/ACSSC.2003.1292216
Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004). https://doi.org/10.1109/TIP.2003.819861
Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions (2016)
Zhang, X., Sun, Y., Zhang, J., Wu, P., Jiao, L.: Hyperspectral unmixing via deep convolutional neural networks. IEEE Geosci. Remote Sens. Lett. 15(11), 1755–1759 (2018). https://doi.org/10.1109/LGRS.2018.2857804
Zhao, H., Gallo, O., Frosio, I., Kautz, J.: Loss functions for neural networks for image processing. CoRR abs/1511.08861 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Bombini, A., Anderlini, L., dell’Agnello, L., Giacomini, F., Ruberto, C., Taccetti, F. (2022). Hyperparameter Optimisation of Artificial Intelligence for Digital REStoration of Cultural Heritages (AIRES-CH) Models. In: Gervasi, O., Murgante, B., Misra, S., Rocha, A.M.A.C., Garau, C. (eds) Computational Science and Its Applications – ICCSA 2022 Workshops. ICCSA 2022. Lecture Notes in Computer Science, vol 13377. Springer, Cham. https://doi.org/10.1007/978-3-031-10536-4_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-10536-4_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-10535-7
Online ISBN: 978-3-031-10536-4
eBook Packages: Computer ScienceComputer Science (R0)