Abstract
This paper is about the creation of an interactive software tool and dataset useful for exploring the unindexed 11-volume set, Pompei: Pitture e Mosaici (PPM), a valuable resource containing over 20,000 annotated historical images of the archaeological site of Pompeii, Italy. The tool includes functionalities such as a word search, and an images and captions similarity search. Searches for similarity are conducted using transfer learning on the data retrieved from the scanned version of PPM. Image processing, convolutional neural networks and natural language processing also had to come into play to extract, classify, and archive the text and image data from the digitized version of the books.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aletras, N., Stevenson, M., Clough, P.: Computing similarity between items in a digital library of cultural heritage. J. Comput. Cult. Heritage (JOCCH) 5(4), 1–19 (2013)
Arnold, T., Tilton, L.: Enriching historic photography with structured data using image region segmentation. In: Proceedings of the 1st International Workshop on Artificial Intelligence for Historical Image Enrichment and Access, pp. 1–10 (2020)
Becattini, F., Ferracani, A., Landucci, L., Pezzatini, D., Uricchio, T., Del Bimbo, A.: Imaging novecento. A mobile app for automatic recognition of artworks and transfer of artistic styles. In: Ioannides, M., et al. (eds.) EuroMed 2016. LNCS, vol. 10058, pp. 781–791. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-48496-9_62
Belhi, A., Bouras, A., Foufou, S.: Leveraging known data for missing label prediction in cultural heritage context. Appl. Sci. 8(10), 1768 (2018)
Díaz-Rodríguez, N., Pisoni, G.: Accessible cultural heritage through explainable artificial intelligence. In: 11th Workshop on Personalized Access to Cultural Heritage (2020)
Gultom, Y., Arymurthy, A.M., Masikome, R.J.: Batik classification using deep convolutional network transfer learning. J. Ilmu Komputer dan Inf. 11(2), 59–66 (2018)
Jboor, N.H., et al.: Towards an inpainting framework for visual cultural heritage. In: 2019 IEEE Jordan International Joint Conference on Electrical Engineering and Information Technology (JEEIT), pp. 602–607. IEEE (2019)
Kulkarni, U., et al.: Classification of cultural heritage sites using transfer learning. In: 2019 IEEE Fifth International Conference on Multimedia Big Data (BigMM), pp. 391–397. IEEE (2019)
Liu, Y., et al.: A many-objective evolutionary algorithm using a one-by-one selection strategy. IEEE Trans. Cybern. 47(9), 2689–2702 (2017)
Palma, V: Towards deep learning for architecture: amonument recognition mobile app. In: International Archives of the Photogrammetry, Remote Sensing & Spatial Information Sciences (2019)
Roullet, C., et al.: An automated technique to recognize and extract images from scanned archaeological documents. In: 2019 International Conference on Document Analysis and Recognition Workshops (ICDARW), vol. 1, pp. 20–25. IEEE (2019)
Schulman, A.J., Mitchell, R.R.: Operating characteristics from yes-no and forced-choice procedures. J. Acoust. Soc. Am. 40(2), 473–477 (1966)
Szegedy, C., et al.: Inception-v4, inception-resnet and the impact of residual connections on learning. arXiv preprint arXiv: 1602.07261 (2016)
Szegedy, C., et al.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818–2826 (2016)
Tan, C., Sun, F., Kong, T., Zhang, W., Yang, C., Liu, C.: A survey on deep transfer learning. In: Kůrková, V., Manolopoulos, Y., Hammer, B., Iliadis, L., Maglogiannis, I. (eds.) ICANN 2018. LNCS, vol. 11141, pp. 270–279. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01424-7_27
Trstenjak, B., Mikac, S., Donko, D.: KNN with TF-IDF based framework for text categorization. Proc. Eng. 69, 1356–1364 (2014)
Wang, J., et al.: Text similarity calculation method based on hybrid model of LDA and TF-IDF. In: Proceedings of the 2019 3rd International Conference on Computer Science and Artificial Intelligence, pp. 1–8 (2019)
Wevers, M., Smits, T.: The visual digital turn: using neural networks to study historical images. Digit. Scholarsh. Hum. 35(1), 194–207 (2020)
Zohar, M., Shimshoni, I., Khateb, F.: GIScience Integrated with Computer Vision for the Interpretation and Analysis of Old Paintings. In: GISTAM, pp. 233–239 (2020)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Roullet, C., Fredrick, D., Gauch, J., Vennarucci, R.G., Loder, W. (2021). Transfer Learning Methods for Extracting, Classifying and Searching Large Collections of Historical Images and Their Captions. In: Del Bimbo, A., et al. Pattern Recognition. ICPR International Workshops and Challenges. ICPR 2021. Lecture Notes in Computer Science(), vol 12667. Springer, Cham. https://doi.org/10.1007/978-3-030-68787-8_13
Download citation
DOI: https://doi.org/10.1007/978-3-030-68787-8_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-68786-1
Online ISBN: 978-3-030-68787-8
eBook Packages: Computer ScienceComputer Science (R0)
