Abstract
In recent years, machine learning and deep learning approaches such as artificial neural networks have gained in popularity for the resolution of automatic puzzle resolution problems. Indeed, these methods are able to extract high-level representations from images, and then can be trained to separate matching image pieces from non-matching ones. These applications have many similarities to the problem of ancient document reconstruction from partially recovered fragments. In this work we present a solution based on a Graph Neural Network, using pairwise patch information to assign labels to edges representing the spatial relationships between pairs. This network classifies the relationship between a source and a target patch as being one of Up, Down, Left, Right or None. By doing so for all edges, our model outputs a new graph representing a reconstruction proposal. Finally, we show that our model is not only able to provide correct classifications at the edge-level, but also to generate partial or full reconstruction graphs from a set of patches.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Papyrology Collection. https://www.lib.umich.edu/collections/collecting-areas/special-collections-and-archives/papyrology-collection
Brodersen, K.H., Ong, C.S., Stephan, K.E., Buhmann, J.M.: The balanced accuracy and its posterior distribution. In: 2010 20th International Conference on Pattern Recognition, pp. 3121–3124. IEEE (2010)
Cho, T.S., Avidan, S., Freeman, W.T.: A probabilistic image jigsaw puzzle solver. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 183–190. IEEE (2010)
Fey, M., Lenssen, J.E.: Fast graph representation learning with PyTorch Geometric. In: ICLR Workshop on Representation Learning on Graphs and Manifolds (2019)
Franz, M., Lopes, C.T., Huck, G., Dong, Y., Sumer, O., Bader, G.D.: Cytoscape.js: a graph theory library for visualisation and analysis. Bioinformatics 32(2), 309–311 (2016)
Jin, S.Y., Lee, S., Azis, N.A., Choi, H.J.: Jigsaw puzzle image retrieval via pairwise compatibility measurement. In: 2014 International Conference on Big Data and Smart Computing (BIGCOMP), pp. 123–127. IEEE (2014)
Kelleher, J.D., Mac Namee, B., D’arcy, A.: Fundamentals of Machine Learning for Predictive Data Analytics: Algorithms, Worked Examples, and Case Studies. MIT Press, Cambridge (2020)
Kleber, F., Sablatnig, R.: A survey of techniques for document and archaeology artefact reconstruction. In: 2009 10th International Conference on Document Analysis and Recognition, pp. 1061–1065. IEEE (2009)
Mondal, D., Wang, Y., Durocher, S.: Robust solvers for square jigsaw puzzles. In: 2013 International Conference on Computer and Robot Vision, pp. 249–256. IEEE (2013)
Ostertag, C., Beurton-Aimar, M.: Matching ostraca fragments using a Siamese neural network. Pattern Recogn. Lett. 131, 336–340 (2020)
Paikin, G., Tal, A.: Solving multiple square jigsaw puzzles with missing pieces. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4832–4839 (2015)
Paumard, M.M., Picard, D., Tabia, H.: Jigsaw puzzle solving using local feature co-occurrences in deep neural networks. In: 2018 25th IEEE International Conference on Image Processing (ICIP), pp. 1018–1022. IEEE (2018)
Pirrone, A., Aimar, M.B., Journet, N.: Papy-S-Net: a Siamese network to match papyrus fragments. In: Proceedings of the 5th International Workshop on Historical Document Imaging and Processing, pp. 78–83 (2019)
Rasheed, N.A., Nordin, M.J.: A survey of classification and reconstruction methods for the 2D archaeological objects. In: 2015 International Symposium on Technology Management and Emerging Technologies (ISTMET), pp. 142–147. IEEE (2015)
Wadhwani, M., Kundu, D., Chakraborty, D., Chanda, B.: Text extraction and restoration of old handwritten documents. arXiv preprint arXiv:2001.08742 (2020)
Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., Philip, S.Y.: A comprehensive survey on graph neural networks. IEEE Trans. Neural Netw. Learn. Syst. 32, 4–24 (2020)
Acknowledgements
This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innova.on programme (grant agreement No. 758907).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Ostertag, C., Beurton-Aimar, M. (2021). Using Graph Neural Networks to Reconstruct Ancient Documents. In: Del Bimbo, A., et al. Pattern Recognition. ICPR International Workshops and Challenges. ICPR 2021. Lecture Notes in Computer Science(), vol 12667. Springer, Cham. https://doi.org/10.1007/978-3-030-68787-8_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-68787-8_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-68786-1
Online ISBN: 978-3-030-68787-8
eBook Packages: Computer ScienceComputer Science (R0)
