Abstract
This paper studies the effect of using various data augmentation by synthetization approaches on historical image data, particularly for font classification. Historical document image datasets often lack the appropriate size to train and evaluate deep learning models, motivating data augmentation and synthetic document generation techniques for creating additional data. This work explores the effect of various semi-synthetic and synthetic historical document images, some of which appear as recent trends and others not published yet, on a font classification task. We use 10K patch samples as baseline dataset, derived from the dataset of Early Printed Books with Multiple Font Groups, and increase its size using DocCreator software and Generative Adversarial Networks (GAN). Furthermore, we fine-tune different pre-trained Convolutional Neural Network (CNN) classifiers as a baseline using the original dataset and then compare the performance with the additional semi-synthetic and synthetic images. We further evaluate the performance using additional real samples from the original dataset in the training process. DocCreator, and the additional real samples improve the performance giving the best results. Finally, for the best-performing architecture, we explore different sizes of training sets and examine how the gradual addition of data affects the performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
Ink depot from the body of a type due to face erosion caused by frequent use of this type.
References
Baird, H.: Document image defect models and their uses. In: Proceedings of 2nd International Conference on Document Analysis and Recognition (ICDAR 1993), pp. 62–67 (1993). https://doi.org/10.1109/ICDAR.1993.395781
Capobianco, S., Marinai, S.: DocEmul: a toolkit to generate structured historical documents (2017)
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009). https://doi.org/10.1109/CVPR.2009.5206848
Ditria, L., Meyer, B.J., Drummond, T.: OpenGAN: open set generative adversarial networks (2020)
Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016). https://doi.org/10.1109/CVPR.2016.265
Goodfellow, I.J., et al.: Generative adversarial networks (2014)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016). https://doi.org/10.1109/CVPR.2016.90
Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2261–2269 (2017). https://doi.org/10.1109/CVPR.2017.243
Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks (2018)
Journet, N., Visani, M., Mansencal, B., Van-Cuong, K., Billy, A.: DocCreator: a new software for creating synthetic ground-truthed document images. J. Imaging 3(4), 62 (2017). https://doi.org/10.3390/jimaging3040062
Kieu, V., Visani, M., Journet, N., Domenger, J.P., Mullot, R.: A character degradation model for grayscale ancient document images. In: Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012), pp. 685–688. IEEE (2012)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization (2017)
Kingma, D.P., Welling, M.: Auto-encoding variational Bayes. arXiv preprint arXiv:1312.6114 (2013)
Lombardi, F., Marinai, S.: Deep learning for historical document analysis and recognition—A survey. J. Imaging 6(10), 110 (2020). https://doi.org/10.3390/jimaging6100110
Mayr, M., Stumpf, M., Nicolaou, A., Seuret, M., Maier, A., Christlein, V.: Spatio-temporal handwriting imitation. In: Bartoli, A., Fusiello, A. (eds.) ECCV 2020. LNCS, vol. 12539, pp. 528–543. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-68238-5_38
Mello, C.A.: Synthesis of images of historical documents for web visualization. In: Proceedings of the 10th International Multimedia Modelling Conference, pp. 220–226. IEEE (2004)
Mirza, M., Osindero, S.: Conditional generative adversarial nets (2014)
Pack, C., Liu, Y., Soh, L.K., Lorang, E.M.: Augmentation-based pseudo-groundtruth generation for deep learning in historical document segmentation for greater levels of archival description and access. ACM J. Comput. Cult. Herit. (2022)
Pondenkandath, V., Alberti, M., Diatta, M., Ingold, R., Liwicki, M.: Historical document synthesis with generative adversarial networks. In: 2019 International Conference on Document Analysis and Recognition Workshops (ICDARW), vol. 5, pp. 146–151 (2019). https://doi.org/10.1109/ICDARW.2019.40096
Seuret, M., Chen, K., Eichenbergery, N., Liwicki, M., Ingold, R.: Gradient-domain degradations for improving historical documents images layout analysis. In: 2015 13th International Conference on Document Analysis and Recognition (ICDAR), pp. 1006–1010 (2015). https://doi.org/10.1109/ICDAR.2015.7333913
Seuret, M., Limbach, S., Weichselbaumer, N., Maier, A., Christlein, V.: Dataset of pages from early printed books with multiple font groups. In: Proceedings of the 5th International Workshop on Historical Document Imaging and Processing, HIP 2019, pp. 1–6. Association for Computing Machinery, New York, NY, USA (2019). https://doi.org/10.1145/3352631.3352640
Seuret, M.: ICDAR 2021 competition on historical document classification. In: Lladós, J., Lopresti, D., Uchida, S. (eds.) ICDAR 2021. LNCS, vol. 12824, pp. 618–634. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-86337-1_41
Shamsolmoali, P., et al.: Image synthesis with adversarial networks: a comprehensive survey and case studies. Inf. Fusion 72, 126–146 (2020)
Shorten, C., Khoshgoftaar, T.M.: A survey on image data augmentation for deep learning. J. Big Data 6(1), 1–48 (2019)
Tan, M., Le, Q.V.: EfficientNet: rethinking model scaling for convolutional neural networks (2020)
Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2242–2251 (2017). https://doi.org/10.1109/ICCV.2017.244
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Nikolaidou, K., Upadhyay, R., Seuret, M., Liwicki, M. (2022). Investigating the Effect of Using Synthetic and Semi-synthetic Images for Historical Document Font Classification. In: Uchida, S., Barney, E., Eglin, V. (eds) Document Analysis Systems. DAS 2022. Lecture Notes in Computer Science, vol 13237. Springer, Cham. https://doi.org/10.1007/978-3-031-06555-2_41
Download citation
DOI: https://doi.org/10.1007/978-3-031-06555-2_41
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-06554-5
Online ISBN: 978-3-031-06555-2
eBook Packages: Computer ScienceComputer Science (R0)
