Abstract
The established paradigm in handwriting recognition techniques involves supervised learning, where training is performed over fully labelled (transcribed) data. In this paper, we propose a weak supervision technique that involves tuning the trained network to perform better on a specific test set, after having completed its training with the standard training data. The proposed technique is based on the notion of a reference model, to which we refer as the “Oracle”, which is used for test data inference and retraining in an iterative manner. As test data that are erroneously labelled will be a hindrance to model retraining, we explore ways to properly weight Oracle labels. The proposed method is shown to improve model performance as much as \(2\%\) for Character Error Rate and \(5\%\) for Word Error Rate. Combined with a competitive convolutional-recurrent architecture, we achieve state-of-the-art recognition results in the IAM and RIMES datasets.
This research has been partially co-financed by the EU and Greek national funds through the Operational Program Competitiveness, Entrepreneurship and Innovation, under the call OPEN INNOVATION IN CULTURE, project Bessarion (T6YB\(\varPi \)-00214).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
A detailed analysis on sequence-level knowledge distillation has been presented in [10].
- 2.
Note that during evaluation we do not take into account these added spaces, as this would bias the result in favour of our method.
References
Chowdhury, A., Vig, L.: An efficient end-to-end neural model for handwritten text recognition. In: British Machine Vision Conference (BMVC) (2018)
Cozman, F.G., Cohen, I., Cirelo, M.: Unlabeled data can degrade classification performance of generative classifiers. In: Flairs Conference, pp. 327–331 (2002)
Dutta, K., Krishnan, P., Mathew, M., Jawahar, C.: Improving CNN-RNN hybrid networks for handwriting recognition. In: 2018 16th International Conference on Frontiers in Handwriting Recognition (ICFHR), pp. 80–85. IEEE (2018)
Frinken, V., Bunke, H.: Self-training strategies for handwriting word recognition. In: Perner, P. (ed.) ICDM 2009. LNCS (LNAI), vol. 5633, pp. 291–300. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-03067-3_23
Giotis, A.P., Sfikas, G., Nikou, C., Gatos, B.: Shape-based word spotting in handwritten document images. In: 13th International Conference on Document Analysis and Recognition (ICDAR), pp. 561–565. IEEE (2015)
Granet, A., Morin, E., Mouchère, H., Quiniou, S., Viard-Gaudin, C.: Transfer learning for a letter-ngrams to word decoder in the context of historical handwriting recognition with scarce resources. In: 27th International Conference on Computational Linguistics (COLING), pp. 1474–1484 (2018)
Grosicki, E., Carre, M., Brodin, J.M., Geoffrois, E.: Rimes evaluation campaign for handwritten mail processing (2008)
Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015)
Keret, S., Wolf, L., Dershowitz, N., Werner, E., Almogi, O., Wangchuk, D.: Transductive learning for reading handwritten tibetan manuscripts. In: 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 214–221. IEEE (2019)
Kim, Y., Rush, A.M.: Sequence-level knowledge distillation. arXiv preprint arXiv:1606.07947 (2016)
Krishnan, P., Dutta, K., Jawahar, C.: Word spotting and recognition using deep embedding. In: 2018 13th IAPR International Workshop on Document Analysis Systems (DAS), pp. 1–6. IEEE (2018)
Lefakis, L.: Transductive learning for document classification and handwritten character recognition. Ph.D. thesis, Utrecht university (2008)
Loshchilov, I., Hutter, F.: SGDR: stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983 (2016)
Markou, K., et al.: A convolutional recurrent neural network for the handwritten text recognition of historical Greek manuscripts. In: International Workshop on Pattern Recognition for Cultural Heritage (PATRECH) (2020)
Marti, U.V., Bunke, H.: The IAM-database: an English sentence database for offline handwriting recognition. Int. J. Doc. Anal. Recogn. 5(1), 39–46 (2002)
Michael, J., Labahn, R., Grüning, T., Zöllner, J.: Evaluating sequence-to-sequence models for handwritten text recognition. In: 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 1286–1293. IEEE (2019)
Mühlberger, G., et al.: Transforming scholarship in the archives through handwritten text recognition. J. Doc. 75(5), 954–976 (2019)
Puigcerver, J.: Are multidimensional recurrent layers really necessary for handwritten text recognition? In: 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), vol. 1, pp. 67–72. IEEE (2017)
Retsinas, G., Louloudis, G., Stamatopoulos, N., Gatos, B.: Efficient learning-free keyword spotting. IEEE Trans. Pattern Anal. Mach. Intell. 41(7), 1587–1600 (2018)
Retsinas, G., Sfikas, G., Gatos, B.: Transferable deep features for keyword spotting. In: Multidisciplinary Digital Publishing Institute Proceedings, vol. 2, p. 89 (2018)
Sfikas, G., Gatos, B., Nikou, C.: Semicca: a new semi-supervised probabilistic CCA model for keyword spotting. In: 2017 IEEE International Conference on Image Processing (ICIP), pp. 1107–1111. IEEE (2017)
Sfikas, G., Giotis, A.P., Louloudis, G., Gatos, B.: Using attributes for word spotting and recognition in polytonic Greek documents. In: 2015 13th International Conference on Document Analysis and Recognition (ICDAR), pp. 686–690. IEEE (2015)
Sfikas, G., Nikou, C., Galatsanos, N., Heinrich, C.: Majorization-minimization mixture model determination in image segmentation. In: CVPR 2011, pp. 2169–2176. IEEE (2011)
Tassopoulou, V., Retsinas, G., Maragos, P.: Enhancing handwritten text recognition with n-gram sequence decomposition and multitask learning. arXiv preprint arXiv:2012.14459 (2020)
Tensmeyer, C., Wigington, C., Davis, B., Stewart, S., Martinez, T., Barrett, W.: Language model supervision for handwriting recognition model adaptation. In: 2018 16th International Conference on Frontiers in Handwriting Recognition (ICFHR), pp. 133–138. IEEE (2018)
Xiao, S., Peng, L., Yan, R., Wang, S.: Deep network with pixel-level rectification and robust training for handwriting recognition. SN Comput. Sci. 1(3), 1–13 (2020)
Yousef, M., Hussain, K.F., Mohammed, U.S.: Accurate, data-efficient, unconstrained text recognition with convolutional neural networks. Pattern Recogn. 108, 107482 (2020)
Zhou, Z.H.: A brief introduction to weakly supervised learning. Natl. Sci. Rev. 5(1), 44–53 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Retsinas, G., Sfikas, G., Nikou, C. (2021). Iterative Weighted Transductive Learning for Handwriting Recognition. In: Lladós, J., Lopresti, D., Uchida, S. (eds) Document Analysis and Recognition – ICDAR 2021. ICDAR 2021. Lecture Notes in Computer Science(), vol 12824. Springer, Cham. https://doi.org/10.1007/978-3-030-86337-1_39
Download citation
DOI: https://doi.org/10.1007/978-3-030-86337-1_39
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-86336-4
Online ISBN: 978-3-030-86337-1
eBook Packages: Computer ScienceComputer Science (R0)
