Abstract
When it comes the need to automatically recognize handwritten information, there may be a scenario in which the capture device (such as the camera of a mobile phone) retrieves several samples—a burst—of the same content. Unlike general classification scenarios, combining different inputs is not straightforward for state-of-the-art handwriting recognition approaches based on neural end-to-end formulations. Moreover, since not all images within the burst may depict enough quality for contributing in the overall recognition task, it is hence necessary to select the appropriate subset. In this work, we propose a pilot study which addresses this scenario in which there exists a burst of pictures of the same manuscript to transcribe. Within this context, we present a straightforward strategy to select a single candidate out of the elements of the burst which achieves the best possible transcription. Our hypothesis is that the best source image is that whose transcription most likely matches a Language Model (LM) estimated on the application domain—in this work implemented by means of n-grams. Our strategy, therefore, processes all the images of the burst and selects the transcription with the highest a priori probability according to the LM. Our experiments recreate this scenario in two typical benchmarks of Handwritten Text Recognition (HTR) and Handwritten Music Recognition (HMR), validating the goodness of our proposal and leaving room for promising future work.
This paper is part of the project I+D+i PID2020-118447RA-I00, funded by MCIN/AEI/10.13039/501100011033. The third author is supported by grant APOSTD/2020/256 from “Programa I+D+i de la Generalitat Valenciana”.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
In the case of music documents, the analogous unit is a staff.
References
Calvo-Zaragoza, J., Rico-Juan, J.R., Gallego, A.-J.: Ensemble classification from deep predictions with test data augmentation. Soft. Comput. 24(2), 1423–1433 (2019). https://doi.org/10.1007/s00500-019-03976-7
Calvo-Zaragoza, J., Toselli, A.H., Vidal, E.: Handwritten music recognition for mensural notation: Formulation, data and baseline results. In: 14th IAPR International Conference on Document Analysis and Recognition, ICDAR 2017, Kyoto, Japan, November 9–15, 2017, pp. 1081–1086. IEEE (2017)
Doermann, D., Tombre, K.: Handbook of Document Image Processing and Recognition. Springer, London (2014). https://doi.org/10.1007/978-0-85729-859-1
Dutta, K., Krishnan, P., Mathew, M., Jawahar, C.: Improving cnn-rnn hybrid networks for handwriting recognition. In: 16th International Conference on Frontiers in Handwriting Recognition, pp. 80–85. IEEE (2018)
Faundez-Zanuy, M.: On-line signature recognition based on VQ-DTW. Pattern Recogn. 40(3), 981–992 (2007)
Fischer, A., Keller, A., Frinken, V., Bunke, H.: Lexicon-free handwritten word spotting using character HMMs. Pattern Recogn. Lett. 33(7), 934–942 (2012)
Granell, E., Martinez-Hinarejos, C.D.: Multimodal crowdsourcing for transcribing handwritten documents. IEEE/ACM Trans. Audio Speech Lang. Process. 25(2), 409–419 (2016)
Graves, A.: Supervised sequence labelling. In: Supervised Sequence Labelling with Recurrent Neural Networks, vol. 385, pp. 5–13. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-24797-2_2
Graves, A., Fernández, S., Gomez, F., Schmidhuber, J.: Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In: Proceedings of the 23rd International Conference on Machine Learning. ICML 2006, New York, NY, USA, pp. 369–376. ACM (2006)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: 3rd International Conference on Learning Representations. San Diego, USA (2015)
Levenshtein, V.I., et al.: Binary codes capable of correcting deletions, insertions, and reversals. In: Soviet Physics Doklady, vol. 10, pp. 707–710. Soviet Union (1966)
López-Gutiérrez, J.C., Valero-Mas, J.J., Castellanos, F.J., Calvo-Zaragoza, J.: Data augmentation for end-to-end optical music recognition. In: Barney Smith, E.H., Pal, U. (eds.) ICDAR 2021. LNCS, vol. 12916, pp. 59–73. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-86198-8_5
Rehman, A., Naz, S., Razzak, M.I.: Writer identification using machine learning approaches: a comprehensive review. Multimedia Tools Appl. 78(8), 10889–10931 (2018). https://doi.org/10.1007/s11042-018-6577-1
Sánchez, J., Romero, V., Toselli, A.H., Villegas, M., Vidal, E.: A set of benchmarks for handwritten text recognition on historical documents. Pattern Recognit. 94, 122–134 (2019)
Shi, B., Bai, X., Yao, C.: An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition. IEEE Trans. Pattern Anal. Mach. Intell. 39(11), 2298–2304 (2017)
Wigington, C., Stewart, S., Davis, B., Barrett, B., Price, B., Cohen, S.: Data augmentation for recognition of handwritten words and lines using a CNN-LSTM network. In: 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), vol. 1, pp. 639–645. IEEE (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Rosello, A., Ayllon, E., Valero-Mas, J.J., Calvo-Zaragoza, J. (2022). Test Sample Selection for Handwriting Recognition Through Language Modeling. In: Pinho, A.J., Georgieva, P., Teixeira, L.F., Sánchez, J.A. (eds) Pattern Recognition and Image Analysis. IbPRIA 2022. Lecture Notes in Computer Science, vol 13256. Springer, Cham. https://doi.org/10.1007/978-3-031-04881-4_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-04881-4_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-04880-7
Online ISBN: 978-3-031-04881-4
eBook Packages: Computer ScienceComputer Science (R0)
