Abstract
Sequence to sequence models have shown significant progress in the field of handwriting recognition. The recent trend has been that the input to these models is fed from a convolutional neural network (CNN) that acts as a generic feature extractor for the handwritten text images. The input to the CNN is usually either a sequence of patches extracted from the text image or the output of a segmentation algorithm applied on the text image to break it up into individual characters. However, patching is unable to convey proper information about character boundaries in the image, and the segmentation-based approach often suffers from over and under segmentation. To this end, we propose a fuzzy-based pseudo segmentation approach for handwritten word recognition using a sequence to sequence model. We use a fuzzy triangular function that generates column wise weights based on the distance of the nearest data pixel from the top of the text image. Thus the probable segmentation regions are assigned higher weights than the other regions. These weights are superimposed on the original text image and this modified input is fed patch wise to the CNN. The features extracted are then encoded as a first part of a sequence to sequence model and then decoded to obtain the sequence of characters in the input. An attention mechanism is used to ensure that the decoder focuses on the appropriate section of the features outputted by the encoder while generating each character. Our method is tested upon the IAM word database after the words in it undergo skew and slant correction. The architecture of the model is optimized based on exhaustive numerical simulations on the IAM database and it shows promising results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Madhvanath, S., Govindaraju, V., Member, S.: The role of holistic paradigms in handwritten word recognition. IEEE Trans. Pattern Anal. Mach. Intell. 23, 149–164 (2001)
Ghosh, S., Bhattacharya, R., Majhi, S., Bhowmik, S., Malakar, S., Sarkar, R.: Textual content retrieval from filled-in form images. In: Sundaram, S., Harit, G. (eds.) DAR 2018. CCIS, vol. 1020, pp. 27–37. Springer, Singapore (2019). https://doi.org/10.1007/978-981-13-9361-7_3
Bhattacharya, R., Malakar, S., Ghosh, S., Bhowmik, S., Sarkar, R.: Understanding contents of filled-in Bangla form images. Multimed. Tools Appl. 1–42 (2020)
Roy, P.P., Bhunia, A.K., Das, A., Dey, P., Pal, U.: HMM-based Indic handwritten word recognition using zone segmentation. Pattern Recognit. 60, 1057–1075 (2016). https://doi.org/10.1016/j.patcog.2016.04.012
Bunke, H., Bengio, S., Vinciarelli, A.: Offline recognition of unconstrained handwritten texts using HMMs and statistical language models. IEEE Trans. Pattern Anal. Mach. Intell. 26, 709–720 (2004)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86, 2278–2324 (1998)
Menasri, F., Louradour, J., Bianne-Bernard, A.-L., Kermorvant, C.: The A2iA French handwriting recognition system at the Rimes-ICDAR2011 competition. In: Document Recognition and Retrieval XIX, p. 82970Y. International Society for Optics and Photonics (2012)
Graves, A., Schmidhuber, J.: Offline handwriting recognition with multidimensional recurrent neural networks. In: Advances in Neural Information Processing Systems, pp. 545–552 (2009)
Bluche, T., Ney, H., Kermorvant, C.: A comparison of sequence-trained deep neural networks and recurrent neural networks optical modeling for handwriting recognition. In: Besacier, L., Dediu, A.-H., Martín-Vide, C. (eds.) SLSP 2014. LNCS (LNAI), vol. 8791, pp. 199–210. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-11397-5_15
Doetsch, P., Kozielski, M., Ney, H.: Fast and robust training of recurrent neural networks for offline handwriting recognition. In: 2014 14th International Conference on Frontiers in Handwriting Recognition, pp. 279–284. IEEE (2014)
Sueiras, J., Ruiz, V., Sanchez, A., Velez, J.F.: Offline continuous handwriting recognition using sequence to sequence neural networks. Neurocomputing 289, 119–128 (2018)
Almazán, J., Gordo, A., Fornés, A., Valveny, E.: Word spotting and recognition with embedded attributes. IEEE Trans. Pattern Anal. Mach. Intell. 36, 2552–2566 (2014)
Poznanski, A., Wolf, L.: Cnn-n-gram for handwriting word recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2305–2314 (2016)
Bluche, T., Louradour, J., Messina, R.: Scan, attend and read: End-to-end handwritten paragraph recognition with mdlstm attention. In: 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), pp. 1050–1055. IEEE (2017)
Majid, N., Smith, E.H.B.: Segmentation-free bangla offline handwriting recognition using sequential detection of characters and diacritics with a Faster R-CNN. In: 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 228–233. IEEE (2019)
Märgner, V., El Abed, H.: ICDAR 2009 Arabic handwriting recognition competition. In: 2009 10th International Conference on Document Analysis and Recognition, pp. 1383–1387. IEEE (2009)
Stahlberg, F., Vogel, S.: The QCRI recognition system for handwritten Arabic. In: Murino, V., Puppo, E. (eds.) ICIAP 2015. LNCS, vol. 9280, pp. 276–286. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-23234-8_26
Gui, L., Liang, X., Chang, X., Hauptmann, A.G.: Adaptive context-aware reinforced agent for handwritten text recognition. In: BMVC, p. 207 (2018)
Wu, X., Chen, Q., You, J., Xiao, Y.: Unconstrained offline handwritten word recognition by position embedding integrated resnets model. IEEE Signal Process. Lett. 26, 597–601 (2019)
Basu, S., Sarkar, R., Das, N., Kundu, M., Nasipuri, M., Basu, D.K.: A fuzzy technique for segmentation of handwritten Bangla word images. In: Proceedings-International Conference on Computing: Theory and Applications, ICCTA 2007, pp. 427–432. IEEE (2007). https://doi.org/10.1109/ICCTA.2007.7
Sarkar, R., Das, N., Basu, S., Kundu, M., Nasipuri, M., Basu, D.K.: A two-stage approach for segmentation of handwritten Bangla word images. In: Proceedings of International Conference on Frontiers in Handwriting Recognitions, pp. 403–408. Citeseer (2008)
Malakar, S., Sarkar, R., Basu, S., Kundu, M., Nasipuri, M.: An image database of handwritten Bangla words with automatic benchmarking facilities for character segmentation algorithms. Neural Comput. Appl. (2020)
Bera, S.K., et al.: A one-pass approach for slope and slant estimation of tri-script handwritten words. J. Intell. Syst. (2018). https://doi.org/10.1515/jisys-2018-0105
Otsu, N.: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man. Cybern. 9, 62–66 (1979)
Chorowski, J.K., Bahdanau, D., Serdyuk, D., Cho, K., Bengio, Y.: Attention-based models for speech recognition. In: Advances in Neural Information Processing Systems, pp. 577–585 (2015)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv Prepr. arXiv1409.1556 (2014)
Bluche, T., Ney, H., Kermorvant, C.: Tandem HMM with convolutional neural network for handwritten word recognition. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 2390–2394. IEEE (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Bhattacharya, R., Malakar, S., Schwenker, F., Sarkar, R. (2021). Fuzzy-Based Pseudo Segmentation Approach for Handwritten Word Recognition Using a Sequence to Sequence Model with Attention. In: Del Bimbo, A., et al. Pattern Recognition. ICPR International Workshops and Challenges. ICPR 2021. Lecture Notes in Computer Science(), vol 12662. Springer, Cham. https://doi.org/10.1007/978-3-030-68790-8_45
Download citation
DOI: https://doi.org/10.1007/978-3-030-68790-8_45
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-68789-2
Online ISBN: 978-3-030-68790-8
eBook Packages: Computer ScienceComputer Science (R0)
