Skip to main content
SpringerLink
Log in

A recurrent neural network based deep learning model for text and non-text stroke classification in online handwritten Devanagari document

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

It is very common for human beings to include both text and non-text data in a handwritten document. Text portion contains alphabets, digits, and mathematical symbols, whereas non-text portion includes various graphical entities like flow chart, transition diagram, etc. This paper proposes a novel method for online handwritten text and non-text stroke classification with text written in Devanagari script, the most popular script in India, using two different architectures of artificial Recurrent Neural Network (RNN)—long-short term memory (LSTM) and bidirectional long-short term memory (BLSTM). In the present work, the classifier classifies an ink as text or non-text stroke when a sequence of strokes of any online handwritten document of both text and non-text data is presented to it. Various structural and directional features related to online handwriting have been extracted from the basic strokes of text and non-text data. The system has been trained in both LSTM and BLSTM architecture based classification platforms. Experiment has also been performed using Convolutional Neural Network (CNN) to make a comparative performance analysis with RNN classifier based results. The classification performance of the present work has been evaluated using a self-generated dataset and it outperforms the CNN based results as well as the existing studies available in the literature in this regard.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Notes

  1. http://deeplearning.net/software/theano/

References

  1. Awal AM, Mouchre H, Viard-Gaudin C (2014) A global learning approach for an on-line handwritten mathematical expression recognition system. Pattern Recogn Lett 35:68–77

    Article  Google Scholar 

  2. Blanchard J, Artieres T (2004) On-line handwritten documents segmentation. In: Proceedings of the 9th international workshop on frontiers in handwriting recognition. IEEE Press, Tokyo, pp 148–153

  3. Bresler M, VanPhan T, Prusa D, Nakagawa M, Hlavc V (2014) Recognition system for online sketched diagrams. In: Proceedings of the 14th international conference on frontiers in handwriting recognition. IEEE Press, Crete, pp 563–568

  4. Delaye A, Lee K (2015) A flexible framework for online document segmentation by pairwise stroke distance learning. Pattern Recogn 48:1197–1210

    Article  Google Scholar 

  5. Delaye A, Liu CL (2013) Graphics extraction from heterogeneous online documents with hierarchical random fields. In: Proceedings of the 12th international conference on document analysis and recognition. IEEE Press, Washington DC, pp 1007–1011

  6. Delaye A, Liu CL (2014) Contextual text/non-text stroke classification in online handwritten notes with conditional random fields. Pattern Recogn 47 (3):959–968

    Article  Google Scholar 

  7. Feng G, Viard-Gaudin C, Sun Z (2009) Online Hand-drawn electric circuit diagram recognition using 2d dynamic programming. Pattern Recogn 42 (12):3215–3223

    Article  Google Scholar 

  8. Ghosh R, Keshri P, Kumar P (2018) RNN based online handwritten word recognition in Devanagari 431 script. In: Proceedings of the 16th international conference on frontiers in handwriting recognition. IEEE 432 Press, Niagra Falls, pp 517–522

  9. Ghosh R, Kumar S, Kumar P (2018). Online Handwritten Text and Non-Text Classification in Devanagari Script using Elliptical Region-wise Features. In: Proceedings of the first international conference on secure cyber computing and communications. IEEE Press, Jalandhar, pp 163–167

  10. Ghosh R, Shanu S, Ranjan S, Kumari K (2019) An approach based on classifier combination for online handwritten text and non-text classification in Devanagari script. SADHANA 44:8

    Article  Google Scholar 

  11. Ghosh R, Vamsi C, Kumar P (2019) RNN based online handwritten word recognition in Devanagari and bengali scripts using horizontal zoning. Pattern Recogn 92:203–218

    Article  Google Scholar 

  12. Graves A, Liwicki M, Fernandez S, Bertolami R, Bunke H, Schmidhuber J (2009) A novel connectionist system for unconstrained handwriting recognition. IEEE Trans Pattern Anal Mach Intell 31(5):855–868

    Article  Google Scholar 

  13. Jaeger S, Manke S, Reichert J, Waibel A (2001) Online handwriting recognition: the NPen++ recognizer. Int J Doc Anal Recogn 3(3):169–180

    Article  Google Scholar 

  14. Liwicki M, Indermuhle E, Bunke H (2007) Online hand written text line detection using dynamic programming. In: Proceedings of the 9th international conference on document analysis and recognition. IEEE Press, Curitiba, pp 447–451

  15. Mochida K, Nakagawa M (2004) Separating figures, mathematical formulas and Japanese text from free handwriting in mixed on-line documents. Int J Pattern Recogn Artif Intell 18(7):1173–1187

    Article  Google Scholar 

  16. Phan TV, Nakagawa M (2014) Text/non-text classification in online handwritten documents with recurrent neural networks. In: Proceedings of the 14th international conference on frontiers in handwriting recognition. IEEE Press, Crete, pp 23–28

  17. Phan TV, Nakagawa M (2016) Combination of global and local contexts for text/non-text classification in heterogeneous online handwritten documents. Pattern Recogn 51:112–124

    Article  Google Scholar 

  18. Schuster M, Paliwal KK (1997) Bidirectional recurrent neural networks. IEEE Trans Signal Process 45(11):2673–2681

    Article  Google Scholar 

  19. Zhou XD, Liu CL (2007) Text/non-text ink stroke classification in Japanese handwriting based on Markov random fields. In: Proceedings of the 9th international conference on document analysis and recognition. IEEE Press, Curitiba, pp 377–381

  20. Zhou XD, Wang DH, Liu C (2009) A robust approach to text line grouping in online handwritten Japanese documents. Pattern Recogn 42(9):2077–2088

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, National Institute of Technology Patna, Patna, India

    Rajib Ghosh

Authors
  1. Rajib Ghosh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rajib Ghosh.

Ethics declarations

Conflict of Interests

The author has no conflict of interest/competing interest to declare that are relevant to the content of this article.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ghosh, R. A recurrent neural network based deep learning model for text and non-text stroke classification in online handwritten Devanagari document. Multimed Tools Appl 81, 24245–24263 (2022). https://doi.org/10.1007/s11042-022-12767-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12767-6

Keywords

Search

Navigation