Skip to main content
Springer Nature Link
Log in

A comparative study of delayed stroke handling approaches in online handwriting

  • Original Paper
  • Published:
International Journal on Document Analysis and Recognition (IJDAR) Aims and scope Submit manuscript

Abstract

Delayed strokes, such as i-dots and t-crosses, cause a challenge in online handwriting recognition by introducing an extra source of variation in the sequence order of the handwritten input. The problem is especially relevant for languages where delayed strokes are abundant and training data are limited. Studies for handling delayed strokes have mainly focused on Arabic and Farsi scripts where the problem is most severe, with less attention devoted for scripts based on the Latin alphabet. This study aims to investigate the effectiveness of the delayed stroke handling methods proposed in the literature. Evaluated methods include the removal of delayed strokes and embedding delayed strokes in the correct writing order, together with their variations. Starting with new definitions of a delayed stroke, we tested each method using both hidden Markov model classifiers separately for English and Turkish and bidirectional long short-term memory networks for English. For both the UNIPEN and Turkish datasets, the best results are obtained with hidden Markov model recognizers by removing all delayed strokes, with up to 2.13% and 2.03% points accuracy increases over the respective baselines. In case of the bidirectional long short-term memory networks, stroke order correction of the delayed strokes by embedding performs the best, with 1.81% (raw) and 1.72% (post-processed) points improvements above the baseline.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. https://github.com/baidu-research/warp-ctc.

  2. https://tinyurl.com/y7m3rv5w.

  3. https://tinyurl.com/yc93rcf5.

References

  1. Abdelazeem, S., Eraqi, H.M.: On-line Arabic handwritten personal names recognition system based on HMM. In: 2011 International Conference on Document Analysis and Recognition, ICDAR 2011, Beijing, China, September 18–21, 2011, pp. 1304–1308 (2011)

  2. Abdelaziz, I., Abdou, S., Al-Barhamtoshy, H.: Large vocabulary Arabic online handwriting recognition system. CoRR. arxiv:abs/1410.4688 (2014)

  3. Alimi, A.M.: An evolutionary neuro-fuzzy approach to recognize on-line Arabic handwriting. In: 4th International Conference Document Analysis and Recognition, ICDAR 1997, 2-Volume Set, August 18–20, 1997, Ulm, Germany, Proceedings, pp. 382–386 (1997)

  4. Biadsy, F., El-Sana, J., Habash, J.: Online Arabic handwriting recognition using Hidden Markov Models. In: Tenth International Workshop on Frontiers in Handwriting Recognition, IWFHR 2007, IAPR. Newyork,USA, 2006 (2006)

  5. Flann, N.S.: Recognition-based segmentation of on-line cursive handwriting. In: Advances in Neural Information Processing Systems 6, 7th NIPS Conference, Denver, Colorado, USA, 1993, pp. 777–784 (1993)

  6. Gauthier, N., Artières, T., Gallinari, P., Dorizzi, B.: Strategies for combining on-line and off-line information in an on-line handwriting recognition system. In: 6th International Conference on Document Analysis and Recognition, ICDAR 2001, 10–13 September 2001, pp. 412–416. Seattle, WA, USA (2001)

  7. Ghods, V., Kabir, E., Razzazi, F.: Effect of delayed strokes on the recognition of online Farsi handwriting. Pattern Recognit. Lett. 34(5), 486–491 (2013)

    Article  Google Scholar 

  8. Graves, A., Jaitly, N.: Towards end-to-end speech recognition with recurrent neural networks. In: Proceedings of the 31st International Conference on Machine Learning, ICML 2014, Beijing, China, 21–26 June 2014, pp. 1764–1772 (2014)

  9. Graves, A., Fernández, S., Gomez, F.J., Schmidhuber, J.: Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In: Machine Learning, Proceedings of the Twenty-Third International Conference (ICML 2006), Pittsburgh, Pennsylvania, USA, June 25–29, 2006, pp. 369–376 (2006)

  10. Graves, A., Fernández, S., Liwicki, M., Bunke, H., Schmidhuber, J.: Unconstrained on-line handwriting recognition with recurrent neural networks. In: Advances in Neural Information Processing Systems 20, Proceedings of the Twenty-First Annual Conference on Neural Information Processing Systems, NIPS, Vancouver, British Columbia, Canada, December 3–6, 2007, pp. 577–584 (2007)

  11. Graves, A., Liwicki, M., Fernandez, S., Bertolami, R., Bunke, H., Schmidhuber, J.: A novel connectionist system for unconstrained handwriting recognition. IEEE Trans. Pattern Anal. Mach. Intell. 31(5), 855–868 (2009)

    Article  Google Scholar 

  12. Günter, S., Bunke, H.: Optimizing the number of states, training iterations and Gaussians in an HMM-based handwritten word recognizer. In: 7th International Conference on Document Analysis and Recognition, (ICDAR 2003), 2-Volume Set, 3–6 August 2003, pp. 472–476. Edinburgh, Scotland, UK (2003)

  13. Ha, J., Oh, S., J.K., Kwon, Y.: Unconstrained handwritten word recognition with interconnected Hidden Markov Models. In: Third International Workshop on Frontiers in Handwriting Recognition, IWFHR 1993, IAPR. Buffalo, USA, May 1993, pp 455–560 (1993)

  14. HTK The Hidden Markov Model toolkit htk. http://htk.eng.cam.ac.uk/. Accessed 29 Oct 2016

  15. Hu, J., Brown, M.K., Turin, W: Handwriting recognition with Hidden Markov Models and grammatical constraints. In: Fourth International Workshop on Frontiers in Handwriting Recognition, IWFHR 1994, IAPR.Taipei, Dec 1994 (1994)

  16. Hu, J., Lim, S.G., Brown, M.K.: Writer independent on-line handwriting recognition using an HMM approach. Pattern Recognit. 33(1), 133–147 (2000)

    Article  Google Scholar 

  17. Jäger, S., Manke, S., Reichert, J., Waibel, A.: Online handwriting recognition: the NPen++ recognizer. IJDAR 3(3), 169–180 (2001)

    Article  Google Scholar 

  18. Koerich, A.L., Sabourin, R., Suen, C.Y.: Large vocabulary off-line handwriting recognition: a survey. Pattern Anal. Appl. 6(2), 97–121 (2003)

    Article  MathSciNet  Google Scholar 

  19. Liwicki, M., Bunke, H.: IAM-OnDB—an on-line English sentence database acquired from handwritten text on a whiteboard. In: Eighth International Conference on Document Analysis and Recognition, ICDAR 2005, 29 August–1 September 2005, pp. 956–961. Seoul, Korea (2005)

  20. Liwicki, M., Bunke, H.: Hmm-based on-line recognition of handwritten whiteboard notes. In: Tenth International Workshop on Frontiers in Handwriting Recognition, IWFHR 2006 (2006)

  21. Liwicki, M., Bunke, H.: Handwriting recognition of whiteboard notes—studying the influence of training set size and type. IJPRAI 21(1), 83–98 (2007)

    Google Scholar 

  22. Liwicki, M., Graves, A., Bunke, H., Schmidhuber, J.: A novel approach to on-line handwriting recognition based on bidirectional long short-term memory networks. In: Proceedings of the 9th International Conference on Document Analysis and Recognition, ICDAR 2007 (2007)

  23. Marukatat, S., Artières, T., Gallinari, P., Dorizzi, B.: Sentence recognition through hybrid neuro-Markovian modeling. In: 6th International Conference on Document Analysis and Recognition, ICDAR 2001, 10–13 September 2001, pp. 731–737. Seattle, WA, USA (2001)

  24. Plamondon, R., Srihari, S.N.: On-line and off-line handwriting recognition: a comprehensive survey. IEEE Trans. Pattern Anal. Mach. Intell. 22(1), 63–84 (2000)

    Article  Google Scholar 

  25. Plötz, T., Fink, G.A.: Markov models for offline handwriting recognition: a survey. IJDAR 12(4), 269–298 (2009)

    Article  Google Scholar 

  26. Rabiner, L.R.: A tutorial on Hidden Markov Models and selected applications in speech recognition. In: Proceedings of the IEEE, pp. 257–286 (1989)

  27. Ratcliff, J.W., Metzener, D.E.: Pattern matching: the gestalt approach. Dr Dobbs J. 13(7), 46–72 (1988)

    Google Scholar 

  28. Schenk, J., Rigoll, G.: Novel hybrid nn/hmm modelling techniques for on-line handwriting recognition. In: 10th International Workshop on Frontiers in Handwriting Recognition, IWFHR 2006, IAPR. , La Baule, France, Oct 2006, pp. 619–6230 (2006)

  29. Schenkel, M., Guyon, I., Henderson, D.: On-line cursive script recognition using time-delay neural networks and Hidden Markov Models. Mach. Vis. Appl. 8(4), 215–223 (1995)

    Article  Google Scholar 

  30. Sutskever, I.: (2013) Training Recurrent Neural Networks. Ph.D. thesis, Toronto, Ont., Canada

  31. The UNIPEN Consortium The UNIPEN project. http://www.unipen.org/products.html. Accessed 20 Oct 2016

  32. Viard-Gaudin, C., Lallican, P.M., Binter, P., Knerr, S.: The IRESTE on/off (IRONOFF) dual handwriting database. In: Fifth International Conference on Document Analysis and Recognition, ICDAR 1999, 20–22 September, 1999, pp. 455–458. Bangalore, India (1999)

  33. Vural, E., Erdogan, H., Oflazer, K., Yanikoglu, B.A.: An online handwriting recognition system for Turkish. In: Document Recognition and Retrieval XII,DRR 2005,San Jose, California, USA, January 16–20, 2005, Proceedings, pp. 56–65 (2005)

Download references

Acknowledgements

This work was supported by The Scientific and Technological Research Council of Turkey (TÜBITAK), under Grant Number 113E062.

Author information

Authors and Affiliations

  1. Faculty of Engineering and Natural Sciences, Sabancı University, 34956, Istanbul, Turkey

    Esma F. Bilgin Tasdemir & Berrin Yanikoglu

Authors
  1. Esma F. Bilgin Tasdemir
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Berrin Yanikoglu
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Esma F. Bilgin Tasdemir.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tasdemir, E.F.B., Yanikoglu, B. A comparative study of delayed stroke handling approaches in online handwriting. IJDAR 22, 15–28 (2019). https://doi.org/10.1007/s10032-018-0313-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10032-018-0313-2

Keywords