Skip to main content
SpringerLink
Log in

Rotation invariant hand-drawn symbol recognition based on a dynamic time warping model

  • Full Paper
  • Published:
International Journal on Document Analysis and Recognition (IJDAR) Aims and scope Submit manuscript

Abstract

One of the major difficulties of handwriting symbol recognition is the high variability among symbols because of the different writer styles. In this paper, we introduce a robust approach for describing and recognizing hand-drawn symbols tolerant to these writer style differences. This method, which is invariant to scale and rotation, is based on the dynamic time warping (DTW) algorithm. The symbols are described by vector sequences, a variation of the DTW distance is used for computing the matching distance, and K-Nearest Neighbor is used to classify them. Our approach has been evaluated in two benchmarking scenarios consisting of hand-drawn symbols. Compared with state-of-the-art methods for symbol recognition, our method shows higher tolerance to the irregular deformations induced by hand-drawn strokes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bartolini I., Ciaccia P., Patella M.: Warp: accurate retrieval of shapes using phase of fourier descriptors and time warping distance. IEEE Trans. Pattern Anal. Mach. Intell. 27(1), 142–147 (2005)

    Article  Google Scholar 

  2. Belongie S., Malik J., Puzicha J.: Shape matching and object recognition using shape contexts. IEEE Trans. Pattern Anal. Mach. Intell. 24(4), 509–522 (2002)

    Article  Google Scholar 

  3. Bunke H.: Attributed programmed graph grammars and their application to schematic diagram interpretation. IEEE Trans. Pattern Anal. Mach. Intell. 4(6), 574–582 (1982)

    Article  MATH  Google Scholar 

  4. Chan K.F., Yeung D.Y.: Mathematical expression recognition: a survey. Int. J. Document Anal. Recognit. 3(1), 3–15 (2000)

    Article  Google Scholar 

  5. Chou K.S., Fan K.C., Fan T.I.: Peripheral and global features for use in coarse classification of Chinese characters. Pattern Recognit. 30(3), 483–489 (1997)

    Article  Google Scholar 

  6. Fonseca M.J., Pimentel C., Jorge J.A.: CALI: an online scribble recognizer for calligraphic interfaces. In: AAAI Spring Symposium on Sketch Understanding, pp. 51–58. (2002)

  7. Fornés, A., Lladós, J., Sánchez, G.: Primitive segmentation in old handwritten music scores. In: Liu, W., Lladós, J. (eds.) Graphics Recognition: Ten Years Review and Future Perspectives, vol. 3926 of Lecture Notes in Computer Science, pp. 279–290. Springer (2006)

  8. Garain U., Chaudhuri B.B.: Recognition of online handwritten mathematical expressions. IEEE Trans. Syst. Man Cybern. Part B: Cybern. 34(6), 2366–2376 (2004)

    Article  Google Scholar 

  9. Golubitsky, O., Watt, S.M.: Online recognition of multi-stroke symbols with orthogonal series. International Conference on Document Analysis and Recognition, vol. 2, pp. 1265–1269 (2009)

  10. Hse, H., Newton, A.R.: Sketched symbol recognition using Zernike moments. In: Proceedings of the 17th International Conference on Pattern Recognition, vol. 1, pp. 367–370 (2004)

  11. Hu, N., Dannenberg, R.B., Tzanetakis, G.: Polyphonic audio matching and alignment for music retrieval. In: IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, pp. 185–188. New Paltz, New York, October (2003)

  12. Keogh E., Ratanamahatana C.A.: Exact indexing of dynamic time warping. Knowl. Inf. Syst. 7(3), 358–386 (2005)

    Article  Google Scholar 

  13. Khotanzad A., Hong Y.H.: Invariant image recognition by Zernike moments. IEEE Trans. Pattern Anal. Mach. Intell. 12(5), 489–497 (1990)

    Article  Google Scholar 

  14. Kim, W.Y., Kim, Y.S.: A new region-based shape descriptor. Technical report, Hanyang University and Konan Technology (1999)

  15. Kornfield, E.M., Manmatha, R., Allan, J.: Text alignment with handwritten documents. In Document Image Analysis for Libraries, pp. 195–209. IEEE Computer Society, Washington, DC, USA, (2004)

  16. Kruskal J.B., Liberman M.: The symmetric time-warping problem: from continuous to discrete. In: Sankoff, D., Kruskal, J.B. (eds) Time Warps, String Edits, and Macromolecules: The Theory and Practice of Sequence Comparison., pp. 125–161. Addison-Wesley Publishing Co, Reading, MA (1983)

    Google Scholar 

  17. Khurshid, K., Faure, C., Vincent, N.: Fusion of word spotting and spatial information for figure caption retrieval in historical document images. 10th International Conference on Document Analysis and Recognition, vol. 1, pp. 266–270 (2009)

  18. Levin, E., Pieraccini, R.: Dynamic planar warping for optical character recognition. In IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 3, pp. 149–152 (1992)

  19. Liang, S., Sun, Z., Li, B.: Sketch retrieval based on spatial relations. Proceedings of International Conference on Computer Graphics, Imaging and Visualization, Beijing, China, pp. 24–29 (2005)

  20. Liu, C.L.: Normalization-cooperated gradient feature extraction for handwritten character recognition. IEEE Trans. Pattern Anal. Mach. Intell., pp. 1465–1469 (2007)

  21. Lladós, J., Valveny, E., Sánchez, G., Martí, E.: Symbol recognition: current advances and perspectives. In Lecture Notes in Computer Science, vol. 2390, pp. 104–128. Springer (2002)

  22. Lladós, J., Martí, E., Villanueva, J.J.: Symbol recognition by error-tolerant subgraph matching between region adjacency graphs. In IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23(10), pp. 1137–1143, October (2001)

  23. Marti U.V., Bunke H.: Using a statistical language model to improve the performance of an hmm-based cursive handwriting recognition system. Int. J. Pattern Recogni. Artif. Intell. 15, 65–90 (2001)

    Article  Google Scholar 

  24. Mas, J., Jorge, J.A., Sánchez, G., Lladós, J.: Representing and parsing sketched symbols using adjacency grammars and a grid-directed parser. In: Ogier eds, J.M., Liu, W., Lladós, J. (eds.), Graphics Recognition: Recent Advances and New Opportunities, Lecture Notes in Computer Science, vol. 5046, pp. 176–187. Springer-Verlag (2008)

  25. Miyao H., Maruyama M.: An online handwritten music symbol recognition system. Int. J. Document Anal. Recogni. 9(1), 49–58 (2007)

    Article  Google Scholar 

  26. Mokhtarian F., Mackworth A.K.: Scale-based description and recognition of planar curves and two-dimensional shapes. IEEE Trans. Pattern Anal. Mach. Intell. 8(1), 34–43 (1986)

    Article  Google Scholar 

  27. Müller S., Rigoll G.: Engineering Drawing Database Retrieval Using Statistical Pattern Spotting Techniques. Graphics Recogni. Recent Adv. Lecture Notes Comput. Sci. 1941, 246–255 (2000)

    Google Scholar 

  28. Niels, R., Vuurpijl, L.: Using dynamic time warping for intuitive handwriting recognition. Advances in Graphonomics, Proceedings of the 12th Conference of the International Graphonomics Society, pp. 217–221 (2005)

  29. Orio, N., Schwarz, D.: Alignment of monophonic and polyphonic music to a score. In San Francisco International Computer Music Association, editor, Proceedings of the International Computer Music Conference, pp. 155–158, Havana, Cuba, September (2001)

  30. Parker, J., Pivovarov, J., Royko, D.: Vector templates for symbol recognition. In: International Conference on Pattern Recognition, vol. 15, pp. 602–605 (2000)

  31. Peng, B., Liu, Y., Wenyin, L., Huang, G.: Sketch recognition based on topological spatial relationship. In: Structural, Syntactic, and Statistical Pattern Recognition: Lecture Notes in Computer Science, vol. 3138, pp. 434–443. Springer-Verlag (2004)

  32. Rabiner L.R.: A tutorial on hidden Markov models and selected applications inspeech recognition. Proc. IEEE 77(2), 257–286 (1989)

    Article  Google Scholar 

  33. Rath, T.M. Manmatha, R.: Word image matching using dynamic time warping. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 521–527 Madison, WI, June 18–20 (2003)

  34. Rusiñol M., Borràs A., Lladós J.: Relational indexing of vectorial primitives for symbol spotting in line-drawing images. Pattern Recogni. Lett. 31(3), 188–201 (2010)

    Article  Google Scholar 

  35. Sanchez, G., Valveny, E., Llados, J., Romeu, J. Mas, Lozano. N.: A platform to extract knowledge from graphic documents. Application to an architectural sketch understanding scenario. In: Dengel, A. Marinai, S., (eds.), Document Analysis Systems VI, Lecture Notes in Computer Science, vol. 3163, pp. 389–400, Florence—Italy, 2004, Springer-Verlag

  36. Shi, Y., Li, H.Y., Soong, F.K.: A unified framework for symbol segmentation and recognition of handwritten mathematical expressions. Ninth International Conference on Document Analysis and Recognition, vol. 2, 854–858, Sept. (2007)

  37. Tabbone, S., Wendling, L.: Technical symbols recognition using the two-dimensional radon transform. In Proceedings of the 16th International Conference on Pattern Recognition, vol. 3, pp. 200–203 (2002)

  38. Uchida, S., Sakoe, H.: A monotonic and continuous two-dimensional warping based on dynamic programming. In Proceedings of 14th International Conference on Pattern Recognition, vol. 1, pp. 521–524, (1998)

  39. Uchida S., Sakoe H.: A survey of elastic matching techniques for handwritten character recognition. IEICE Trans. Inf. Syst. E88-D, 1781–1790 (2005)

    Article  Google Scholar 

  40. Valveny, E., Marti, E.: Hand-drawn symbol recognition in graphic documents using deformable template matching and a bayesian framework. Proceedings of the 15th International Conference on Pattern Recognition, vol. 2, pp. 239–242 (2000)

  41. Vinciarelli, A., Bengio, S., Bunke, H.: Offline recognition of unconstrained handwritten texts using HMMs and statistical language models. IEEE Trans. Pattern Anal. Mach. Intell. pp. 709–720 (2004)

  42. Vuori V., Laaksonen J., Oja E., Kangas J.: Experiments with adaptation strategies for a prototype-based recognition system for isolated handwritten characters. Int. J. Document Anal. Recogni. 3(3), 150–159 (2001)

    Article  Google Scholar 

  43. Wilfong G., Sinden F., Ruedisueli L.: On-line recognition of handwritten symbols. IEEE Trans. Pattern Anal. Mach. Intell. 18(9), 935–940 (1996)

    Article  Google Scholar 

  44. Xin, G., Cuiyun, L., Jihong, P., Weixin, X.: HMM based online hand-drawn graphic symbol recognition. In Proceedings of the 6th International Conference on Signal Processing, vol. 2, pp. 1067–1070 (2002)

  45. Yang S.: Symbol recognition via statistical integration of pixel-level constraint histograms: a new descriptor. IEEE Trans. Pattern Anal. Mach. Intell. 27(2), 278–281 (2005)

    Article  Google Scholar 

  46. Zhang, D.S., Lu, G.: Generic Fourier descriptor for shape-based image retrieval. In Proceedings of the IEEE International Conference on Multimedia and Expo, vol. 1, pp. 425–428 (2002)

  47. Zhang, W., Liu, W.: A new syntactic approach to graphic symbol recognition. International Conference on Document Analysis and Recognition vol. 1, pp. 516–520 (2007)

  48. Zhang W., Liu W., Zhang K.: Symbol recognition with kernel density matching. IEEE Trans. Pattern Anal. Mach. Intell. 28(12), 2020–2024 (2006)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Vision Center, Department of Computer Science, Universitat Autònoma de Barcelona, Edifici O, 08193, Bellaterra, Spain

    Alicia Fornés, Josep Lladós, Gemma Sánchez & Dimosthenis Karatzas

Authors
  1. Alicia Fornés
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Josep Lladós
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gemma Sánchez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dimosthenis Karatzas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alicia Fornés.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Fornés, A., Lladós, J., Sánchez, G. et al. Rotation invariant hand-drawn symbol recognition based on a dynamic time warping model. IJDAR 13, 229–241 (2010). https://doi.org/10.1007/s10032-010-0114-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10032-010-0114-8

Keywords

Search

Navigation