Skip to main content
SpringerLink
Log in

Recognizing handwritten Arabic words using grapheme segmentation and recurrent neural networks

  • Original Paper
  • Published:
International Journal on Document Analysis and Recognition (IJDAR) Aims and scope Submit manuscript

Abstract

The Arabic alphabet is used in around 27 languages, including Arabic, Persian, Kurdish, Urdu, and Jawi. Many researchers have developed systems for recognizing cursive handwritten Arabic words, using both holistic and segmentation-based approaches. This paper introduces a system that achieves high accuracy using efficient segmentation, feature extraction, and recurrent neural network (RNN). We describe a robust rule-based segmentation algorithm that uses special feature points identified in the word skeleton to segment the cursive words into graphemes. We show that careful selection from a wide range of features extracted during and after the segmentation stage produces a feature set that significantly reduces the label error. We demonstrate that using same RNN recognition engine, the segmentation approach with efficient feature extraction gives better results than a holistic approach that extracts features from raw pixels. We evaluated this segmentation approach against an improved version of the holistic system MDLSTM that won the ICDAR 2009 Arabic handwritten word recognition competition. On the IfN/ENIT database of handwritten Arabic words, the segmentation approach reduces the average label error by 18.5 %, the sequence error by 22.3 %, and the execution time by 31 %, relative to MDLSTM. This approach also has the best published accuracies on two IfN/ENIT test sets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Abandah, G., Jamour, F.: Recognizing handwritten Arabic script through efficient skeleton-based grapheme segmentation algorithm. In: International Conference Intelligent Systems Design and Applications, pp. 977–982 (2010)

  2. Abandah, G., Jamour, F.: A word matching algorithm in handwritten Arabic recognition using multiple-sequence weighted edit distances (2014). Submitted.

  3. Abandah, G., Khedher, M.: Analysis of handwritten Arabic letters using selected feature extraction techniques. Int. J. Comput. Process. Lang. 22(1), 49–73 (2009)

    Article  Google Scholar 

  4. Abandah, G., Malas, T.: Feature selection for recognizing handwritten Arabic letters. Dirasat Eng. Sci. J. 37(2), 242–256 (2010)

    Google Scholar 

  5. Al-Hajj, R., Likforman-Sulem, L., Mokbel, C.: Arabic handwriting recognition using baseline dependant features and hidden Markov modeling. In: International Conference Document Analysis and Recognition, pp. 893–897 (2005)

  6. Alginahi, Y.M.: A survey on Arabic character segmentation. Int. J. Doc. Anal. Recogn. 16(2), 105–126 (2013)

    Article  Google Scholar 

  7. Alkhoury, I., Giménez, A., Juan, A.: Arabic handwriting recognition using Bernoulli HMMs. In: Märgner, V., El Abed, H. (eds.) Guide to OCR for Arabic Scripts, pp. 255–272. Springer, London (2012)

    Chapter  Google Scholar 

  8. Amin, A.: Arabic character recognition. In: Bunke, H., Wang, P. (eds.) Handbook of Character Recognition and Document Image Analysis, pp. 397–420. World Scientific, Singapore (1997)

  9. Arica, N., Yarman-Vural, F.: Optical character recognition for cursive handwriting. IEEE Trans. Pattern Anal. Mach. Intell. 24(6), 801–813 (2002)

    Article  Google Scholar 

  10. Azeem, S.A., Ahmed, H.: Effective technique for the recognition of offline Arabic handwritten words using hidden Markov models. Int. J. Doc. Anal. Recogn. 16(4), 399–412 (2013)

    Google Scholar 

  11. Chang, F., Chen, C.J., Lu, C.J.: A linear-time component-labeling algorithm using contour tracing technique. Comput. Vis. Image Underst. 93(2), 206–220 (2004)

    Article  Google Scholar 

  12. Deutsch, E.: Thinning algorithms on rectangular, hexagonal, and triangular arrays. Commun. ACM 15(9), 827–837 (1972)

    Article  Google Scholar 

  13. Douglas, D., Peucker, T.: Algorithms for the reduction of the number of points required to represent a line or its caricature. Canad. Cartogr. 10(2), 112–122 (1973)

    Article  Google Scholar 

  14. Dreuw, P., Rybach, D., Heigold, G., Ney, H.: RWTH OCR: A large vocabulary optical character recognition system for Arabic scripts. In: Märgner, V., El Abed, H. (eds.) Guide to OCR for Arabic Scripts, pp. 215–254. Springer, London (2012)

  15. El Abed, H., Märgner, V.: Comparison of different preprocessing and feature extraction methods for offline recognition of handwritten Arabic words. In: International Conference Document Analysis and Recognition, pp. 974–978 (2007)

  16. El-Hajj, R., Likforman-Sulem, L., Mokbel, C.: Arabic handwriting recognition using baseline dependant features and hidden Markov modeling. In: International Conference Document Analysis and Recognition, pp. 893–897 (2005)

  17. Freeman, H.: On the encoding of arbitrary geometric configurations. IRE Trans. Electron. Comput. 10(2), 260–268 (1961)

    Article  MathSciNet  Google Scholar 

  18. Gers, F.: Long short-term memory in recurrent neural networks. Ph.D. thesis, Ecole Polytechnique Fédérale de Lausanne (2001)

  19. Graves, A.: RNNLIB: a recurrent neural network library for sequence learning problems. http://sourceforge.net/projects/rnnl/

  20. Graves, A.: Supervised sequence labelling with recurrent neural networks. Ph.D. thesis, Technische Universität München (2008)

  21. Graves, A.: Offline Arabic handwriting recognition with multidimensional recurrent neural networks. In: Märgner, V., El Abed, H. (eds.) Guide to OCR for Arabic Scripts, pp. 297–313. Springer, London (2012)

    Chapter  Google Scholar 

  22. Graves, A.: Supervised Sequence Labelling with Recurrent Neural Networks. Studies in Computational Intelligence, vol. 382. Springer, Berlin (2012)

    Book  Google Scholar 

  23. Graves, A., Fernández, S., Gomez, F., Schmidhuber, J.: Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks. In: International Conference Machine Learning (2006)

  24. Graves, A., Fernández, S., Liwicki, M., Bunke, H., Schmidhuber, J.: Unconstrained online handwriting recognition with recurrent neural networks. Adv. Neural Inf. Process. Syst. 20, 1–8 (2008)

    Google Scholar 

  25. Graves, A., Fernández, S., Schmidhuber, J.: Multi-dimensional recurrent neural networks. In: International Conference Artificial Neural Networks (2007)

  26. Graves, A., Liwicki, M., Fernández, S., Bertolami, R., Bunke, H., Schmidhuber, J.: A novel connectionist system for unconstrained handwriting recognition. IEEE Trans. Pattern Anal. Mach. Intell. 31(5), 855–868 (2009)

    Article  Google Scholar 

  27. Graves, A., Schmidhuber, J.: Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Netw. 18(5), 602–610 (2005)

    Article  Google Scholar 

  28. Graves, A., Schmidhuber, J.: Offline Handwriting Recognition with Multidimensional Recurrent Neural Networks. In: Advances in Neural Information Processing Systems, NIPS’22, vol. 22, chap., pp. 545–552. MIT Press, Vancouver (2009)

  29. Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. J. Mach. Learn. Res. 3(1), 1157–1182 (2003)

    MATH  Google Scholar 

  30. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  31. Kuhl, F., Giardina, C.: Elliptic Fourier features of a closed contour. Comput. Graph. Image Process. 18(3), 236–258 (1982)

    Article  Google Scholar 

  32. Kundu, A., Hines, T., Phillips, J., Huyck, B.D., Van Guilder, L.C.: Arabic handwriting recognition using variable duration HMM. In: International Conference Document Analysis and Recognition, pp. 644–648 (2007)

  33. Lee, H., Verma, B.: Binary segmentation algorithm for English cursive handwriting recognition. Pattern Recogn. 45(4), 1306–1317 (2012)

    Article  Google Scholar 

  34. Lewis, M.P. (ed.): Ethnologue: Languages of the World. SIL International, Dallas (2009)

    Google Scholar 

  35. Likforman-Sulem, L., Mohammad, R.A.H., Mokbel, C., Menasri, F., Bianne-Bernard, A.L., Kermorvant, C.: Features for HMM-based Arabic handwritten word recognition systems. In: Märgner, V., El Abed, H. (eds.) Guide to OCR for Arabic Scripts, pp. 123–143. Springer, London (2012)

    Chapter  Google Scholar 

  36. Liu, C.L.: Handwritten Chinese character recognition: effects of shape normalization and feature extraction. In: Doermann, D., Jaeger, S. (eds.) Arabic and Chinese Handwriting Recognition, vol. LNCS 4768, pp. 104–128. Springer, Berlin (2008)

    Chapter  Google Scholar 

  37. Lorigo, L., Govindaraju, V.: Segmentation and pre-recognition of Arabic handwriting. In: International Conference Document Analysis and Recognition, pp. 605–609 (2005)

  38. Lorigo, L., Govindaraju, V.: Offline Arabic handwriting recognition: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 28(5), 712–724 (2006)

    Article  Google Scholar 

  39. Maddouri, S.S., El-Abed, H., Samoud, F.B., Bouriel, K., Ellouze, N.: Baseline extraction: comparison of six methods on IFN/ENIT database. In: International Conference Frontiers in Handwriting Recognition (2008)

  40. Märgner, V., El Abed, H.: ICDAR 2005—Arabic handwriting recognition competition. In: International Conference Document Analysis and Recognition, pp. 70–74 (2005)

  41. Märgner, V., El Abed, H.: ICDAR 2007—Arabic handwriting recognition competition. In: International Conference Document Analysis and Recognition, pp. 1274–1278 (2007)

  42. Märgner, V., El Abed, H.: ICDAR 2009—Arabic handwriting recognition competition. In: International Conference Document Analysis and Recognition, pp. 1383–1387 (2009)

  43. Märgner, V., El Abed, H.: ICFHR 2010—Arabic handwriting recognition competition. In: International Conference Frontiers in Handwriting Recognition, pp. 709–714 (2010)

  44. Märgner, V., El Abed, H.: ICDAR 2011—Arabic handwriting recognition competition. In: International Conference Document Analysis and Recognition, pp. 1444–1448 (2011)

  45. Motawa, D., Amin, A., Sabourin, R.: Segmentation of Arabic cursive script. In: International Conference Document Analysis and Recognition, pp. 625–628 (1997)

  46. Murray, A., Edwards, P.: Synaptic weight noise during multilayer perceptron training: fault tolerance and training improvements. IEEE Trans. Neural Netw. 4(4), 722–725 (1993)

    Google Scholar 

  47. Pechwitz, M., El Abed, H., Märgner, V.: Handwritten Arabic word recognition using the IFN/ENIT-database. In: Märgner, V., El Abed, H. (eds.) Guide to OCR for Arabic Scripts, pp. 169–213. Springer, London (2012)

    Chapter  Google Scholar 

  48. Pechwitz, M., Maddouri, S.S., Märgner, V., Ellouze, N., Amiri, H.: IFN/ENIT - database of handwritten Arabic words. In: Colloque Int’l Francophone sur l’Ecrit et le Document, pp. 129–136 (2002)

  49. Peng, H.: mRMR (minimum redundancy maximum relevance feature selection). http://penglab.janelia.org/proj/mRMR/

  50. Peng, H., Long, F., Ding, C.: Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans. Pattern Anal. Mach. Intell. 27(8), 1226–1238 (2005)

    Article  Google Scholar 

  51. Qaralleh, M., Abandah, G., Jamour, F.: Tuning recurrent neural networks for recognizing handwritten Arabic words. J. Softw. Eng. Appl. 6(10), 533–542 (2013)

    Article  Google Scholar 

  52. Ratcliff, J., Metzener, D.: Pattern matching: the Gestalt approach. Dr. Dobb’s J. 13(7), 46–72 (1988)

    Google Scholar 

  53. Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning internal representations by error propagation. In: Rumelhart, D.E., McClelland, J.L. (eds.) Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. 1: Foundations, Chap., pp. 318–362. MIT, Cambridge (1986)

    Google Scholar 

  54. Safabakhsh, R., Adibi, P.: Nastaaligh handwritten word recognition using a continuous-density variable-duration HMM. Arab. J. Sci. Eng. 30(1B), 95–118 (2005)

    Google Scholar 

  55. Sari, T., Souici, L., Sellami, M.: Off-line handwritten Arabic character segmentation algorithm: ACSA. In: International Workshop on Frontiers in Handwriting Recognition, pp. 452–457 (2002)

  56. Schambach, M.P., Rottland, J., Alary, T.: How to convert a Latin handwriting recognition system to Arabic. In: International Conference Frontiers in Handwriting Recognition (2008)

  57. Schuster, M., Paliwal, K.K.: Bidirectional recurrent neural networks. IEEE Trans. Signal Process. 45, 2673–2681 (1997)

    Article  Google Scholar 

  58. Smith, R.: An overview of the Tesseract OCR engine. In: International Conference Document Analysis and Recognition, pp. 629–633 (2007)

  59. Wshah, S., Shi, Z., Govindaraju, V.: Segmentation of Arabic handwriting based on both contour and skeleton segmentation. In: International Conference Document Analysis and Recognition, pp. 793–797 (2009)

Download references

Acknowledgments

This work was supported by the Deanship of the Scientific Research in the University of Jordan. Some of this research was completed when G. Abandah was in a sabbatical leave in Princess Sumaya University for Technology. We would like to thank Alex Graves for making the RNNLIB publically available [19], for giving us a copy of the latest RNNLIB version, and for his help in using it. We also thank him for providing parts of the RNN sequence transcriber description included in Sect. 4. We would also like to thank Hanchuan Peng for making mRMR tools publically available [49]. We would like also to thank Haikal El Abed for giving us copies of sets f and s of the IfN/ENIT database.

Author information

Authors and Affiliations

  1. Computer Engineering Department, The University of Jordan, Amman, 11942, Jordan

    Gheith A. Abandah

  2. King Abdullah University of Science and Technology, Thuwal, Saudi Arabia

    Fuad T. Jamour

  3. Princess Sumaya University for Technology, Amman, Jordan

    Esam A. Qaralleh

Authors
  1. Gheith A. Abandah
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fuad T. Jamour
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Esam A. Qaralleh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gheith A. Abandah.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Abandah, G.A., Jamour, F.T. & Qaralleh, E.A. Recognizing handwritten Arabic words using grapheme segmentation and recurrent neural networks. IJDAR 17, 275–291 (2014). https://doi.org/10.1007/s10032-014-0218-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10032-014-0218-7

Keywords

Search

Navigation