Skip to main content
SpringerLink
Log in

Improving MDLSTM for Offline Arabic Handwriting Recognition Using Dropout at Different Positions

  • Conference paper
  • First Online:
Book cover Artificial Neural Networks and Machine Learning – ICANN 2016 (ICANN 2016)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 9887))

Included in the following conference series:

Abstract

RNN and LSTM are now a state-of-the-art technology that provide a very good performance on different machine learning tasks as handwritten Arabic word recognition. This field remains an on-going research problem due to its cursive appearance, the variety of writers and the diversity of styles. In this work, we propose a new offline Arabic handwriting recognition system based on a particular RNN named the MDLSTM on which we propose to apply dropout technique in different positions such as before, after or inside the MDLSTM layers. This regularization technique has the advantages of preventing our system against overfitting problem and reducing the error recognition rate. We carried out experiments on the well-known IFN/ENIT Database.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Sundermeyer, M., Schlüter, R., Ney, H.: LSTM neural networks for language modeling. In: INTERSPEECH, pp. 194–197, September 2012

    Google Scholar 

  2. Sak, H., Senior, A.W., Beaufays, F.: Long short-term memory recurrent neural network architectures for large scale acoustic modeling. In: INTERSPEECH, pp. 338–342, September 2014

    Google Scholar 

  3. Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems, pp. 3104–3112 (2014)

    Google Scholar 

  4. Vinyals, O., Toshev, A., Bengio, S., Erhan, D.: Show and tell: a neural image caption generator. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3156–3164 (2015)

    Google Scholar 

  5. Graves, A., Liwicki, M., Bunke, H., Schmidhuber, J., Fernández, S.: Unconstrained on-line handwriting recognition with recurrent neural networks. In: Advances in Neural Information Processing Systems, pp. 577–584 (2008)

    Google Scholar 

  6. Graves, A.: Offline arabic handwriting recognition with multidimensional recurrent neural networks. In: Märgner, V., El Abed, H. (eds.) Guide to OCR for Arabic Scripts, pp. 297–313. Springer, London (2012)

    Chapter  Google Scholar 

  7. Pechwitz, M., Maddouri, S.S., Märgner, V., Ellouze, N., Amiri, H.: IFN/ENIT-database of handwritten Arabic words. In: Proceedings of the CIFED, vol. 2, pp. 127–136, October 2002

    Google Scholar 

  8. Slimane, F., Zayene, O., Kanoun, S., Alimi, A.M., Hennebert, J., Ingold, R.: New features for complex Arabic fonts in cascading recognition system. In: 2012 21st International Conference on Pattern Recognition (ICPR), pp. 738–741. IEEE, November 2012

    Google Scholar 

  9. Dreuw, P., Doetsch, P., Plahl, C., Ney, H.: Hierarchical hybrid MLP/HMM or rather MLP features for a discriminatively trained gaussian HMM: a comparison for offline handwriting recognition. In: 2011 18th IEEE International Conference on Image Processing (ICIP), pp. 3541–3544. IEEE, September 2011

    Google Scholar 

  10. Graves, A., Mohamed, A.R., Hinton, G.: Speech recognition with deep recurrent neural networks. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6645–6649. IEEE, May 2013

    Google Scholar 

  11. Kozielski, M., Doetsch, P., Ney, H.: Improvements in RWTH’s system for off-line handwriting recognition. In: 2013 12th International Conference on Document Analysis and Recognition (ICDAR), pp. 935–939. IEEE, August 2013

    Google Scholar 

  12. Graves, A.: Supervised sequence labelling, pp. 5–13. Springer, Heidelberg (2012)

    MATH  Google Scholar 

  13. Hinton, G.E., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.R.: Improving neural networks by preventing co-adaptation of feature detectors (2012). arXiv preprint: arXiv:1207.0580

  14. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)

    MathSciNet  MATH  Google Scholar 

  15. Miao, Y., Metze, F.: Improving low-resource CD-DNN-HMM using dropout and multilingual DNN training (2013)

    Google Scholar 

  16. Zhang, S., Bao, Y., Zhou, P., Jiang, H., Dai, L.: Improving deep neural networks for LVCSR using dropout and shrinking structure. In: 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6849–6853. IEEE, May 2014

    Google Scholar 

  17. Pham, V., Bluche, T., Kermorvant, C., Louradour, J.: Dropout improves recurrent neural networks for handwriting recognition. In: 2014 14th International Conference on Frontiers in Handwriting Recognition (ICFHR), pp. 285–290. IEEE, September 2014

    Google Scholar 

  18. Maalej, R., Tagougui, N., Kherallah, M.: Online Arabic handwriting recognition with dropout applied in deep recurrent neural networks. In: 2016 12th IAPR International Workshop on Document Analysis Systems (DAS), pp. 418–421. IEEE, April 2016

    Google Scholar 

  19. Maalej, R., Tagougui, N., Kherallah, M.: Recognition of handwritten Arabic words with dropout applied in MDLSTM. In: Campilho, A., Karray, F. (eds.) ICIAR 2016. LNCS, vol. 9730, pp. 746–752. Springer, Heidelberg (2016). doi:10.1007/978-3-319-41501-7_83

    Chapter  Google Scholar 

  20. El Abed, H., Märgner, V.: ICDAR 2009-Arabic handwriting recognition competition. Int. J. Doc. Anal. Recogn. (IJDAR) 14(1), 3–13 (2011)

    Article  Google Scholar 

  21. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  22. Graves, A., Fernández, S., Gomez, F., Schmidhuber, J.: Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks, June 2006

    Google Scholar 

  23. Baldi, P., Sadowski, P.J.: Understanding dropout. In: Advances in Neural Information Processing Systems, pp. 2814–2822 (2013)

    Google Scholar 

  24. Wang, S.I., Manning, C.D.: Fast dropout training. In: ICML, vol. 2, pp. 118–126 (2013)

    Google Scholar 

  25. Bayer, J., Osendorfer, C., Korhammer, D., Chen, N., Urban, S., van der Smagt, P.: On fast dropout and its applicability to recurrent networks (2013). arXiv preprint: arXiv:1311.0701

Download references

Author information

Authors and Affiliations

  1. Research Group on Intelligent Machines, National School of Engineers of Sfax, Sfax University, Sfax, Tunisia

    Rania Maalej

  2. Faculty of Sciences, Sfax University, Sfax, Tunisia

    Monji Kherallah

Authors
  1. Rania Maalej
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Monji Kherallah
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rania Maalej .

Editor information

Editors and Affiliations

  1. University of Lausanne, Lausanne, Switzerland

    Alessandro E.P. Villa

  2. University of Lausanne, Lausanne, Switzerland

    Paolo Masulli

  3. Universitat Politécnica de Catalunya, Terrrassa, Spain

    Antonio Javier Pons Rivero

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Maalej, R., Kherallah, M. (2016). Improving MDLSTM for Offline Arabic Handwriting Recognition Using Dropout at Different Positions. In: Villa, A., Masulli, P., Pons Rivero, A. (eds) Artificial Neural Networks and Machine Learning – ICANN 2016. ICANN 2016. Lecture Notes in Computer Science(), vol 9887. Springer, Cham. https://doi.org/10.1007/978-3-319-44781-0_51

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-44781-0_51

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-44780-3

  • Online ISBN: 978-3-319-44781-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation