Abstract
In recent years, systems based on deep learning have gained great popularity in the pattern recognition filed. This is basically to benefit from the hierarchical representations used to unlabeled data which is becoming the focus of many researchers since it represents the easiest way to deal with a huge amount of data. Most of the architecture in deep learning is constructed by a stack of feature extractors, such as Restricted Boltzmann Machine and Auto-Encoder. In this paper, we highlight how these deep learning techniques can be effectively applied for recognizing Arabic Handwritten Script (AHS) and this by investigating two deep architectures: Deep Belief Networks (DBN) and Convolutional Deep Belief Networks (CDBN) which are applied respectively on low-level dimension and high-level dimension in textual images. The experimental study has proved promising results which are comparable to the state-of-the-art Arabic OCR.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Mota, R., Scott, D.: Education and Innovation. Chapter in Education for Innovation and Independent Learning, pp. 55–71 (2014)
Porwal, U., Shi, Z., Setlur, S.: Machine learning in handwritten arabic text recognition. In: Chapter in Handbook of Statistics, vol. 31, pp. 443–46 (2013)
Bengio, Y., Lamblin, P., Popovici, D., Larochelle, H., Greedy layer-wise training of deep networks. In: Proceedings of Annual Conference on Neural Information Processing Systems (NIPS), pp. 153–160 (2006)
Ranzato, M., Boureau, Y., LeCun, Y.: Sparse feature learning for deep belief networks. In: Proceedings of Annual Conference on Neural Information Processing Systems (NIPS) (2007)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proceedings of the IEEE 86(11), 2278–2324 (1998)
Ranzato, M., Huang, F., Boureau, Y., LeCun, Y.: Unsupervised learning of invariant feature hierarchies with applications to object recognition. In: Proc. Computer Vision and Pattern Recognition Conference (CVPR). IEEE Press (2007)
Rumelhart, D., Hinton, G., Williams, R.: Learning representations by back-propagating errors. Nature 323, 533–536 (1986)
Larochelle, H., Bengio, Y., Louradour, J., Lamblin, P.: Exploring strategies for training deep neural networks. The Journal of Machine Learning Research 10, 1–40 (2009)
Hinton, G.E., Osindero, S., The, Y.W.: A fast learning algorithm for deep belief nets. Neural Computing 18(7), 1527–1554 (2006)
Olshausen, B.A., Field, D.J.: Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996)
Lee, H., Battle, A., Raina, R., Ng, A.Y.: Efficient sparse coding algorithms. In: Proceedings of Annual Conference on Neural Information Processing Systems (NIPS) (2007)
Hinton, G., Salakhutdinov, R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)
Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., Manzagol, P.A.: Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion. Journal of Machine Learning Research 11, 3371–3408 (2010)
Mohamed, A., Dahl, G., Hinton, G.: Acoustic modeling using deep belief networks. IEEE Transactions on Audio, Speech, and Language Processing 20(1), 14–22 (2011)
Dahl, G.E., Ranzato, M., Mohamed, A., Hinton, G.E.: Phone recognition with the mean-covariance restricted Boltzmann machine. Advances in Neural Information Processing Systems 23, 469–477 (2010)
Porwal, U., Zhou, Y., Govindaraju, V.: Handwritten arabic text recognition using deep belief networks. In: 21st International Conference on Pattern Recognition (ICPR) (2012)
Lee, H., Grosse, R., Ranganath, R., Ng, A.Y.: Unsupervised learning of hierarchical representations with convolutional deep belief networks. Communications of the ACM 54(10), 95–103 (2011)
Huang, G.-B., Zhou, H., Ding, X., Zhang, R.: Extreme Learning Machine for Regression and Multiclass Classification. IEEE Transactions on Systems, Man, and Cybernetics - Part B: Cybernetics 42(2), 513–529 (2012)
Lee, H., Pham, P.T., Largman, Y., Ng, A.Y.: Unsupervised feature learning for audio classification using convolutional deep belief networks. In: Advances in Neural Information Processing Systems (NIPS), pp. 1096–1104 (2009)
Ren, Y., Wu, Y.: Convolutional deep belief networks for feature extraction of EEG signal. In: International Joint Conference on Neural Networks (IJCNN), pp. 2850–2853 (2014)
Lee, H., Grosse, R., Ranganath, R., Ng, A.Y.: Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In: ICML (2009)
Mohamed, A., Sainath, T.N., Dahl, G., Ramabhadran, B., Hinton, G.E., Picheny, M.A.: Deep belief networks using discriminative features for phone recognition. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 5060–5063, May 2011
Hinton, G.E.: Training products of experts by minimizing contrastive divergence. Neural Computation 14(8), 1771–1800 (2002)
Hinton, G.E.: A practical guide to training restricted boltzmann machines. In: Montavon, G., Orr, G.B., Müller, K.-R. (eds.) Neural Networks: Tricks of the Trade. LNCS, vol. 7700, pp. 599–619. Springer, Heidelberg (2012)
Jarrett, K., Kavukcuoglu, K., Ranzato, M., LeCun, Y.: What is the best multistage architecture for object recognition? In: ICCV (2009)
Hinton, G.E.: Training products of experts by minimizing contrastive divergence. Neural Computation 14(8), 1771–1800 (2002)
Carreira-Perpinan, M.A., Hinton, G.E.: On contrastive divergence learning. In: Proceedings of the tenth International Workshop on Artificial Intelligence and Statistics, pp. 33–40, January 2005
Elleuch, M., Tagougui, N., Kherallah, M.: Arabic handwritten characters recognition using deep belief neural networks. In: 12th International Multi-Conference on Systems, Signals and Devices - Conference on Communication & Signal Processing (2015) (in press)
Lawgali, A., Angelova, M., Bouridane, A.: HACDB: handwritten arabic characters database for automatic character recognition. In: EUropean Workshop on Visual Information Processing (EUVIP), pp. 255–259 (2013)
Pechwitz, M., Maddouri, S.S., Märgner, V., Ellouze, N., Amiri, H.: IFN/ENIT database of handwritten Arabic words. In: Colloque International Francophone sur l’Ecrit et le Document (CIFED), pp. 127–136 (2002)
Albakoor, M., Saeed, K., Sukkar, F.: Intelligent system for arabic character recognition. In: World Congress on Nature & Biologically Inspired Computing (NaBIC), pp. 982–987 (2009)
Hamdi, R., Bouchareb, F., Bedda, M.: Handwritten arabic character recognition based on SVM classifier. In: 3rd International Conference on Information and Communication Technologies: From Theory to Applications (ICTTA) (2008)
Boubaker, H., Tagougui, N., Elbaati, A., Kherallah, M., Elabed, H., Alimi, A.M.: Online arabic databases and applications. In: Märgner, V., El Abed, H. (eds.) Guide to OCR for Arabic Scripts Chp. Part IV: Applications. Springer (2012)
AlKhateeb, H., Ren, J., Jiang, J., Al-Muhtaseb, H.: Offline handwritten Arabic cursive text recognition using Hidden Markov Models and re-ranking. Pattern Recognition Letters 32(8), 1081–1088 (2011)
Saabni, R.M., El-Sana, J.A.: Comprehensive synthetic Arabic database for on/off-line script recognition research. Int. J. Doc. Anal. Recognition (IJDAR) 16(3), 285–294 (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Elleuch, M., Tagougui, N., Kherallah, M. (2015). Deep Learning for Feature Extraction of Arabic Handwritten Script. In: Azzopardi, G., Petkov, N. (eds) Computer Analysis of Images and Patterns. CAIP 2015. Lecture Notes in Computer Science(), vol 9257. Springer, Cham. https://doi.org/10.1007/978-3-319-23117-4_32
Download citation
DOI: https://doi.org/10.1007/978-3-319-23117-4_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-23116-7
Online ISBN: 978-3-319-23117-4
eBook Packages: Computer ScienceComputer Science (R0)