Abstract
The same Machine Translation (MT) approach may not work for European languages as for Arabic, because of its structure. MT based on Neural Networks methods has recently become an alternative approach to the statistical MT. In this paper, a case study is presented on how different sequence to sequence Deep Learning (DL) models perform in the task of Arabic MT. A comprehensive comparison between these models based mainly on: Long Short Term Memory (LSTM), Gated Recurrent Unit (GRU), Bidirectional LSTM (BiLSTM) and Bidirectional GRU (BiGRU) is presented. Specifically, each input sequence will be translated into English one using an Encoder-Decoder model based on the four architectures with an attention mechanism. Furthermore, we study the impact of different preprocessing techniques on Arabic MT.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abdelali, A., Darwish, K., Durrani, N., Mubarak, H. Farasa: a fast and furious segmenter for Arabic. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations, pp. 11–16 (2016)
Alqudsi, A., Omar, N., Shaker, K.: Arabic machine translation: a survey. Artif. Intell. Rev. 42(4), 549–572 (2014)
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate (2014). Accepted at ICLR 2015 as oral presentation
Bensalah, N., Ayad, H., Adib, A., El farouk, A.I.: Combining word and character embeddings in Arabic chatbots. In: Advanced Intelligent Systems for Sustainable Development, AI2SD 2020, Tangier, Morocco (2020)
Bojanowski, P., Grave, E., Joulin, A., Mikolov, T.: Enriching word vectors with subword information. Trans. Assoc. Comput. Linguist. 5, 135–146 (2017)
Bouny, L.E., Khalil, M., Adib, A.: ECG heartbeat classification based on multi-scale wavelet convolutional neural networks. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain, Mai 4–8 (2020)
Cho, K., van Merriënboer, B., Bahdanau, D., Bengio, Y.: On the properties of neural machine translation: encoder–decoder approaches. In: Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation (2014)
Cho, K., van Merriënboer, B., Gülçehre, a., Bahdanau, D., Bougares, F., Schwenk, H., Bengio, Y.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: EMNLP ACL (2014)
Devlin, J., Zbib, R., Huang, Z., Lamar, T., Schwartz, R., Makhoul, J.: Fast and robust neural network joint models for statistical machine translation. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (2014)
El Kholy, A., Habash, N. Orthographic and morphological processing for English-Arabic statistical machine translation. Mach. Transl. 26, 25–45 (2012)
Feurer, M., Hutter, F.: Hyperparameter optimization. In: Automated Machine Learning. Springer, pp. 3–33 (2019)
Graves, A., Schmidhuber, J.: Framewise phoneme classification with bidirectional lstm and other neural network architectures. Neural Netw. 18(5–6), 602–610 (2005)
Habash, N., Sadat, F.: Arabic preprocessing schemes for statistical machine translation. In: Proceedings of Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics (2006)
Hanin, B.: Which neural net architectures give rise to exploding and vanishing gradients? In: NeurIPS (2018)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. (1997)
Hutchins, W.J., Somers, H.L.: An Introduction to Machine Translation. Academic Press, Cambridge (1992)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: 3rd International Conference on Learning Representations, ICLR (2015)
Nagao, M.: A framework of a mechanical translation between Japanese and English by analogy principle. In: Artificial and Human Intelligence, North-Holland (1984)
Papineni, K., Roukos, S., Ward, T., Zhu, W.-J.: Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pp. 311–318. Association for Computational Linguistics (2002)
Schuster, M., Paliwal, K.K.: Bidirectional recurrent neural networks. IEEE Trans. Sign. Process. 45(11), 2673–2681 (1997)
Sekkate, S., Khalil, M., Adib, A., Jebara, S.B.: An investigation of a feature-level fusion for noisy speech emotion recognition. Comput. 8, 91 (2019)
Tiedemann, J.: Parallel data, tools and interfaces in OPUS. In: Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC 2012) Istanbul, Turkey, pp. 2214–2218 (2012)
Turing, A.M.: Computing machinery and intelligence. Mind (1950)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Bensalah, N., Ayad, H., Adib, A., Ibn El Farouk, A. (2021). LSTM vs. GRU for Arabic Machine Translation. In: Abraham, A., et al. Proceedings of the 12th International Conference on Soft Computing and Pattern Recognition (SoCPaR 2020). SoCPaR 2020. Advances in Intelligent Systems and Computing, vol 1383. Springer, Cham. https://doi.org/10.1007/978-3-030-73689-7_16
Download citation
DOI: https://doi.org/10.1007/978-3-030-73689-7_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-73688-0
Online ISBN: 978-3-030-73689-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)