Abstract
Machine Translation (MT) is one of the challenging tasks in the field of Natural Language Processing (NLP). The Convolutional Neural Network (CNN)-based approaches and Recurrent Neural Network (RNN)-based techniques have shown different capabilities in representing a piece of text. In this work, an hybrid CNN-RNN attention-based neural network is proposed. During training, Adam optimizer algorithm is used, and then, a popular regularization technique named dropout is applied in order to prevent some learning problems such as overfitting. The experiment results show the impact of our proposed system on the performance of Arabic machine translation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alayba, A.M., Palade, V., England, M., Iqbal, R.: A combined cnn and lstm model for arabic sentiment analysis. In: International Cross-Domain Conference for Machine Learning and Knowledge Extraction, pp. 179–191 (2018)
Alqudsi, A., Omar, N., Shaker, K.: Arabic machine translation: a survey. Artif. Intell. Rev. 42(4), 549–572 (2014)
Antoun, W., Baly, F., Hajj, H.M.: Arabert: transformer-based model for arabic language understanding (2020) . CoRR abs/2003.00104
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: Bengio, Y., LeCun, Y. (eds) 3rd International Conference on Learning Representations, ICLR (2015)
Bastings, J., Filippova, K.: The elephant in the interpretability room: why use attention as explanation when we have saliency methods? In: Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, pp. 149–155. Association for Computational Linguistics (2020)
Bensalah, N., Ayad, H., Adib, A., el farouk, A.I.: Combining word and character embeddings in Arabic Chatbots. In: Advanced Intelligent Systems for Sustainable Development, AI2SD’2020, Tangier, Morocco (2020)
Bensalah, N., Ayad, H., Adib, A., Farouk, A.I.E.: LSTM or GRU for Arabic machine translation? Why not both! In: International Conference on Innovation and New Trends in Information Technology, INTIS 2019, Tangier, Morocco, Dec 20–21 (2019)
Bensalah, N., Ayad, H., Adib, A., Farouk, A.I.E.: Arabic machine translation based on the combination of word embedding techniques. In: Intelligent Systems in Big Data, Semantic Web and Machine Learning (2020)
Bensalah, N., Ayad, H., Adib, A., Farouk, A.I.E.: Arabic sentiment analysis based on 1-D convolutional neural network. In: International Conference on Smart City Applications, SCA20 (2020)
Bojanowski, P., Grave, E., Joulin, A., Mikolov, T.: Enriching word vectors with subword information. Trans. Assoc. Comput. Linguist., 135–146 (2017)
Cheng, J., Dong, L., Lapata, M.: Long short-term memory-networks for machine reading. In: Su, J., Carreras, X., Duh, K. (eds) Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP, pp. 551–561 (2016)
Cho, K., van Merrienboer, B., Bahdanau, D., Bengio, Y.: On the properties of neural machine translation: encoder-decoder approaches. In: Proceedings of SSST@EMNLP 2014, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation, pp. 103–111. Association for Computational Linguistics (2014)
Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., Bengio, Y.: Learning phrase representations using rnn encoder-decoder for statistical machine translation (2014). arXiv preprint arXiv:1406.1078
Ciresan, D.C., Meier, U., Schmidhuber, J.: Multi-column deep neural networks for image classification. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, pp. 3642–3649. IEEE Computer Society (2012)
Elman, J.L.: Finding structure in time. Cognit. Sci. 14(2), 179–211 (1990)
Feurer, M., Hutter, F.: Hyperparameter optimization. In: Automated Machine Learning, pp. 3–33. Springer (2019)
Gehring, J., Auli, M., Grangier, D., Dauphin, Y.N.: A convolutional encoder model for neural machine translation. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, pp. 123–135. Association for Computational Linguistics (2017)
Graves, A., Schmidhuber, J.: Framewise phoneme classification with bidirectional lstm and other neural network architectures. Neural Netw. 18(5–6), 602–610 (2005)
Guo, L., Zhang, D., Wang, L., Wang, H., Cui, B.: Cran: a hybrid CNN-RNN attention-based model for text classification. In: International Conference on Conceptual Modeling, pp. 571–585. Springer (2018)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P., et al.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Lin, Z., Feng, M., dos Santos, C.N., Yu, M., Xiang, B., Zhou, B., Bengio, Y.: A structured self-attentive sentence embedding. In: 5th International Conference on Learning Representations, ICLR (2017)
Luong, M., Pham, H., Manning, C.D.: Effective Approaches to Attention-based Neural Machine Translation. CoRR abs/1508.04025 (2015)
Papineni, K., Roukos, S., Ward, T., Zhu, W.-J.: Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pp. 311–318. Association for Computational Linguistics (2002)
Schuster, M., Paliwal, K.K.: Bidirectional recurrent neural networks. IEEE Trans. Sig. Process. 45(11), 2673–2681 (1997)
Sutskever, I., Vinyals, O., Le, Q.: Sequence to sequence learning with neural networks. Advances in NIPS (2014)
Wang, Y.-Y., Acero, A., Chelba, C.: Is word error rate a good indicator for spoken language understanding accuracy. In: 2003 IEEE Workshop on Automatic Speech Recognition and Understanding (IEEE Cat. No. 03EX721), pp. 577–582. IEEE (2003)
Wu, Y., Schuster, M., Chen, Z., Le, Q.V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., Klingner, J., Shah, A., Johnson, M., Liu, X., Kaiser, L., Gouws, S., Kato, Y., Kudo, T., Kazawa, H., Stevens, K., Kurian, G., Patil, N., Wang, W., Young, C., Smith, J., Riesa, J., Rudnick, A., Vinyals, O., Corrado, G., Hughes, M., Dean, J.: Google’s neural machine translation system: bridging the gap between human and machine translation. CoRR abs/1609.08144 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Bensalah, N., Ayad, H., Adib, A., Ibn El Farouk, A. (2022). CRAN: An Hybrid CNN-RNN Attention-Based Model for Arabic Machine Translation. In: Ben Ahmed, M., Teodorescu, HN.L., Mazri, T., Subashini, P., Boudhir, A.A. (eds) Networking, Intelligent Systems and Security. Smart Innovation, Systems and Technologies, vol 237. Springer, Singapore. https://doi.org/10.1007/978-981-16-3637-0_7
Download citation
DOI: https://doi.org/10.1007/978-981-16-3637-0_7
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-3636-3
Online ISBN: 978-981-16-3637-0
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)