Skip to main content
SpringerLink
Log in

Transformer Model and Convolutional Neural Networks (CNNs) for Arabic to English Machine Translation

  • Conference paper
  • First Online:
Proceedings of the 5th International Conference on Big Data and Internet of Things (BDIoT 2021)

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 489))

Included in the following conference series:

Abstract

Arabic remains one of the richest languages in the world in terms of vocabulary with a large set of morphological features and relatively few resources compared to English. Given the challenge posed by the morphological richness of this language, Arabic Natural Language Processing (NLP) tasks like Named Entity Recognition (NER), Sentiment Analysis (SA), Question Answering (QA) and Machine Translation (MT) have proved to be very challenging to handle. Recently, the transformers based models, have proved to be very effective in terms of language understanding, and have obtained state-of-the-art results for many NLP tasks and in particular MT. In this paper, we proposed a novel Deep Learning architecture based on the Convolutional Neural Networks (CNNs) and the transformer model to further improve the results obtained on the Arabic-English Neural Machine Translation task. Moreover, a special preprocessing of the Arabic sentences based mainly on Farasa and AraBERT is carried out. Experiments on the UN Arabic-English datasets show that our approach outperforms the state-of-the-art Arabic MT systems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 219.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 279.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abdelali, A., Darwish, K., Durrani, N., Mubarak, H.: Farasa: a fast and furious segmenter for arabic. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations, pp. 11–16 (2016)

    Google Scholar 

  2. Abdelali, A., Guzmán, F., Sajjad, H., Vogel, S.: The AMARA corpus: building parallel language resources for the educational domain. In: Proceedings of the Ninth International Conference on Language Resources and Evaluation, LREC, pp. 1856–1862 (2014)

    Google Scholar 

  3. Alqudsi, A., Omar, N., Shaker, K.: Arabic machine translation: a survey. Artifi. Intell. Rev. 42(4), 549–572 (2012). https://doi.org/10.1007/s10462-012-9351-1

    Article  Google Scholar 

  4. Alrajeh, A.: A recipe for Arabic-English neural machine translation. CoRR abs/1808.06116 (2018)

    Google Scholar 

  5. Hadj Ameur, M.S., Guessoum, A., Meziane, F.: Improving Arabic neural machine translation via n-best list re-ranking. Mach. Translation 33(4), 279–314 (2019). https://doi.org/10.1007/s10590-019-09237-6

    Article  Google Scholar 

  6. Antoun, W., Baly, F., Hajj, H.M.: Arabert: transformer-based model for arabic language understanding. CoRR abs/2003.00104 (2020)

    Google Scholar 

  7. Ba, L.J., Kiros, J.R., Hinton, G.E.: Layer normalization. CoRR abs/1607.06450 (2016)

    Google Scholar 

  8. Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7–9, 2015, Conference Track Proceedings (2015)

    Google Scholar 

  9. Bensalah, N., Ayad, H., Adib, A., Ibn el farouk, A.: Combining word and character embeddings for Arabic chatbots. In: Kacprzyk, J., Balas, V.E., Ezziyyani, M. (eds.) AI2SD 2020. AISC, vol. 1417, pp. 571–578. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-90633-7_48

    Chapter  Google Scholar 

  10. Bensalah, N., Ayad, H., Adib, A., El Farouk, A.I.: Arabic machine translation based on the combination of word embedding techniques. In: Gherabi, N., Kacprzyk, J. (eds.) Intelligent Systems in Big Data, Semantic Web and Machine Learning. AISC, vol. 1344, pp. 247–260. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-72588-4_17

    Chapter  Google Scholar 

  11. Bensalah, N., Ayad, H., Adib, A., Farouk, A.I.E.: Arabic sentiment analysis based on 1-D convolutional neural network. In: International Conference on Smart City Applications, SCA20, Safranbolu, Turkey(2020)

    Google Scholar 

  12. Chen, B., Cettolo, M., Federico, M.R.: Rules for phrase-based statistical machine translation. In: International workshop on spoken language translation. IWSLT 2006, pp. 182–189 (2006)

    Google Scholar 

  13. Chung, J., Gülçehre, Ç., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. CoRR abs/1412.3555 (2014)

    Google Scholar 

  14. Gehring, J., Auli, M., Grangier, D., Dauphin, Y.N.: A convolutional encoder model for neural machine translation. In: Barzilay, R., Kan, M. (eds.) Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL, pp. 123–135 (2017)

    Google Scholar 

  15. Habash, N., Sadat, F.: Arabic preprocessing schemes for statistical machine translation. In: Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics (2006)

    Google Scholar 

  16. Habash, N., Zalmout, N., Taji, D., Hoang, H., Alzate, M.: A parallel corpus for evaluating machine translation between Arabic and European languages. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, Vol. 2, Short Papers, pp. 235–241 (2017)

    Google Scholar 

  17. Hadj Ameur, M., Meziane, F., Guessoum, A.: Arabic machine translation: a survey of the latest trends and challenges. Comput. Sci. Rev. 38, 22 (2020)

    Google Scholar 

  18. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016. IEEE Computer Society, pp. 770–778 (2016)

    Google Scholar 

  19. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  20. Józefowicz, R., Vinyals, O., Schuster, M., Shazeer, N., Wu, Y.: Exploring the limits of language modeling. CoRR abs/1602.02410 (2016)

    Google Scholar 

  21. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. In: Proceedings of the IEEE (1998)

    Google Scholar 

  22. Luong, M., Pham, H., Manning, C.D.: Effective approaches to attention-based neural machine translation. CoRR abs/1508.04025 (2015)

    Google Scholar 

  23. Meng, F., Lu, Z., Wang, M., Li, H., Jiang, W., Liu, Q.: Encoding source language with convolutional neural network for machine translation. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL, pp. 20–30 (2015)

    Google Scholar 

  24. Ombabi, A.H., Ouarda, W., Alimi, A.M.: Deep learning CNN–LSTM framework for Arabic sentiment analysis using textual information shared in social networks. Soc. Netw. Anal. Min. 10(1), 1–13 (2020). https://doi.org/10.1007/s13278-020-00668-1

    Article  Google Scholar 

  25. Oudah, M., Almahairi, A., Habash, N.: The impact of preprocessing on Arabic-English statistical and neural machine translation. In: Proceedings of Machine Translation Summit XVII vol. 1: Research Track, MTSummit, pp. 214–221 (2019)

    Google Scholar 

  26. Papineni, K., Roukos, S., Ward, T., Zhu, W.-J.: Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, Association for Computational Linguistics, pp. 311–318 (2002)

    Google Scholar 

  27. Sadat, F., Habash, N.: Combination of Arabic preprocessing schemes for statistical machine translation. In: ACL 2006, 21st International Conference on Computational Linguistics (2006)

    Google Scholar 

  28. Shapiro, P., Duh, K.: Morphological word embeddings for Arabic neural machine translation in low-resource settings. In: Proceedings of the Second Workshop on Subword/Character LEvel Models, pp. 1–11 (2018)

    Google Scholar 

  29. Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, Montreal, Quebec, Canada, 8–13 December 2014, pp. 3104–3112 (2014)

    Google Scholar 

  30. Tiedemann, J.: Parallel data, tools and interfaces in OPUS. In: Proceedings of the Eighth International Conference on Language Resources and Evaluation, LREC, pp. 2214–2218 (2012)

    Google Scholar 

  31. Vaswani, A., Shazeer, N., Parmar, N.: Attention is all you need. In: NIPS (2017)

    Google Scholar 

  32. Yao, Y., Rosasco, L., Caponnetto, A.: On early stopping in gradient descent learning. Constructive Approximation 26, 289–315 (2007)

    Article  MathSciNet  Google Scholar 

  33. Zhang, G., Fan, X., Jin, C., Wu, M.: Open-domain document-based automatic QA models based on CNN and attention mechanism. In: 2019 IEEE International Conference on Big Knowledge, ICBK, pp. 326–332 (2019)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Team Networks, Telecoms and Multimedia, University of Hassan II Casablanca, Casablanca, 20000, Morocco

    Nouhaila Bensalah, Habib Ayad & Abdellah Adib

  2. Teaching, Languages and Cultures Laboratory Mohammedia, Casablanca, Morocco

    Abdelhamid Ibn El Farouk

Authors
  1. Nouhaila Bensalah
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Habib Ayad
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Abdellah Adib
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Abdelhamid Ibn El Farouk
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nouhaila Bensalah .

Editor information

Editors and Affiliations

  1. ENSIAS, Mohammed V University, Rabat, Morocco

    Mohamed Lazaar

  2. UNILEHAVRE, UNIROUEN, Normandie Université, Le Havre, France

    Claude Duvallet

  3. Vrije Universiteit Brussel, Brussels, Belgium

    Abdellah Touhafi

  4. ENSA, Abdelmalek Essaâdi University, Tetuan, Morocco

    Mohammed Al Achhab

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bensalah, N., Ayad, H., Adib, A., Farouk, A.I.E. (2022). Transformer Model and Convolutional Neural Networks (CNNs) for Arabic to English Machine Translation. In: Lazaar, M., Duvallet, C., Touhafi, A., Al Achhab, M. (eds) Proceedings of the 5th International Conference on Big Data and Internet of Things. BDIoT 2021. Lecture Notes in Networks and Systems, vol 489. Springer, Cham. https://doi.org/10.1007/978-3-031-07969-6_30

Download citation

Publish with us

Policies and ethics

Search

Navigation