Skip to main content
SpringerLink
Log in

An Arabic Chatbot Leveraging Encoder-Decoder Architecture Enhanced with BERT

  • Conference paper
  • First Online:
Recent Trends in Image Processing and Pattern Recognition (RTIP2R 2023)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 2026))

  • 106 Accesses

Abstract

This paper introduces a new method for developing an Arabic chatbot, utilizing the encoder-decoder architecture enriched with BERT embeddings. Our distinct dataset, constructed manually, aids the model in comprehending intricate questions and prompts, thereby generating coherent and contextually accurate responses in Arabic. The dataset comprises 81,659 manually created conversation pairs. Our model successfully delivered the anticipated answers. We employed a model with a warm-start using the BERT2BERT encoder and decoder. It achieved a BLEU score of 3.52 and a PPL of 36.3.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Luger, E., Sellen, A.: “Like having a really bad PA”: the gulf between user expectation and experience of conversational agents. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (2016)

    Google Scholar 

  2. https://lingua.edu/the-most-spoken-languages-in-the-world/

  3. Habash, N.: Introduction to Arabic natural language processing. In: Introduction to Arabic Natural Language Processing (2010)

    Google Scholar 

  4. Hasanuzzaman, H.: Arabic language: characteristics and importance. Echo. J. Humanit. Soc. Sci. 1(3), 11–16 (2013)

    Google Scholar 

  5. Ahmed, A., Ali, N., Alzubaidi, M.S., Zaghouani, W., Abd-alrazaq, A.A., Househ, M.J.: Arabic chatbot technologies: a scoping review. Comput. Methods Programs Biomed. Update (2022)

    Google Scholar 

  6. Reimers, N., Gurevych, I.: Sentence-BERT: sentence embeddings using siamese BERT-networks. In: Conference on Empirical Methods in Natural Language Processing (2019)

    Google Scholar 

  7. Yu, S.Y., Chen, Y., Zaidi, H.: AVA: a financial service chatbot based on deep bidirectional transformers. Front. Appl. Math. Stat. 7, 604842 (2020)

    Article  Google Scholar 

  8. Ezen-Can, A.: A Comparison of LSTM and BERT for Small Corpus. arXiv abs/2009.05451 (2020)

    Google Scholar 

  9. Amer, E., Hazem, A., Farouk, O., Louca, A., Mohamed, Y., Ashraf, M.: A proposed chatbot framework for COVID-19. In: 2021 International Mobile, Intelligent, and Ubiquitous Computing Conference (MIUCC), pp. 263–268 (2021)

    Google Scholar 

  10. Kanodia, N., Ahmed, K., Miao, Y.: Question answering model based conversational chatbot using BERT model and google dialogflow. In: 2021 31st International Telecommunication Networks and Applications Conference (ITNAC), pp. 19–22 (2021)

    Google Scholar 

  11. Dong, L., et al.: Unified language model pre-training for natural language understanding and generation. In: Neural Information Processing Systems (2019)

    Google Scholar 

  12. Park, H., Moon, G., Kim, K.S.: Classification of covid-19 symptom for chatbot using bert (2021)

    Google Scholar 

  13. Yoo, S., Jeong, O.: An intelligent chatbot utilizing BERT model and knowledge graph. J. Soc. e-Bus. Stud. 24(3) (2020)

    Google Scholar 

  14. Varitimiadis, S., Kotis, K., Pittou, D., Konstantakis, G.: Graph-based conversational AI: towards a distributed and collaborative multi-chatbot approach for museums. Appl. Sci. (2021)

    Google Scholar 

  15. Fuad, A., Al-Yahya, M.M.: Recent developments in arabic conversational AI: a literature review. IEEE Access 10, 23842–23859 (2022)

    Article  Google Scholar 

  16. El-Ramly, M., et al.: CairoDep: detecting depression in Arabic posts using BERT transformers. In: 2021 Tenth International Conference on Intelligent Computing and Information Systems (ICICIS), pp. 207–212 (2021)

    Google Scholar 

  17. Helwe, C., Dib, G., Shamas, M., Elbassuoni, S.: A semi-supervised BERT approach for Arabic named entity recognition. In: Workshop on Arabic Natural Language Processing (2020)

    Google Scholar 

  18. Gaanoun, K., Benelallam, I.: Arabic dialect identification: an Arabic-BERT model with data augmentation and ensembling strategy. In: Workshop on Arabic Natural Language Processing (2020)

    Google Scholar 

  19. Jarrar, M., Khalilia, M., Ghanem, S.: Wojood: nested Arabic named entity corpus and recognition using BERT. In: International Conference on Language Resources and Evaluation (2022)

    Google Scholar 

  20. Ghaddar, A., et al.: JABER: junior Arabic BERt. arXiv abs/2112.04329 (2021)

    Google Scholar 

  21. Almaliki, M., Almars, A.M., Gad, I., Atlam, E.: ABMM: Arabic BERT-mini model for hate-speech detection on social media. Electronics 12(4), 1048 (2023)

    Article  Google Scholar 

  22. Chouikhi, H., Chniter, H., Jarray, F.: Arabic sentiment analysis using BERT model. In: International Conference on Computational Collective Intelligence (2021)

    Google Scholar 

  23. Talafha, B., et al.: Multi-dialect Arabic BERT for Country-level Dialect Identification. arXiv abs/2007.05612 (2020)

    Google Scholar 

  24. Boussakssou, M., Ezzikouri, H., Erritali, M.: Chatbot in Arabic language using seq to seq model. Multimedia Tools Appl. 81, 2859–2871 (2021)

    Article  Google Scholar 

  25. Antoun, W., Baly, F., Hajj, H.M.: AraELECTRA: Pre-Training Text Discriminators for Arabic Language Understanding. arXiv abs/2012.15516 (2020)

    Google Scholar 

  26. Naous, T., Antoun, W., Mahmoud, R.A., Hajj, H.M.: Empathetic BERT2BERT conversational model: learning Arabic language generation with little data. In: Workshop on Arabic Natural Language Processing (2021)

    Google Scholar 

  27. Rothe, S., Narayan, S., Severyn, A.: Leveraging pre-trained checkpoints for sequence generation tasks. Trans. Assoc. Comput. Linguist. 8, 264–280 (2019)

    Article  Google Scholar 

  28. Alruqi, T.N., Alzahrani, S.M.: Evaluation of an Arabic Chatbot Based on Extractive Question-Answering Transfer Learning and Language Transformers. AI (2023)

    Google Scholar 

  29. Roller, S., et al.: Recipes for building an open-domain chatbot. In: Conference of the European Chapter of the Association for Computational Linguistics (2020)

    Google Scholar 

  30. Alammary, A.S.: BERT models for Arabic text classification: a systematic review. Appl. Sci. 12(11), 5720 (2022)

    Article  Google Scholar 

  31. Abuzayed, A., Al-Khalifa, H.S.: BERT for Arabic topic modeling: an experimental study on BERTopic technique. In: International Conference on Arabic Computational Linguistics (2021)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Data4earth Laboratory, FST Beni Mellal, Sultan Moulay Slimane University, Beni Mellal, Morocco

    Mohamed Boussakssou & Mohamed Erritali

Authors
  1. Mohamed Boussakssou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohamed Erritali
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohamed Boussakssou .

Editor information

Editors and Affiliations

  1. University of South Dakota, Vermillion, SD, USA

    KC Santosh

  2. University of Derby, Derby, UK

    Aaisha Makkar

  3. University of Derby, Derby, UK

    Myra Conway

  4. Indian Institute of Information Technology, Bhopal, India

    Ashutosh K. Singh

  5. University of Clermont Auvergne, Aubière, France

    Antoine Vacavant

  6. ENSA - Cadi Ayyad University, Marrakesh, Morocco

    Anas Abou el Kalam

  7. Halmstad University, Halmstad, Sweden

    Mohamed-Rafik Bouguelia

  8. Central University of Karnataka, Kalaburagi, India

    Ravindra Hegadi

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Boussakssou, M., Erritali, M. (2024). An Arabic Chatbot Leveraging Encoder-Decoder Architecture Enhanced with BERT. In: Santosh, K., et al. Recent Trends in Image Processing and Pattern Recognition. RTIP2R 2023. Communications in Computer and Information Science, vol 2026. Springer, Cham. https://doi.org/10.1007/978-3-031-53082-1_21

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-53082-1_21

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-53081-4

  • Online ISBN: 978-3-031-53082-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation