Abstract
This paper introduces DALex, a Dialectal Arabic Lexicon that comprises 4 major regional Arabic dialects: Maghrebi, Egyptian, Levantine and Peninsular or Gulf Arabic, covering about 75 % of the Arabic dialects spoken sub-regionally. We describe the approach used to compile the lexicon and the inference algorithms that extract semantically similar words from the different dialects lists of the word-topic distribution generated by Probabilistic Topic Models sampled on dialectal comparable documents that are harvested from web blogs and forums. The power of this Lexicon lies in its automatic, dialect independent way of construction that makes it easily created and expanded. Furthermore, the wide coverage of multiple dialectal variants, we believe, will be a great resource for future Computational Linguistics research.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
- 3.
Arabic transliteration is presented in the Habash-Soudi-Buckwalter scheme [22]
. - 4.
.References
Ferguson, C.A.: Diglossia. Word, 15, 325–340. Reprinted in PP Giglioli. Language and social context, 232–251 (1959)
Zaidan, O.F., Callison-Burch, C.: The arabic online commentary dataset: an annotated dataset of informal arabic with high dialectal content. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: Short Papers (2011)
Cotterell, R., Callison-Burch, C., A multi-dialect, multi-genre corpus of informal written Arabic. In: Proceedings of the Language Resources and Evaluation Conference (LREC) (2014)
Graff, D., Maamouri, M., Developing LMF-XML bilingual dictionaries for colloquial Arabic dialects. In: LREC (2012)
Diab, M., Al-Badrashiny, M., Aminian, M., Attia, M., Dasigi, P., Elfardy, H., Eskander, R., Habash, N., Hawwari, A., Salloum, W.: Tharwa: a large scale dialectal arabic-standard Arabic-English lexicon. In: Proceedings of the Language Resources and Evaluation Conference (LREC) (2014)
Hamdi, A., Gala, N., Nasr, A., Automatically building a Tunisian lexicon for deverbal nouns. In: COLING 2014 (2014)
Boujelbane, R., Khemekhem, M.E., Belguith, L.H.: Mapping rules for building a tunisian dialect lexicon and generating corpora. In: Proceedings of the International Joint Conference on Natural Language Processing (2013)
Al-Sabbagh, R., Girju, R., Mining the web for the induction of a dialectical Arabic lexicon. In: LREC (2010)
Habash, N.Y.: Introduction to Arabic Natural Language Processing, vol. 3. Morgan & Claypool Publishers, Dordrecht (2010)
Watson, J.C.: The Phonology and Morphology of Arabic. Oxford University Press, Oxford (2007)
Darwish, K.: Arabizi detection and conversion to Arabic. In: CoRR (2013)
Boudelaa, S., Marslen-Wilson, W.D.: Aralex: a lexical database for modern standard Arabic. Behav. Res. Methods 42(2), 481–487 (2010)
Mimno, D., Wallach, H.M., Naradowsky, J., Smith, D.A., McCallum, A.: Polylingual topic models. In: Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing (2009)
De Smet, W., Moens, M.-F., Cross-language linking of news stories on the Web using interlingual topic modeling. In: Proceedings of the CIKM 2009 Workshop on Social Web Search and Mining (2009)
De Smet, W., Tang, J., Moens, M.-F.: Knowledge transfer across multilingual corpora via latent topics. In: Proceedings of the PAKDD: the 15th Pacific-Asia Conference on Knowledge Discovery and Data Mining (2011)
Ibrahim, H.S., Abdou, S.M., Gheith, M.: Idioms-proverbs lexicon for modern standard Arabic and colloquial sentiment analysis. Int. J. Comput. Appl. 118, 26–31 (2015)
Malmasi, S., Refaee, E., Dras, M.: Arabic dialect identification using a parallel multidialectal corpus. In: Proceedings of the 14th Conference of the Pacific Association for Computational Linguistics (PACLING 2015), Bali, Indonesia (2015)
Jeblee, S., Feely, W., Bouamor, H., Lavie, A., Habash, N., Oflazer, K.: Domain and dialect adaptation for machine translation into Egyptian Arabic. In ANLP 2014 (2014)
Ibrahim, H.S., Abdou, S.M., Gheith, M.: Sentiment analysis for modern standard arabic and colloquial. Int. J. Nat. Lang. Comput. (IJNLC) 4(2) (2015)
Duwairi, R.M.: Sentiment analysis for dialectical Arabic. In: 6th International Conference on Information and Communication Systems (ICICS), IEEE (2015)
El-Makky, N., Nagi, K., El-Ebshihy, A., Apady, E., Hafez, O., Mostafa, S., Ibrahim, S.: Sentiment analysis of colloquial Arabic tweets. In: ASE BigData/SocialInformatics/PASSAT/BioMedCom 2014 Conference, Harvard University, Dec 2014
Habash, N., Soudi, A., Buckwalter, T.: On Arabic transliteration. Arabic Computational Morphology: Speech and language technology, 38, 15–22. Springer (2007)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Elmarakshy, R., Ismail, M.A. (2016). Compiling a Dialectal Arabic Lexicon Using Latent Topic Models. In: Gaber, T., Hassanien, A., El-Bendary, N., Dey, N. (eds) The 1st International Conference on Advanced Intelligent System and Informatics (AISI2015), November 28-30, 2015, Beni Suef, Egypt. Advances in Intelligent Systems and Computing, vol 407. Springer, Cham. https://doi.org/10.1007/978-3-319-26690-9_44
Download citation
DOI: https://doi.org/10.1007/978-3-319-26690-9_44
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-26688-6
Online ISBN: 978-3-319-26690-9
eBook Packages: Computer ScienceComputer Science (R0)