Skip to main content
SpringerLink
Log in

Automatic translation of Arabic text-to-Arabic sign language

  • Long Paper
  • Published:
Universal Access in the Information Society Aims and scope Submit manuscript

Abstract

Arabic sign language (ArSL) is a full natural language that is used by the deaf in Arab countries to communicate in their community. Unfamiliarity with this language increases the isolation of deaf people from society. This language has a different structure, word order, and lexicon than Arabic. The translation between ArSL and Arabic is a complete machine translation challenge, because the two languages have different structures and grammars. In this work, we propose a rule-based machine translation system to translate Arabic text into ArSL. The proposed system performs a morphological, syntactic, and semantic analysis on an Arabic sentence to translate it into a sentence with the grammar and structure of ArSL. To transcribe ArSL, we propose a gloss system that can be used to represent ArSL. In addition, we develop a parallel corpus in the health domain, which consists of 600 sentences, and will be freely available for researchers. We evaluate our translation system on this corpus and find that our translation system provides an accurate translation for more than 80% of the translated sentences.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. http://www.who.int/mediacentre/factsheets/fs300/en/.

  2. Habash-Soudi-Buckwalter transliteration system [31].

  3. We will explain the bracketing (glossing) system in Sect. 5.

  4. http://www.bu.edu/asllrp/cslgr/.

  5. www.signwriting.org/forums/linguistics/ling006.html.

  6. www.signwriting.org/forums/linguistics/ling007.html.

References

  1. Tolba, M.F., Elons, A.S.: Recent developments in sign language recognition systems. In: Computer Engineering and Systems (ICCES), 2013 8th International Conference on, IEEE, 2013

  2. Sidig, A.A.I., Luqman, H., Mahmoud, S.A.: Transform-based Arabic sign language recognition. Proc. Comput. Sci. 117, 2–9 (2017). https://doi.org/10.1016/j.procs.2017.10.087

    Article  Google Scholar 

  3. Al-Fityani, K., Padden, C.: Sign language geography in the Arab world, sign languages. In: Brentari, D. (ed.) Sign languages: a Cambridge survey. Cambridge University Press, New York, pp. 433–450 (2010)

    Chapter  Google Scholar 

  4. LAS: First part of the Unified Arabic Sign Dictionary. The League of Arab States & the Arab League Educational, Cultural and Scientific Organization, Tunisia (2000) (in Arabic)

    Google Scholar 

  5. Mohandes, M., Deriche, M., Liu, J.: Image-based and sensor-based approaches to Arabic sign language recognition. IEEE Trans. Hum. Mach. Syst. 44(4), 551–557 (2014)

    Article  Google Scholar 

  6. Almohimeed, A., Wald, M., Damper, R.: Arabic text to Arabic sign language translation system for the deaf and hearing-impaired community. In: Proceedings of the Second Workshop on Speech and Language Processing for Assistive Technologies, Association for Computational Linguistics, pp. 101–109 (2011)

  7. Sidig, A.A.I., Luqman, H., Mahmoud, S.A.: Arabic sign language recognition using optical flow-based features and HMM. In: Recent Trends in Information and Communication Technology, pp. 297–305. Springer International Publishing, Cham (2018)

    Google Scholar 

  8. Zhao, L., Kipper, K., Schuler, W., Vogler, C., Badler, N., Palmer, M.: A machine translation system from English to American sign language. In: Conference of the Association for Machine Translation in the Americas, pp. 54–67. Springer, Berlin (2000)

    Chapter  Google Scholar 

  9. Marshall, I., Sáfár, É.: A prototype text to British sign language (BSL) translation system. In: Proceedings of the 41st Annual Meeting on Association for Computational Linguistics, vol. 2, pp. 113–116 (2003)

  10. Morrissey, S., Way, A.: An example-based approach to translating sign language. In: Proceedings of the workshop in example-based machine translation (MT Summit X), Phuket, Thailand, pp. 109–116 (2005)

  11. Cox, S., Lincoln, M., Tryggvason, J., Nakisa, M., Wells, M., Tutt, M., Abbott, S.: The development and evaluation of a speech-to-sign translation system to assist transactions. Int. J. Hum. Comput. Interact. 16(2), 141–161 (2003)

    Article  Google Scholar 

  12. Veale, T., Conway, A., Collins, B.: The challenges of cross-modal translation: English-to-sign-language translation in the zardoz system. Mach. Transl. 13(1), 81–106 (1998)

    Article  Google Scholar 

  13. d’Armond, L.S.: Representation of American sign language for machine translation, Ph.D. thesis, Georgetown University (2002)

  14. Marshall, I., Sáfár, É.: Extraction of semantic representations from syntactic SMU link grammar linkages. In: Proceedings of Recent Advances in Natural Language Processing, pp. 154–159 (2001)

  15. Grieve-Smith, A.B.: English to American sign language machine translation of weather reports. In: Proceedings of the Second High Desert Student Conference in Linguistics (HDSL2), Albuquerque, NM, pp. 23–30 (1999)

  16. Van Zijl, L., Combrink, A.: The South African sign language machine translation project: issues on non-manual sign generation. In: Proceedings of the 2006 annual research conference of the South African institute of computer scientists and information technologists on IT research in developing countries, South African Institute for Computer Scientists and Information Technologists, pp. 127–134 (2006)

  17. San-Segundo, R., Montero, J.M., Macías-Guarasa, J., Córdoba, R., Ferreiros, J., Pardo, J.M.: Proposing a speech to gesture translation architecture for spanish deaf people. J. Vis. Lang. Comput. 19(5), 523–538 (2008)

    Article  Google Scholar 

  18. Huenerfauth, M., Marcus, M., Palmer, M.: Generating American sign language classifier predicates for English-to-ASL machine translation, Ph.D. thesis, University of Pennsylvania (2006)

  19. Othman, A., Jemni, M.: Statistical sign language machine translation: from English written text to American sign language gloss. Int. J. Comput. Sci. Issues 8(5), 65–73 (2011)

    Google Scholar 

  20. Bonham, M.E.: English to ASL Gloss Machine Translation. M. Art thesis, Brigham Young University (2015)

  21. Almasoud, A.M., Al-Khalifa, H.S.: Semsignwriting: a proposed semantic system for Arabic text-to-signwriting translation. J. Softw. Eng. Appl. 5, 604–612 (2012)

    Article  Google Scholar 

  22. El, A.E.E., Basuony, E.M.M.R., Atawy, E.S.M.: Intelligent Arabic text to Arabic sign language translation for easy deaf communication. Int. J. Comput. Appl. 92(8), 22–29 (2014)

    Google Scholar 

  23. Al-Rikabi, S., Hafner, V.: A humanoid robot as a translator from text to sign language. In: 5th Language and Technology Conference: Human Language Technologies as a Challenge for Computer Science and Linguistics (LTC 2011), pp. 375–379 (2011)

  24. Mohandes, M.: Automatic translation of Arabic text to Arabic sign language. AIML J. 6(4), 15–19 (2006)

    Google Scholar 

  25. Halawani, S.M.: Arabic sign language translation system on mobile devices. IJCSNS Int. J. Comput. Sci. Netw. Secur. 8(1), 251–256 (2008)

    Google Scholar 

  26. Al-Khalifa, H.S.: Introducing Arabic sign language for mobile phones. In: International Conference on Computers for Handicapped Persons, Springer, New York, pp. 213–220 (2010)

    Google Scholar 

  27. Al Ameiri, F., Zemerly, M.J., Al Marzouqi, M.: M-learning and chatting using indexed Arabic sign language. Int. J. Infonomics (IJI) 5, 10 (2012)

    Google Scholar 

  28. Al-Nafjan, A., Al-Arifi, B., Al-Wabil, A.: Design and development of an educational Arabic sign language mobile application: collective impact with Tawasol. In: International Conference on Universal Access in Human-Computer Interaction, Springer, New York, pp. 319–326 (2015)

    Chapter  Google Scholar 

  29. Abdel-Fattah, M.A.: Arabic sign language: a perspective. J. Deaf Stud. Deaf Educ. 10(2), 212–221 (2005)

    Article  Google Scholar 

  30. Semreen, S., Albinali, M.: The rules of Arab Qatari sign standardized language. Supreme Council of Family Affairs, Qatar (2010) (in Arabic)

    Google Scholar 

  31. Habash, N., Soudi, A., Buckwalter, T.: On Arabic transliteration. In: Arabic computational morphology, Springer, New York, pp. 15–22 (2007)

  32. Habash, N., Faraj, R., Roth, R.: Syntactic annotation in the columbia Arabic treebank. In: Proceedings of MEDAR International Conference on Arabic Language Resources and Tools, Cairo, Egypt (2009)

  33. Filhol, M., Hadjadj, M.N., Testu, B.: A rule triggering system for automatic text-to-sign translation. Univers. Access Inf. Soc. 15(4), 487–498 (2016)

    Article  Google Scholar 

  34. Morrissey, S.: Data-driven machine translation for sign languages, Ph.D. thesis, Dublin City University (2008)

  35. Almohimeed, A., Wald, M., Damper, R.: An Arabic sign language corpus for instructional language in school. In: LREC 2010: 4th Workshop on the Representation and Processing of Sign Languages: Corpora and Sign Language Technologies, pp. 81–82 (2010)

  36. Assaleh, K., Shanableh, T., Fanaswala, M., Amin, F., Bajaj, H.: Continuous Arabic sign language recognition in user dependent mode. J. Intell. Learn. Syst. Appl. 2(01), 19–27 (2010)

    Google Scholar 

  37. Elhadj, Y.O.M., Zemirli, Z., Ayyadi, K.: Development of a bilingual parallel corpus of Arabic and Saudi sign language: Part i. In: Intelligent Informatics, Springer, New York, pp. 285–295 (2013)

  38. Zahedi, M., Dreuw, P., Rybach, D., Deselaers, T., Ney, H.: Continuous sign language recognition-approaches from speech recognition and available data resources. In: Second Workshop on the Representation and Processing of Sign Languages: Lexicographic Matters and Didactic Scenarios, pp. 21–24 (2006)

  39. Dreuw, P., Forster, J., Deselaers, T., Ney, H.: Efficient approximations to model-based joint tracking and recognition of continuous sign language. In: Automatic Face and Gesture Recognition, 2008. FG’08. 8th IEEE International Conference on, IEEE, pp. 1–6 (2008)

  40. Dreuw, P., Neidle, C., Athitsos, V., Sclaroff, S., Ney, H.: Benchmark databases for video-based automatic sign language recognition. In: Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC’08) (2008)

  41. Athitsos, V., Neidle, C., Sclaroff, S., Nash, J., Stefan, A., Thangali, A., Wang, H., Yuan, Q.: Large lexicon project: American sign language video corpus and sign language indexing/retrieval algorithms. In: Workshop on the Representation and Processing of Sign Languages: Corpora and Sign Language Technologies (CSLT), pp. 11–14 (2010)

  42. Schembri, A., Fenlon, J., Rentelis, R., Cormier, K.: British sign language corpus project: a corpus of digital video data of British sign language 2008–2010, 1st edn. University College London. http://www.bslcorpusproject.org (2011). Accessed 13 June 2018

  43. Crasborn, O.A., Zwitserlood, I.: The corpus NGT: an online corpus for professionals and laymen. In: 3rd Workshop on the Representation and Processing of Sign Languages (LREC), ELDA, pp. 44–49 (2008)

  44. Leeson, L., Saeed, J., Byrne-Dunne, D., Macduff, A., Leonard, C.: Moving heads and moving hands: developing a digital corpus of Irish sign language. The ’signs of Ireland’ corpus development project. In: IT&T conference, Carlow (2006)

  45. Chiu, Y.-H., Wu, C.-H., Su, H.-Y., Cheng, C.-J.: Joint optimization of word alignment and epenthesis generation for Chinese to Taiwanese sign synthesis. IEEE Trans. Pattern Anal. Mach. Intell. 29(1), 28–39 (2007)

    Article  Google Scholar 

  46. Bungeroth, J., Stein, D., Dreuw, P., Zahedi, M., Ney, H.: A german sign language corpus of the domain weather report. In: Fifth International Conference on Language Resources and Evaluation, pp. 2000–2003 (2006)

  47. Stokoe, W.C.: Sign language structure: an outline of the visual communication systems of the American deaf. J. Deaf Stud. Deaf Educ. 10(1), 3–37 (2005)

    Article  Google Scholar 

  48. Bentele, S.: About the HamNoSys system. http://www.signwriting.org/forums/ linguistics/ling007.html. Accessed Apr 2017

  49. Aouiti, N., Jemni, M., Semreen, S.: Arab gloss annotation system for Arabic sign language. In: Information and Communication Technology and Accessibility (ICTA), 2015 5th International Conference on, IEEE, pp. 1–6 (2015)

  50. Liddell, S.K.: Grammar, gesture, and meaning in American Sign Language. Cambridge University Press, Cambridge (2003)

    Book  Google Scholar 

  51. Habash, N.Y.: Introduction to Arabic natural language processing. Synth. Lect. Hum. Lang. Technol. 3(1), 1–187 (2010)

    Article  Google Scholar 

  52. Pasha, A., Al-Badrashiny, M., Diab, M.T., El Kholy, A., Eskander, R., Habash, N., Pooleery, M., Rambow, O., Roth, R.: Madamira: a fast, comprehensive tool for morphological analysis and disambiguation of arabic., In: LREC, vol. 14, pp. 1094–1101 (2014)

  53. Shahrour, A., Khalifa, S., Taji, D., Habash, N.: Camelparser: a system for Arabic syntactic analysis and morphological disambiguation. In: Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: System Demonstrations, pp. 228–232 (2016)

  54. Nivre, J., Hall, J., Nilsson, J., Chanev, A., Eryigit, G., Kübler, S., Marinov, S., Marsi, E.: Maltparser: a language-independent system for data-driven dependency parsing. Nat Lang Eng 13(02), 95–135 (2007)

    Article  Google Scholar 

  55. Elkateb, S., Black, W., Rodríguez, H., Alkhalifa, M., Vossen, P., Pease, A., Fellbaum, C.: Building a wordnet for Arabic. In: Proceedings of The fifth international conference on Language Resources and Evaluation (LREC 2006) (2006)

  56. Heafield, K.: Kenlm: faster and smaller language model queries. In: Proceedings of the Sixth Workshop on Statistical Machine Translation, Association for Computational Linguistics, pp. 187–197 (2011)

  57. Al-Jefri, M.: Real-word error detection and correction in Arabic text, Master’s thesis, King Fahd University of Petroleum and Minerals (2013)

  58. Stolcke, A.: SRILM-an extensible language modeling toolkit. In: Proceedings International Conference on Spoken Language Processing, pp. 257–286 (2002)

  59. Federico, M., Bertoldi, N., Cettolo, M.: IRSTLM: an open source toolkit for handling large scale language models. In: Interspeech, pp. 1618–1621 (2008)

  60. Arabic sign language dictionary. http://www.menasy.com/. Accessed Apr 2017

  61. Marton, Y., Habash, N., Rambow, O.: Dependency parsing of Modern Standard Arabic with lexical and inflectional features. Comput. Linguist. 39(1), 161–194 (2013)

    Article  Google Scholar 

  62. Hadla, L.S., Hailat, T.M., Al-Kabi, M.N.: Evaluating Arabic to english machine translation. Int. J. Adv. Comput. Sci. Appl. (IJACSA) 5(11), 68–73 (2014)

    Google Scholar 

  63. Gonàlez, M., Giménez, J., Màrquez, L.: A Graphical interface for MT evaluation and error analysis. In: The 50th Annual Meeting of the Association for Computational Linguistics (2012)

  64. Papineni, K., Roukos, S., Ward, T., Zhu, W.-J.: Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th annual meeting on association for computational linguistics, Association for Computational Linguistics, pp. 311–318 (2002)

  65. Nießen, S., Och, F.J., Leusch, G., Ney, H., Informatik, L.F.: An evaluation tool for machine translation: fast evaluation for MT research. In: In Proceedings of the 2nd International Conference on Language Resources and Evaluation (LREC-2000) (2000)

  66. Snover, M., Dorr, B., Schwartz, R., Micciulla, L., Makhoul, J.: A study of translation edit rate with targeted human annotation. In: Proceedings of association for machine translation in the Americas, vol. 200 (2006)

Download references

Acknowledgements

The authors would like to thank the referees for their constructive comments. We thank Dr. Nizar Habash for his helpful conversations, resources and feedback. We also like to thank Manal Al-Ashwal, Dr. Sameer Semreen, and Ayman Al-Qadsi who actively participated in this work. In addition, we would like to acknowledge the support provided by King Fahd University of Petroleum & Minerals (KFUPM) for funding this work through project number IN151008.

Author information

Authors and Affiliations

  1. King Fahd University of Petroleum and Minerals, Dhahran, 31261, Saudi Arabia

    Hamzah Luqman & Sabri A. Mahmoud

Authors
  1. Hamzah Luqman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sabri A. Mahmoud
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hamzah Luqman.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Luqman, H., Mahmoud, S.A. Automatic translation of Arabic text-to-Arabic sign language. Univ Access Inf Soc 18, 939–951 (2019). https://doi.org/10.1007/s10209-018-0622-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10209-018-0622-8

Keywords

Search

Navigation