Skip to main content
SpringerLink
Log in

A machine translation system from Arabic sign language to Arabic

  • Long Paper
  • Published:
Universal Access in the Information Society Aims and scope Submit manuscript

Abstract

Arabic sign language (ArSL) is one of the sign languages that is used in Arab countries. This language has structure and grammar that differ from spoken Arabic. Available ArSL recognition systems perform direct mapping between the recognized sign in the ArSL sentence and its corresponding Arabic word. This results in persevering the structure and grammar of the ArSL sentence. ArSL translation involves converting the recognized ArSL sentence into Arabic sentence that meets the structure and grammar of Arabic. We propose in this work a rule-based machine translation system between ArSL and Arabic. The proposed system performs morphological and syntactic analysis to translate the ArSL sentence lexically and syntactically into Arabic. To evaluate this work, we perform manual and automatic evaluation using a corpus on the health domain. The obtained results show that our translation system provides an accurate translation for more than 80% of the translated sentences.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. http://www.who.int/pbd/deafness/news/Millionslivewithhearingloss.pdf?ua=1.

  2. We will use the gloss annotation system proposed by [4] to represent ArSL sign words. This system encloses each sign word between two brackets.

  3. http://www.almaany.com.

  4. Each sign repetition is represented in ArSL gloss notation by ’+’ symbol

References

  1. Sidig, A.A.I., Luqman, H., Mahmoud, S.A.: Transform-based Arabic sign language recognition. Proc. Comput. Sci. 117, 2–9 (2017). https://doi.org/10.1016/j.procs.2017.10.087

    Article  Google Scholar 

  2. Arab League Educational Cultural, Scientific Organization, First part of the Unified Arabic Sign language Dictionary, League Arab States Arab League Educ. Cult. Sci. Organ, (2001)

  3. Arab League Educational Cultural, Scientific Organization, Second part of the Unified Arabic Sign language Dictionary, League Arab States Arab League Educ. Cult. Sci. Organ, (2007)

  4. Luqman, H., Mahmoud, S.A.: Automatic translation of Arabic text-to-Arabic sign language. Universal Access in the Information Society 1–13 (2018)

  5. Mitra, S., Acharya, T.: Gesture recognition: a survey. IEEE Trans Syst Man Cybern Part C (Appl Rev) 37(3), 311–324 (2007)

    Article  Google Scholar 

  6. Cooper, H., Holt, B., Bowden, R.: Sign language recognition. In: Visual Analysis of Humans, pp. 539–562, Springer, Berlin (2011)

  7. Abdel-Fattah, M.A.: Arabic sign language: a perspective. J. Deaf Stud. Deaf Educ. 10(2), 212–221 (2005)

    Article  Google Scholar 

  8. Semreen, S., Albinali, M.: The rules of Arab Qatari sign standardized language, Supreme Council of Family Affairs, (2010)

  9. Abdel-Fattah, M.: Arabic sign language: a perspective. J. Deaf Stud. Deaf Educ. 10(2), 212–221 (2005). https://doi.org/10.1093/deafed/eni007

    Article  Google Scholar 

  10. Trujillo, A.: Translation Engines: Techniques for Machine Translation. Springer, Berlin (2012)

    MATH  Google Scholar 

  11. Morrissey, S.: Data-driven machine translation for sign languages, Ph.D. thesis, Dublin City University (2008)

  12. Sidig, A.A.I., Luqman, H., Mahmoud, S.A.: Arabic Sign Language Recognition Using Optical Flow-Based Features and HMM, pp. 297–305, Springer, Cham (2018). https://doi.org/10.1007/978-3-319-59427-9-32

  13. Almohimeed, A., Wald, M., Damper, R.: Arabic text to Arabic sign language translation system for the deaf and hearing-impaired community. In: Proceedings of the Second Workshop on Speech and Language Processing for Assistive Technologies, Association for Computational Linguistics, pp. 101–109 (2011)

  14. Mohandes, M.: Automatic translation of Arabic text to Arabic sign language. AIML J. 6(4), 15–19 (2006)

    Google Scholar 

  15. Halawani, S.M.: Arabic sign language translation system on mobile devices. IJCSNS Int. J. Comput. Sci. Netw. Secur. 8(1), 251–256 (2008)

    Google Scholar 

  16. Al Ameiri, F., Zemerly, M. J., Al Marzouqi, M.: M-learning and chatting using indexed Arabic sign language, Int. J. Inf. (IJI) 5: 10 (2012)

  17. Al-Nafjan, A., Al-Arifi, B., Al-Wabil, A.: Design and development of an educational Arabic sign language mobile application: collective impact with Tawasol. In: International Conference on Universal Access in Human–Computer Interaction, pp. 319–326, Springer, Berlin (2015)

  18. Al-Khalifa, H. S.: Introducing Arabic sign language for mobile phones. In: International Conference on Computers for Handicapped Persons, pp. 213–220, Springer, Berlin (2010)

  19. Al-Rikabi, S., Hafner, AV.: Humanoid robot as a translator from text to sign language. In: 5th Language and Technology Conference: Human Language Technologies as a Challenge for Computer Science and Linguistics (LTC 2011), pp. 375–379 (2011)

  20. Almasoud, A.M., Al-Khalifa, H.S.: Semsignwriting: a proposed semantic system for Arabic text-to-signwriting translation. J. Softw. Eng. Appl. 5, 604–612 (2012)

    Article  Google Scholar 

  21. El Alfi, A.E.E., El Basuony, M.M.R., Atawy, S.M.: Intelligent Arabic text to Arabic sign language translation for easy deaf communication. Int. J. Comput. Appl. 92(8), 22–29 (2014)

    Google Scholar 

  22. Ritchings, T., Khadragi, A., Saeb, M.: An intelligent computer-based system for sign language tutoring. Assist. Technol. 24(4), 299–308 (2012). https://doi.org/10.1080/10400435.2012.680662

    Article  Google Scholar 

  23. Mohandes, M., Deriche, M.: Arabic sign language recognition by decisions fusion using Dempster–Shafer theory of evidence, 2013 Computing, Commun. IT Appl. Conf. (ComComAp) 2013, 90–94 (2013). https://doi.org/10.1109/ComComAp.2013.6533615

    Article  Google Scholar 

  24. Mohandes, M., Quadri, S., Deriche, M.: Arabic sign language recognition an image-based approach. In: 21st International Conference on Advanced Information Networking and Applications Workshops (AINAW’07), pp. 272–276 (2007) https://doi.org/10.1109/AINAW.2007.98

  25. Shanableh, T., Assaleh, K., Al-Rousan, M.: Spatio-temporal feature-extraction techniques for isolated gesture recognition in Arabic sign language. IEEE Trans. Syst. Man Cybern. Part B Cybern. Publ. IEEE Syst. Man Cybern. Soc. 37(3), 641–650 (2007)

    Article  Google Scholar 

  26. AL-Rousan, M., Assaleh, K., Tala’a, a: Video-based signer-independent Arabic sign language recognition using hidden Markov models. Appl. Soft Comput. 9(3), 990–999 (2009). https://doi.org/10.1016/j.asoc.2009.01.002

    Article  Google Scholar 

  27. Ahmed, A., Aly, S.: Appearance-based Arabic sign language recognition using hidden Markov models. In: 2014 International Conference on Engineering and Technology (ICET), 2014, pp. 1–6. https://doi.org/10.1109/ICEngTechnol.2014.7016804

  28. Aly, S., Mohammed, S.: Arabic sign language recognition using Spatio-Temporal Local Binary Patterns and Support Vector Machine. In: Hassanien, A., Tolba, M., Taher Azar, A. (eds.) Advanced Machine Learning Technologies and Applications, Vol. 488 of Communications in Computer and Information Science, Springer, Berlin, 2014, pp. 36–45. https://doi.org/10.1007/978-3-319-13461-1-5

  29. Tharwat, A., Gaber, T., Hassanien, A. E., Shahin, M., Refaat, B.: Sift-based Arabic sign language recognition system. In: Afro-European Conference for Industrial Advancement, pp. 359–370, Springer, Berlin (2015)

  30. Aujeszky, T., Eid, M.: A gesture recognition architecture for Arabic sign language communication system. Multimed. Tools Appl. 75(14), 8493–8511 (2016)

    Article  Google Scholar 

  31. Aly, S., Osman, B., Aly, W., Saber, M.: Arabic sign language fingerspelling recognition from depth and intensity images. In: 2016 12th International Computer Engineering Conference (ICENCO), pp. 99–104, IEEE (2016)

  32. Cooper, H., Ong, E.-J., Pugeault, N., Bowden, R.: Sign language recognition using sub-units. J. Mach. Learn. Res. 13, 2205–2231 (2012)

    Google Scholar 

  33. Paulraj, M. P., Yaacob, S., Zanar Azalan, M. S., Palaniappan, R.: A phoneme based sign language recognition system using 2D moment invariant interleaving feature and neural network. In: Proceedings—2011 IEEE Student Conference on Research and Development, SCOReD 2011, pp. 111–116 (2011) https://doi.org/10.1109/SCOReD.2011.6148718

  34. Elons, A.S., Abull-ela, M., Tolba, M.: A proposed PCNN features quality optimization technique for pose-invariant 3D Arabic sign language recognition. Appl. Soft Comput. 13(4), 1646–1660 (2013). https://doi.org/10.1016/j.asoc.2012.11.036

    Article  Google Scholar 

  35. Tolba, M., Samir, A., Abull-ela, M.: 3D Arabic sign language recognition using liner combination of Multible 2D views. In: 2012 8th International Conference on Informatics and Systems (INFOS), IEEE, Cairo (2012)

  36. Sutherland, A., Awad, G., Han, J.: Boosted subunits: a framework for recognising sign language from videos, IET Image Processing 7 (September 2012) (2013) 70–80. https://doi.org/10.1049/iet-ipr.2012.0273

  37. Awad, G., Han, J., Sutherland, A.: A unified system for segmentation and tracking of face and hands in sign language recognition. In: Proceedings—International Conference on Pattern Recognition, vol. 1, pp. 239–242 (2006). https://doi.org/10.1109/ICPR.2006.194

  38. Pasha, A., Al-Badrashiny, M., Diab, M. T., El Kholy, A., Eskander, R., Habash, N., Pooleery, M.,  Rambow, O., Roth, R.: MADAMIRA: a fast, comprehensive tool for morphological analysis and disambiguation of Arabic. In: LREC, Vol. 14, 2014, pp. 1094–1101

  39. Heafield, K., KenLM: Faster and smaller language model queries. In: Proceedings of the Sixth Workshop on Statistical Machine Translation, Association for Computational Linguistics, pp. 187–197 (2011)

  40. Stolcke, A.: SRILM-an extensible language modeling toolkit. In: Proceedings International Conference on Spoken Language Processing, pp. 257–286 (2002)

  41. Federico, M., Bertoldi, N., Cettolo, M.: IRSTLM: an open source toolkit for handling large scale language models. In: Interspeech, pp. 1618–1621 (2008)

  42. Neme, A.A., Laporte, E.: Pattern-and-root inflectional morphology: the arabic broken plural. Lang. Sci. 40, 221–250 (2013)

    Article  Google Scholar 

  43. Attia, M., Pecina, P., Tounsi, L., Toral, A., Van Genabith, J.: Lexical profiling for Arabic. In: Proceedings of eLex, pp. 23–33 (2011)

  44. Elghamry, K.: A lexical-syntactic solution to the problem of broken plural in Arabic. In: G. U. R. T. G. 2010) (Ed.), Arabic Natural Language Processing Track, Washington, USA, (2010)

  45. Yaqoub, E. B.: The detailed dictionary in the plurals. (In Arabic), Scientific Book House, (2004)

  46. Chennoufi, A., Mazroui, A.: Morphological, syntactic and diacritics rules for automatic diacritization of Arabic sentences. J. King Saud Univ. Comput. Inf. Sci. 29(2), 156–163 (2017)

    Google Scholar 

  47. Gonàlez, M., Giménez, J., Màrquez, L.: A Graphical interface for MT evaluation and error analysis. In: The 50th Annual Meeting of the Association for Computational Linguistics (2012)

  48. Papineni, K., Roukos, S., Ward, T., Zhu, W.-J.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, Association for Computational Linguistics, pp. 311–318 (2002)

  49. Snover, M., Dorr, B., Schwartz, R., Micciulla, L., Makhoul, J.: A study of translation edit rate with targeted human annotation. In: Proceedings of Association for Machine Translation in the Americas, Vol. 200, (2006)

Download references

Acknowledgements

The authors would like to thank Dr. Nizar Habash for his helpful conversations, resources, and feedback. In addition, we would like to acknowledge the support provided by King Fahd University of Petroleum & Minerals (KFUPM) for funding this work through Project Number IN151008.

Author information

Authors and Affiliations

  1. King Fahd University of Petroleum and Minerals, Dhahran, 31261, Saudi Arabia

    Hamzah Luqman & Sabri A. Mahmoud

Authors
  1. Hamzah Luqman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sabri A. Mahmoud
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hamzah Luqman.

Ethics declarations

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Luqman, H., Mahmoud, S.A. A machine translation system from Arabic sign language to Arabic. Univ Access Inf Soc 19, 891–904 (2020). https://doi.org/10.1007/s10209-019-00695-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10209-019-00695-6

Keywords

Search

Navigation