Skip to main content
SpringerLink
Log in

Handling of Simultaneous Morphology of Sign Languages: Concerns for Cross-modal Machine Translation of Marathi to Indian Sign Language

  • Original Research
  • Published:
SN Computer Science Aims and scope Submit manuscript

Abstract

Effective communication can be challenging when individuals are unfamiliar with each other's spoken language. This problem becomes more complex when the languages involved differ in modality such as spoken language and sign language. Machine translation has emerged as a valuable tool for bridging this communication gap in various scenarios. This research addresses the unique challenges presented by cross-modal machine translation systems, specifically in translating from spoken language to sign language. While spoken languages typically exhibit linear linguistic characteristics, sign languages employ simultaneous expressions through manual, non-manual, and spatial features. This divergence in delivery mechanisms necessitates careful consideration in the development of a translation system. The primary objective of this research is to develop a framework that translates simple Marathi sentences into Indian sign language, incorporating computational techniques to account for the simultaneous nature of sign language grammar. A comprehensive linguistic analysis is conducted to identify the specific divergences between Marathi and Indian sign language. The system is evaluated using a database comprising vocabulary relevant to daily communication. This paper provides a detailed account of the system architecture, experimental setup, results, and evaluation. The development of a cross-modal translation system holds significant potential in bridging the communication gap between individuals using spoken languages and those who rely on sign languages as their primary means of communication. The system-generated ISL can facilitate communication for hearing-impaired persons and, thus, it contributes to the inclusivity of hearing-impaired communities in the mainstream.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Bhagwat, S.R., Bhavsar, R.P., Pawar, B.V.: Translation from simple Marathi sentences to Indian sign language using the phrase-based approach. In: 2021 International Conference on Emerging Smart Computing and Informatics (ESCI), pp. 367–373 (2021). https://doi.org/10.1109/ESCI50559.2021.9396900

  2. Pandharipande RV. Marathi. Descriptive grammar series. London: Routledge; 1997.

    Google Scholar 

  3. Damle MK. Shastriya Marathi Vyakaran. Damodar Savalaram ani Mandali, (1965). https://books.google.co.in/books?id=5i7UoQEACAAJ

  4. Wali K, of Language Studies II. Marathi: A Study in Comparative South Asian Languages. Indian Institute of Language Studies, (2006). https://books.google.co.in/books?id=IK9jAAAAMAAJ

  5. Vasishta M, Woodward JWK. Sign language in India: regional variation within the deaf population. Indian J Appl Linguist. 1978;2:66–74.

    Google Scholar 

  6. Zeshan U, et al. Sign language in Indo-Pakistan. A description of a signed language. Amsterdam: Benjamins; 2000.

    Book  Google Scholar 

  7. Sinha S. A grammar of Indian sign language. Technical report (2009)

  8. Veale T, Conway A. Cross-modal comprehension in Zardoz an English to the sign-language translation system. In: Proceedings of the Seventh International Workshop on Natural Language Generation. Proceedings of the Seventh International Workshop on Natural Language Generation. pp. 249–252 (1994)

  9. Zhao L, Kipper K, Schuler W, Vogler C, Badler N, Palmer M. A machine translation system from English to American sign language. In: Conference of the Association for Machine Translation in the Americas. Conference of the Association for Machine Translation in the Americas. pp. 54–67 (2000)

  10. Hanke T. Hamnosys—representing sign language data in language resources and language processing contexts. (2004)

  11. Schulmeister R. The visicast project: translation into sign language and generation of sign language. In: HCI. HCI. pp. 431–435 (2001)

  12. Morrissey S. Data-driven machine translation for sign languages (2008)

  13. Kar P, Reddy M, Mukherjee A, Raina AM. Ingit: limited domain formulaic translation from Hindi strings to Indian sign language. ICON. 2007;52:53–4.

    Google Scholar 

  14. Dasgupta T, Basu A. Prototype machine translation system from text to Indian sign language. In: Proceedings of the 13th International Conference on Intelligent User Interfaces. Proceedings of the 13th international conference on Intelligent user interfaces. pp. 313–316 (2008)

  15. Sugandhi, Kumar P, Kaur S. Sign language generation system based on Indian sign language grammar. ACM Trans Asian Low-Resource Lang Inf Process. 2020. https://doi.org/10.1145/3384202.

    Article  Google Scholar 

  16. Verma A, Kaur S. Indian sign language animation generation system for Gurumukhi script. International Journal of Computer Science and Technology [IJCST] 6 (2015)

  17. Vij P, Bhatia P: Translator of Hindi text to ISL and extension of ISL dictionary with wordnet (2016)

  18. Supalla S, Cripps J, Byrne A. Why American sign language gloss must matter. Am Ann Deaf. 2017;161:540–51. https://doi.org/10.1353/aad.2017.0004.

    Article  Google Scholar 

  19. https://vh.cmp.uea.ac.uk/index.php/JASigning_Demos#JASigning.

Download references

Acknowledgements

We are deeply thankful to Vallari Singh (Government Authorized ISL Interpreter), Sruti Awale (Government Authorized ISL Interpreter), and special education teachers from Ayodhya Charitable Trust’s, Deaf and Dump School, Wanworie, Pune, for their valuable inputs during the development of this system. They have also given valuable contributions to evaluation.

Author information

Author notes

  1. Suvarna R. Bhagwat, R. P. Bhavsar, and B. V. Pawar have equally contributed to this work.

Authors and Affiliations

  1. Department of Artificial Intelligence and Data Science, Vishwakarma Institute of Information Technology, Pune, Maharashtra, 411037, India

    Suvarna R. Bhagwat

  2. School of Computer Sciences, KBC North Maharashtra University, Umavi Nagar, Jalgaon, Maharashtra, 425001, India

    R. P. Bhavsar & B. V. Pawar

Authors
  1. Suvarna R. Bhagwat
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. R. P. Bhavsar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. B. V. Pawar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Suvarna R. Bhagwat, R. P. Bhavsar or B. V. Pawar.

Ethics declarations

Conflict of Interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Ethical Approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bhagwat, S.R., Bhavsar, R.P. & Pawar, B.V. Handling of Simultaneous Morphology of Sign Languages: Concerns for Cross-modal Machine Translation of Marathi to Indian Sign Language. SN COMPUT. SCI. 4, 629 (2023). https://doi.org/10.1007/s42979-023-02128-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s42979-023-02128-x

Keywords

Search

Navigation