Skip to main content
SpringerLink
Log in

Sign Language Recognition for Low Resource Languages Using Few Shot Learning

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2023)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1964))

Included in the following conference series:

  • 349 Accesses

Abstract

Sign Language Recognition (SLR) with machine learning is challenging due to the scarcity of data for most low-resource sign languages. Therefore, it is crucial to leverage a few-shot learning strategy for SLR. This research proposes a novel skeleton-based sign language recognition method based on the prototypical network [20] called ProtoSign. Furthermore, we contribute to the field by introducing the first publicly accessible dynamic word-level Sinhala Sign Language (SSL) video dataset comprising 1110 videos over 50 classes. To our knowledge, this is the first publicly available SSL dataset. Our method is evaluated using two low-resource language datasets, including our dataset. The experiments show the results in 95% confidence level for both 5-way and 10-way in 1-shot, 2-shot, and 5-shot settings.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Deafness and hearing loss (2023). https://www.who.int/news-room/fact-sheets/detail/deafness-and-hearing-loss. Accessed 29 May 2023

  2. Amin, M.S., Rizvi, S.T.H., Hossain, M.M.: A comparative review on applications of different sensors for sign language recognition. J. Imaging 8(4), 98 (2022)

    Article  Google Scholar 

  3. Bochkovskiy, A., Wang, C.Y., Liao, H.Y.M.: Yolov4: optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934 (2020)

  4. Boháček, M., Hrúz, M.: Sign pose-based transformer for word-level sign language recognition. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 182–191 (2022)

    Google Scholar 

  5. Camgoz, N.C., Koller, O., Hadfield, S., Bowden, R.: Multi-channel transformers for multi-articulatory sign language translation. In: Bartoli, A., Fusiello, A. (eds.) ECCV 2020. LNCS, vol. 12538, pp. 301–319. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-66823-5_18

    Chapter  Google Scholar 

  6. Cui, R., Liu, H., Zhang, C.: Recurrent convolutional neural networks for continuous sign language recognition by staged optimization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7361–7369 (2017)

    Google Scholar 

  7. Izutov, E.: ASL recognition with metric-learning based lightweight network. arXiv preprint arXiv:2004.05054 (2020)

  8. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: andYann LeCun, Y.B. (ed.) Proceedings of the 3rd International Conference on Learning Representations, ICLR (2015)

    Google Scholar 

  9. Koller, O., Zargaran, O., Ney, H., Bowden, R.: Deep sign: hybrid CNN-hmm for continuous sign language recognition. In: Proceedings of the British Machine Vision Conference 2016 (2016)

    Google Scholar 

  10. Lee, B.G., Lee, S.M.: Smart wearable hand device for sign language interpretation system with sensors fusion. IEEE Sens. J. 18(3), 1224–1232 (2017)

    Article  Google Scholar 

  11. Lugaresi, C., et al.: MediaPipe: a framework for building perception pipelines. arXiv preprint arXiv:1906.08172 (2019)

  12. Paszke, A., et al.: Pytorch: an imperative style, high-performance deep learning library. In: Wallach, H., Larochelle, H., Beygelzimer, A., d’ Alché-Buc, F., Fox, E., Garnett, R. (eds.) Proceedings of the Advances in Neural Information Processing Systems, pp. 8024–8035 (2019)

    Google Scholar 

  13. Rao, G.A., Syamala, K., Kishore, P., Sastry, A.: Deep convolutional neural networks for sign language recognition. In: 2018 Conference on Signal Processing and Communication Engineering Systems (SPACES), pp. 194–197. IEEE (2018)

    Google Scholar 

  14. Rastgoo, R., Kiani, K., Escalera, S.: Hand sign language recognition using multi-view hand skeleton. Expert Syst. Appl. 150, 113336 (2020)

    Article  Google Scholar 

  15. Ronchetti, F., Quiroga, F., Estrebou, C.A., Lanzarini, L.C., Rosete, A.: Lsa64: an argentinian sign language dataset. In: XXII Congreso Argentino de Ciencias de la Computación (CACIC 2016). (2016)

    Google Scholar 

  16. Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D., Lillicrap, T.: Meta-learning with memory-augmented neural networks. In: International Conference on Machine Learning, pp. 1842–1850. PMLR (2016)

    Google Scholar 

  17. Saunders, B., Camgoz, N.C., Bowden, R.: Continuous 3D multi-channel sign language production via progressive transformers and mixture density networks. Int. J. Comput. Vision 129(7), 2113–2135 (2021)

    Article  Google Scholar 

  18. Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 815–823 (2015)

    Google Scholar 

  19. Si, J., et al.: Dual attention matching network for context-aware feature sequence based person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5363–5372 (2018)

    Google Scholar 

  20. Snell, J., Swersky, K., Zemel, R.: Prototypical networks for few-shot learning. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  21. Theodorakis, S., Pitsikalis, V., Maragos, P.: Dynamic-static unsupervised sequentiality, statistical subunits and lexicon for sign language recognition. Image Vis. Comput. 32(8), 533–549 (2014)

    Article  Google Scholar 

  22. Tunga, A., Nuthalapati, S.V., Wachs, J.: Pose-based sign language recognition using GCN and BERT. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 31–40 (2021)

    Google Scholar 

  23. Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  24. Vinyals, O., Blundell, C., Lillicrap, T., Wierstra, D., et al.: Matching networks for one shot learning. In: Advances in Neural Information Processing Systems, vol. 29 (2016)

    Google Scholar 

  25. Wang, Y., Yao, Q., Kwok, J.T., Ni, L.M.: Generalizing from a few examples: a survey on few-shot learning. ACM Comput. Surv. 53(3), 1–34 (2020)

    Article  Google Scholar 

  26. Weerasooriya, A.A., Ambegoda, T.D.: Sinhala fingerspelling sign language recognition with computer vision. In: 2022 Moratuwa Engineering Research Conference (MERCon), pp. 1–6. IEEE (2022)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, University of Moratuwa, Moratuwa, Sri Lanka

    Kaveesh Charuka, Sandareka Wickramanayake, Thanuja D. Ambegoda, Pasan Madhushan & Dineth Wijesooriya

Authors
  1. Kaveesh Charuka
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sandareka Wickramanayake
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thanuja D. Ambegoda
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Pasan Madhushan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dineth Wijesooriya
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sandareka Wickramanayake .

Editor information

Editors and Affiliations

  1. School of Automation, Central South University, Changsha, China

    Biao Luo

  2. Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Long Cheng

  3. Institute of Cyber-Systems and Control, Zhejiang University, Hangzhou, China

    Zheng-Guang Wu

  4. School of Automation, Guangdong University of Technology, Guangzhou, China

    Hongyi Li

  5. School of Electrical Engineering and Telecommunications, UNSW Sydney, Sydney, NSW, Australia

    Chaojie Li

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Charuka, K., Wickramanayake, S., Ambegoda, T.D., Madhushan, P., Wijesooriya, D. (2024). Sign Language Recognition for Low Resource Languages Using Few Shot Learning. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Communications in Computer and Information Science, vol 1964. Springer, Singapore. https://doi.org/10.1007/978-981-99-8141-0_16

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-8141-0_16

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-8140-3

  • Online ISBN: 978-981-99-8141-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation