Skip to main content
SpringerLink
Log in

A Serious Game for Learning Portuguese Sign Language - “iLearnPSL”

  • Conference paper
  • First Online:
Online Engineering & Internet of Things

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 22))

Abstract

Several thousands of people in Portugal use Portuguese sign language (PSL). Children belonging to this community have difficulties with communication and learning processes. There are some applications that focuses on teaching/learning PSL but with a limited interaction with the user, such as presentation of images of a hand gesture picture or avatar where the user must mimic the respective PSL hand gesture. Some other applications are more advanced using sensors to detect movements and gestures performed by the user. However, these applications do not implement an automatic interaction with the user of PSL. Following this trend, the main idea of this project is to develop a solution where deaf child can learn PSL in an interactive, dynamic and funny way. Therefore, this paper describes the first insights in the development of an interactive and didactic virtual game tool to learn the Portuguese Sign Language, in particular the numbers from 0 to 9. The alphabet and colors will be also included in the future. The target group considers deaf children in the first cycle of education level (primary Portuguese school). The developed game promotes the automatic interaction with the user through the Leap Motion controller. The system captures the hands and fingers gestures performed by the user and the graphical interface returns the adequate feedback. The preliminary results obtained show a good level of user experience with therapists.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 259.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 329.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Carvalho, C.A.F.: A narrativa em crianças surdas: papel da Língua Gestual Portuguesa (2013). (in Portuguese)

    Google Scholar 

  2. Informação - Comunidade. http://www.apsurdos.org.pt/index.php?option=com_content&view=article&id=43&Itemid=57. Accessed 7 July 2015. (in Portuguese)

  3. Weichert, F., Bachmann, D., Rudak, B., Fisseler, D.: Analysis of the accuracy and robustness of the leap motion controller. Sensors (Basel) 13(5), 6380–6393 (2013)

    Article  Google Scholar 

  4. Potter, L.E., Araullo, J., Carter, L.: The leap motion controller. In: Proceedings of the 25th Australian Computer-Human Interaction Conference on Augmentation, Application, Innovation, Collaboration, OzCHI 2013, pp. 175–178 (2013)

    Google Scholar 

  5. Guna, J., Jakus, G., Pogačnik, M., Tomažič, S., Sodnik, J.: An analysis of the precision and reliability of the leap motion sensor and its suitability for static and dynamic tracking. Sensors 14(2), 3702–3720 (2014)

    Article  Google Scholar 

  6. API Overview—Leap Motion C# SDK v2.3 documentation. https://developer.leapmotion.com/documentation/csharp/devguide/Leap_Overview.html#hands. Accessed 4 Apr 2016

  7. Sun, C., Zhang, T., Xu, C.: Latent support vector machine modeling for sign language recognition with kinect. ACM Trans. Intell. Syst. Technol. 6(2), 1–20 (2015)

    Article  Google Scholar 

  8. Zafrulla, Z., Brashear, H., Starner, T., Hamilton, H., Presti, P.: American sign language recognition with the kinect. In: Proceedings of the 13th International Conference on Multimodal Interfaces - ICMI 2011, p. 279 (2011)

    Google Scholar 

  9. Funasaka, M., Ishikawa, Y., Takata, M., Joe, K.: Sign language recognition using leap motion controller. In: International Conference on Parallel and Distributed Processing Techniques and Applications, PDPTA 2015, pp. 263–269 (2015)

    Google Scholar 

  10. Khelil, B., Amiri, H.: Hand gesture recognition using leap motion controller for recognition of arabic sign language. In: 3rd International Conference on Automation, Control, Engineering and Computer Science, ACECS 2016, Proceedings of Engineering and Technology (PET), pp. 233–238 (2016)

    Google Scholar 

  11. Quesada, L., López, G., Guerrero, L.A.: Sign Language Recognition Using Leap Motion, pp. 277–288. Springer, Heidelberg (2015)

    Google Scholar 

  12. Chuan, C.-H., Regina, E., Guardino, C.: American Sign Language Recognition Using Leap Motion Sensor. In: 13th International Conference on Machine Learning Application, pp. 541–544 (2014)

    Google Scholar 

  13. Simos, M., Nikolaidis, N.: Greek sign language alphabet recognition using the leap motion device. In: Proceedings of the 9th Hellenic Conference on Artificial Intelligence - SETN 2016, pp. 1–4 (2016)

    Google Scholar 

  14. Mohandes, M., Aliyu, S., Deriche, M.: Prototype Arabic sign language recognition using multi-sensor data fusion of two leap motion controllers. In: 2015 IEEE 12th International Multi-conference on Systems, Signals and Devices, SSD15, pp. 1–6 (2015)

    Google Scholar 

  15. LIVPSIC - Livraria de Psicologia e Ciências da Educação. http://www.livpsic.com/v4/detalhe01.php?id=1340. Accessed 8 July 2015. (in Portuguese)

  16. ASP - Associação de Surdos do Porto. http://www.asurdosporto.org.pt/artigo.asp?idartigo=1250. Accessed 8 July 2015. (in Portuguese)

  17. Four Online Kids’ Games to Learn Sign Language. http://www.brighthubeducation.com/special-ed-hearing-impairments/2910-learning-asl-with-internet-browser-games/. Accessed 8 July 2015

  18. Sign 4 Me for iPad - A Signed English Translator para iPhone, iPod touch e iPad na App Store no iTunes. https://itunes.apple.com/pt/app/sign-4-me-for-ipad-signed/id383462870?mt=8. Accessed 8 July 2015

  19. ASL American Sign Language – Aplicações Android no Google Play. https://play.google.com/store/apps/details?id=com.teachersparadise.aslamericansignlanguage. Accessed 8 July 2015

  20. Wix.com culturas_surdos created by leticia_nadia_lgp based on kindergarten. http://leticia-nadia-lgp.wix.com/culturas_surdos#!page-8. Accessed 16 Apr 2016

  21. RogerVoice, An Android App That Helps The Deaf Have A Conversation On The Phone - Forbes. http://www.forbes.com/sites/federicoguerrini/2014/09/26/tech-that-matters-rogervoice-will-allow-the-deaf-finally-have-a-conversation-on-the-phone/. Accessed 8 July 2015

  22. Tech Tackles Sign Language – MotionSavvy - Forbes. http://www.forbes.com/sites/karstenstrauss/2014/10/27/tech-tackles-sign-language-motionsavvy/. Accessed 8 July 2015

  23. Leap Motion App Store|Leap Motion Apps for Motion Control. https://apps.leapmotion.com/. Accessed 16 Apr 2016

  24. Fingerspeller|Leap Motion Developers. https://developer.leapmotion.com/libraries/462. Accessed 16 Apr 2016

  25. Leap Motion App Store|ASL Digits. https://apps.leapmotion.com/apps/asl-digits/windows. Accessed 16 Apr 2016

  26. Escola Virtual de Língua Gestual Portuguesa - Página Inicial. http://www.lgpescolavirtual.pt/index.php?module=home. Accessed 2 Oct 2015. (in Portuguese)

  27. Soares, F., Sena Esteves, J., Carvalho, V., Lopes, G., Barbosa, F., Ribeiro, P.: Development of a serious game for Portuguese Sign Language. In: 7th International Congress on Ultra Modern Telecommunications and Control Systems and Workshops (ICUMT), Brno, Czech Republic, 6–8 October 2015

    Google Scholar 

  28. O’Leary, R.: LeapTrainer (2013)

    Google Scholar 

  29. Soares, F., Sena Esteves, J., Carvalho, V., Moreira, C., Lourenço, P.: Sign language learning using the hangman videogame. In: 7th International Congress on Ultra Modern Telecommunications and Control Systems and Workshops (ICUMT), Brno, Czech Republic, 6–8 October 2015

    Google Scholar 

  30. ISEP. http://www.isep.ipp.pt/new/viewnew/4146. Accessed 6 Oct 2015

  31. Unity - Game engine, tools and multiplatform. http://unity3d.com/pt/unity. Accessed 4 Apr 2016

  32. SDK Libraries—Leap Motion Unity SDK v3.1 documentation. https://developer.leapmotion.com/documentation/orion/unity/devguide/Leap_SDK_Overview.html#unity. Accessed 4 Apr 2016

  33. Wix.com culturas_surdos created by leticia_nadia_lgp based on kindergarten. http://leticia-nadia-lgp.wix.com/culturas_surdos#!alfabeto-e-números. Accessed 16 Apr 2016

  34. Torres, M., Carvalho, V., Soares, F.: iLearnPSL – development of an interactive application for learning Portuguese sign language: first insight. In: CISPEE16, Vila Real, Portugal, 19–21 October 2016

    Google Scholar 

Download references

Acknowledgments

This work has been supported by COMPETE: POCI-01-0145-FEDER-007043 and FCT – “Fundação para a Ciência e Tecnologia” within the Project Scope: UID/CEC/00319/2013.

Author information

Authors and Affiliations

  1. R&D ALGORITMI Centre, University of Minho, Guimaraes, Portugal

    Vítor Carvalho & Filomena Soares

  2. IPCA – Polytechnic Institute of Cávado and Ave, Barcelos, Portugal

    Marcus Torres & Vítor Carvalho

Authors
  1. Marcus Torres
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vítor Carvalho
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Filomena Soares
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vítor Carvalho .

Editor information

Editors and Affiliations

  1. Carinthia University of Applied Sciences , Villach, Austria

    Michael E. Auer

  2. Carinthia University of Applied Sciences , Villach, Austria

    Danilo G. Zutin

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG

About this paper

Cite this paper

Torres, M., Carvalho, V., Soares, F. (2018). A Serious Game for Learning Portuguese Sign Language - “iLearnPSL”. In: Auer, M., Zutin, D. (eds) Online Engineering & Internet of Things. Lecture Notes in Networks and Systems, vol 22. Springer, Cham. https://doi.org/10.1007/978-3-319-64352-6_55

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-64352-6_55

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-64351-9

  • Online ISBN: 978-3-319-64352-6

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation