Skip to main content
SpringerLink
Log in

Apollo SignSound: an intelligent system applied to ubiquitous healthcare of deaf people

  • Original Article
  • Published:
Journal of Reliable Intelligent Environments Aims and scope Submit manuscript

Abstract

In Brazil, there are more than 9.3 million people with Hearing Impairment (PWHI) and deaf who face daily accessibility difficulties. On the other hand, there is the growth of the use of mobile devices and the application of the Internet of Things. The motivation for the development of this work lies in the absence of user interface systems based on sign language and customizable according to the profile that is applied to the prevention of risks external to the health of the deaf. This paper proposes the Apollo SignSound system, which promotes accessibility for PWHI and deaf people in a smart home environment, especially regarding safety. The scientific contribution of Apollo SignSound lies in the detection of ambient risks using neural networks. Besides this, SignSound also considers the user profile, mainly the degree of deafness, to generate accessible notifications represented in Brazilian Sign Language (LIBRAS in Portuguese). The notifications of risk sent to the smartphone of PWHI or deaf also can vibrate or turn on the light of the device. We implemented a prototype of a smart home that collects environmental sounds and notifies the deaf user. The scenario-based assessment included three activities of daily living events of a deaf user: a kettle boiling over the stove, a dog barking, and one person knocking on the door. The results indicate the means of the f score of 0.73 for accuracy evaluation. Usability and acceptance evaluations were performed by five deaf users and the results indicate the approval of 90% in the perceived ease of use and perceived usefulness.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

Notes

  1. Available at: https://bit.ly/2s5AEZE

References

  1. Abafire (2020) Sirenes pne. https://abafire.com.br/c/alarme-pne/sirenes-pne/. Accessed 4 Oct 2020

  2. Abech M, Da Costa CA, Barbosa JLV, Rigo SJ, Da Rosa Righi R (2016) A model for learning objects adaptation in light of mobile and context-aware computing. Pers Ubiquit Comput 20:167–184. https://doi.org/10.1007/s00779-016-0902-3

    Article  Google Scholar 

  3. Barbosa J (2015) Ubiquitous computing: Applications and research opportunities (invited talk). In: VI IEEE International conference on computational intelligence and computing research (ICCIC), Madurai, Índia pp 1–8. https://ieeexplore.ieee.org/document/7435625

  4. Barbosa J, Hahn R, Barbosa D, Saccol A (2011) A ubiquitous learning model focused on learner interaction. Int J Learn Technol 6(1):62–83. https://doi.org/10.1504/IJLT.2011.040150

    Article  Google Scholar 

  5. Barbosa DNF, Barbosa JLV, Bassani PBS, Rosa JH, Lewis M, Nino CP (2012) Content management in a ubiquitous learning environment. Int J Comput Appl Technol 46:24. https://doi.org/10.1504/IJCAT.2013.051385

    Article  Google Scholar 

  6. Barbosa JLV, Barbosa DNF, Wagner A (2012) Learning in ubiquitous computing environments. Int J Inf Commun Technol Educ 8:64–77. https://doi.org/10.4018/jicte.2012070108

    Article  Google Scholar 

  7. Barbosa J, Martins C, Franco L, Barbosa D (2016) Trailtrade: a model for trail-aware commerce support. Comput Ind 80:45–53. https://doi.org/10.1016/j.compind.2016.04.006

    Article  Google Scholar 

  8. Barbosa J, Tavares J, Cardoso I, Mota B, Martini B (2018) Trailcare: an indoor and outdoor context-aware system to assist wheelchair users. Int J Hum Comput Stud 104:1–14. https://doi.org/10.1016/j.ijhcs.2018.04.001

    Article  Google Scholar 

  9. Bavaresco R, Barbosa JLV, Vianna HD, Buttenbender PC, Dias LPS (2020) Design and evaluation of a context-aware model based on psychophysiology. Comput Methods Progr Biomed 189:105–299. https://doi.org/10.1016/j.cmpb.2019.105299

    Article  Google Scholar 

  10. Bempong J, Stainslow J, Behm G (2020) Accessible smart home system for the deaf and hard-of-hearing. https://www.rit.edu/ntid/nyseta/sites/rit.edu.ntid.nyseta/files/docs/fullpapers_PDFs/StanislowJoeFullPaper.pdf

  11. BrazilianMinistryOfHealth: Acessibilidade. http://portalms.saude.gov.br/acessibilidade (2020). Accessed 4 Oct 2020

  12. Carneiro C, Barbosa J (2016) Hermes: um modelo para acessibilidade ubíqua dedicado à deficiência auditiva. Revista Brasileira de Computação Aplicada 8(3):19–33. https://doi.org/10.5335/rbca.v8i3.5816

    Article  Google Scholar 

  13. Cho H, Kim H (2020) Home environmental sound alert system for deaf and hard-of-hearing users. http://spandh.dcs.shef.ac.uk/chat2017/papers/CHAT_2017_cho.pdf

  14. da Surdez C (2020) Babá eletrônica vibratória para surdos. https://cronicasdasurdez.com/baba-eletronica-vibratoria-surdos/. Accessed 4 Oct 2020

  15. da Surdez C (2020) Os perigos da surdez. http://cronicasdasurdez.com/os-perigos-da-surdez/. Accessed 4 Oct 2020

  16. Damasceno VH, Victória Barbosa JL (2019) A scalable model for building context-aware applications for noncommunicable diseases prevention. Inf Process Lett 148:1–6. https://doi.org/10.1016/j.ipl.2019.03.010

    Article  MATH  Google Scholar 

  17. Davis H, Silverman R (1970) Hearing and deafness. Holt, Rinehart and Winston, New York, p 522

    Google Scholar 

  18. Dey AK (2001) Understanding and using context. Pers Ubiquit Comput 5:4–7. https://doi.org/10.1007/s007790170019

    Article  Google Scholar 

  19. Dias LPS, Barbosa JLV, Vianna HD (2018) Gamification and serious games in depression care: a systematic mapping study. Telemat Informat 35:213–224. https://doi.org/10.1016/j.tele.2017.11.002

    Article  Google Scholar 

  20. Endsley MR (1995) Toward a theory of situation awareness in dynamic systems. Hum Fact 37(1):32–64. https://doi.org/10.1518/001872095779049543

    Article  Google Scholar 

  21. Ferreira LGA, Matter VK, Barbosa DNF, Gluz JC, Barbosa JLV (2020) Using learners group profile for content recommendation in ubiquitous environments. Int J Inf Commun Technol Educ 16:1–19. https://doi.org/10.4018/IJICTE.2020100101

    Article  Google Scholar 

  22. Franco LK, Rosa JH, Barbosa JLV, Costa CA, Yamin AC (2011) Mucs: a model for ubiquitous commerce support. Electron Commerce Res Appl 10:237–246. https://doi.org/10.1016/j.elerap.2010.08.006

    Article  Google Scholar 

  23. Google (2020) Google forms. https://www.google.com/forms/about/. Accessed 4 Oct 2020

  24. Gunnarsdóttir K, Arribas-Ayllon M (2020) Ambient intelligence: a narrative in search of users. https://www.academia.edu/1080720/Ambient_Intelligence_an_innovation_narrative. Accessed 4 Oct 2020

  25. Handtalk (2020) Tradutor libras. https://www.handtalk.me/. Accessed 4 Oct 2020

  26. Heart-it (2020) O que é deficiência. https://www.hear-it.org/pt/deficiencia-auditiva. Accessed 4 Oct 2020

  27. Helfer GA, Barbosa JLV, Santos RB, Costa AB (2020) A computational model for soil fertility prediction in ubiquitous agriculture. Comput Eletron Agric 175:1–26. https://doi.org/10.1016/j.compag.2020.105602

    Article  Google Scholar 

  28. IBGE (2020) Censo demográfico 2010. https://www.ibge.gov.br/estatisticas-novoportal/sociais/populacao/9662-censo-demografico-2010.html?edicao=9749&t=destaques. Accessed 4 Oct 2020

  29. Kim G, Shin S, Kim J, Kim H (2018) Sound event detection and haptic vibration based home monitoring assistant system for the deaf and hard-of-hearing. In: Proceedings of the 2018 workshop on multimedia for accessible human computer interface (MAHCI’18), pp 1–7. ACM, New York, NY, USA. https://doi.org/10.1145/3264856.3264857

  30. Kokar MM, Matheus CJ, Baclawski K (2009) Ontology-based situation awareness. Inf Fus Spec Issue High Level Inf Fus Situat Awareness 10(1):83–98

    Google Scholar 

  31. Larentis A, Barbosa DNF, Silva CR, Barbosa JL (2019) Applied computing to education on noncommunicable chronic diseases: a systematic mapping study. Telemed e-Health 1:1–10. https://doi.org/10.1089/tmj.2018.0282

    Article  Google Scholar 

  32. Likert R (1932) A technique for the measurement of attitudes. Arch Psychol 22:1–55

    Google Scholar 

  33. Pittoli F, Vianna HD, Victória Barbosa JL, Butzen E, Gaedke MA, Dias da Costa JS, Scherer do Santos RB (2018) An intelligent system for prognosis of noncommunicable diseases’ risk factors. Telemat Informat 1:1–34. https://doi.org/10.1016/j.tele.2018.02.005

    Article  Google Scholar 

  34. Planalto (2020) Libras law. http://www.planalto.gov.br/ccivil_03/leis/2002/l10436.htm. Accessed 4 Oct 2020

  35. ProDeaf (2020) Tradutor libras. https://www.prodeaf.net/. Accessed 4 Oct 2020

  36. Sadri F (2011) Ambient intelligence: a survey. ACM Comput Surv 43:36. https://doi.org/10.1145/1978802.1978815

    Article  Google Scholar 

  37. Santos B (2020) Internet das coisas: da teoria à prática. http://homepages.dcc.ufmg.br/~mmvieira/cc/papers/internet-das-coisas.pdf. Accessed 4 Oct 2020

  38. Satyanarayanan M (2001) Pervasive computing: vision and challenges. IEEE Pers Commun 8:10–17. https://doi.org/10.1109/98.943998

    Article  Google Scholar 

  39. Scenihr (2020)Health risks from exposure to noise from personal music players. http://ec.europa.eu/health/ph_risk/committees/04_scenihr/docs/scenihr_o_018.pdf. Accessed 4 Oct 2020

  40. SilenceProject (2020) Projeto silence. https://www.projetosilence.com/processo. Accessed 4 Oct 2020

  41. SNPD (2020) Acessibilidade. https://www.mdh.gov.br/navegue-por-temas/pessoa-com-deficiencia/programas/acessibilidade. Accessed 4 Oct 2020

  42. Tavares J, Barbosa J, Cardoso I, Costa C, Yamin A, Real R (2016) Hefestos: an intelligent system applied to ubiquitous accessibility. Univ Access Inf Soc 8:1–19. https://doi.org/10.1007/s10209-015-0423-2

    Article  Google Scholar 

  43. Thomas T (2020) Smart assistant for deaf and dumb. https://www.mepits.com/project/288/wireless/smart-assistant-for-deaf-and-dumb. Accessed 4 Oct 2020

  44. Vanderheiden G (2008) Ubiquitous accessibility, common technology core, and micro assistive technology: Commentary on “computers and people with disabilities. ACM Trans Access Comput 10:1–10

    Article  Google Scholar 

  45. Vianna HD, Barbosa JLV (2020) Pompilos, a model for augmenting health assistant applications with social media content. J Univ Comput Sci 26:4–32. http://www.jucs.org/jucs_26_1/pompilos_a_model_for

  46. Vianna HD, Barbosa JLV (2014) A model for ubiquitous care of noncommunicable diseases. IEEE J Biomed Health Informat 18:1597–1606. https://doi.org/10.1109/jbhi.2013.2292860

    Article  Google Scholar 

  47. Wagner A, Barbosa JLV, Barbosa DNF (2014) A model for profile management applied to ubiquitous learning environments. Expert Syst Appl 41:2023–2034. https://doi.org/10.1016/j.eswa.2013.08.098

    Article  Google Scholar 

  48. Weiser M (1991) The computer for the 21st century. Sci Am 265:94–104. https://doi.org/10.1145/329124.329126

    Article  Google Scholar 

  49. Yoon C, Kim S (2007) Convenience and tam in a ubiquitous computing environment: the case of wireless lan. Electron Commer Res Appl 6:102–112. https://doi.org/10.1016/j.elerap.2006.06.009

    Article  Google Scholar 

Download references

Acknowledgements

The authors thank the Rio Grande do Sul State Research Support Foundation (Fundação de Amparo à Pesquisa do Estado do Rio Grande do Sul (FAPERGS)), the Coordination for the Improvement of Higher Education Personnel—Brazil (Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES))—Financing Code 001, the National Council for Scientific and Technological Development (Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)), and the University of Vale do Rio dos Sinos (Universidade do Vale do Rio dos Sinos (Unisinos)) for supporting the development of this study. The authors especially acknowledge the support of the Applied Computing Graduate Program (Programa de Pós-Graduação em Computação Aplicada (PPGCA)) and the Mobile Computing Laboratory (Laboratório de Computação Móvel (Mobilab)) at Unisinos.

Author information

Authors and Affiliations

  1. Applied Computing Graduate Program (PPGCA) of University of Vale do Rio dos Sinos (UNISINOS), Av. Unisinos, 950-Bairro Cristo Rei, São Leopoldo, RS, CEP: 93.022-750, Brazil

    João Elison da Rosa Tavares & Jorge Luis Victória Barbosa

Authors
  1. João Elison da Rosa Tavares
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jorge Luis Victória Barbosa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to João Elison da Rosa Tavares.

Ethics declarations

Declaration of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Ethics statement

This research did not require ethical approval in accordance with the regulations of the University of Vale do Rio dos Sinos (UNISINOS). The subjects who participated in the evaluation were not patients in treatment, but rather academic volunteers and teachers. They assessed the usability aspects of Apollo SignSound and not the effectiveness of its application. In addition, the participants agreed to participate in the evaluation of Apollo SignSound. Informed consent was obtained from all individual participants included in the study and the data were anonymized.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

da Rosa Tavares, J.E., Victória Barbosa, J.L. Apollo SignSound: an intelligent system applied to ubiquitous healthcare of deaf people. J Reliable Intell Environ 7, 157–170 (2021). https://doi.org/10.1007/s40860-020-00119-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40860-020-00119-w

Keywords

Search

Navigation