skip to main content
10.1145/3471391.3471392acmotherconferencesArticle/Chapter ViewAbstractPublication PagesinteraccionConference Proceedingsconference-collections
research-article

Automatic captions on video calls, a must for the elderly: Using Mozilla DeepSpeech for the STT

Published: 22 September 2021 Publication History

Abstract

Due to the COVID 19 pandemic, the use of video conferencing and video calls increased in education and work, but also in the family environment due to the dangers of face-to-face meetings. Many elderly people suffer from hearing problems, which makes it difficult to take full advantage of video calls, which is why an automatic subtitling tool for conversations was designed using Speech to Text. It uses the free Mozilla DeepSpeech tool. This platform independent video calling software enables the elderly or anyone with hearing impairments to enjoy video calls. A transparent interface was designed that is superimposed on the video call, and has simple options that allow us to change the settings for the size and color of the text. It must also be taken into account that many elderly people have visual problems, so it is important to be able to adapt the text. Tests have been done in Spanish and English, but we have trained models that allow us to easily add dozens of languages.

References

[1]
Assistant Secretary for Public Affairs. 2013. System Usability Scale (SUS). system- usability- scale.html
[2]
Rosana Ardila, Megan Branson, Kelly Davis, Michael Henretty, Michael Kohler, Josh Meyer, Reuben Morais, Lindsay Saunders, Francis M. Tyers, and Gregor Weber. 2020. Common Voice: A Massively-Multilingual Speech Corpus. arXiv:1912.06670 [cs] (March 2020). http://arxiv.org/abs/1912.06670 arXiv: 1912.06670 version: 2.
[3]
A.M.Irfan Taufan Asfar and Andi Muhamad Iqbal Akbar Asfar. 2020. How To Using Online Meeting On Jitsi Meet Aplication. (2020). https://doi.org/10.13140/RG.2.2.28536.80644
[4]
BigBlueButton. 2021. bigbluebutton/deepspeech-web. https://github.com/bigbluebutton/deepspeech-web
[5]
E. Gabriella Coleman. 2013. Coding freedom: the ethics and aesthetics of hacking. Princeton University Press, Princeton.
[6]
Coqui. 2021. Coqui - Models. https://coqui.ai/models
[7]
Coqui. 2021. coqui-ai/STT. https://github.com/coqui-ai/STT original-date: 2021-03-04T04:54:42Z.
[8]
Mary C. Dyson and Mark Haselgrove. 2001. The influence of reading speed and line length on the effectiveness of reading from screen. International Journal of Human-Computer Studies 54, 4 (April 2001), 585--612. https://doi.org/10.1006/ijhc.2001.0458
[9]
Theodoros Giannakopoulos. 2021. tyiannak/pyAudioAnalysis. https://github.com/tyiannak/pyAudioAnalysis original-date: 2014-08-27T12:43:13Z.
[10]
Google. 2021. Use captions in a video call - Computer - Google Meet Help. https://support.google.com/meet/answer/9300310
[11]
Francisco Grimaldo, Ariadna Fuertes, Máximo Cobos, Miguel Arevalillo, Carlos Pérez, Paloma Moreno, and Ricardo Ferrís. 2014. BigBlueButton, una alternativa de código abierto para la comunicación interactiva en actividades educativas. Universidad de Oviedo. Escuela de Ingeniería Informática. https://upcommons.upc.edu/handle/2099/15454
[12]
S.R. Gulliver and G. Ghinea. 2003. How level and type of deafness affect user perception of multimedia video clips. Universal Access in the Information Society 2, 4 (Nov. 2003), 374--386. https://doi.org/10.1007/s10209-003-0067-5
[13]
G. Hickish. 1989. Hearing problems of elderly people. BMJ: British Medical Journal 299, 6713 (Dec. 1989), 1415--1416. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1838323/
[14]
Leena Häkkinen. 1984. Vision in the elderly and its use in the social environment. Scandinavian Journal of Social Medicine. Supplementum 35 (1984), 1--60. https://www.jstor.org/stable/45199751
[15]
BigBlueButton Inc. 2021. BigBlueButton. https://docs.bigbluebutton.org/
[16]
Jaco-Assistant. 2020. Jaco-Assistant / Scribosermo. https://gitlab.com/Jaco-Assistant/Scribosermo
[17]
Jeff Sauro. [n.d.]. MeasuringU: Measuring Usability with the System Usability Scale (SUS). https://measuringu.com/sus/
[18]
Jitsi. 2021. Architecture - Jitsi Meet Handbook. https://jitsi.github.io/handbook/
[19]
Allen St John. 2020. It's Not Just Zoom. Google Meet, Microsoft Teams, and Webex Have Privacy Issues, Too. https://www.consumerreports.org/videoconferencing-services/videoconferencing-privacy-issues-google-microsoft-webex/
[20]
Ankur Joshi, Saket Kale, Satish Chandel, and D. K. Pal. 2015. Likert Scale: Explored and Explained. Current Journal of Applied Science and Technology (Feb. 2015), 396--403. https://doi.org/10.9734/BJAST/2015/14975
[21]
Riverbank Computing Limited. 2021. PyQt5: Python bindings for the Qt cross platform application toolkit. https://www.riverbankcomputing.com/software/pyqt/
[22]
Paulo Renato C. Mendes, Eduardo S. Vieira, Pedro Vinicius A. de Freitas, Antonio José G. Busson, Álan Lívio V. Guedes, Carlos de Salles Soares Neto, and Sérgio Colcher. 2020. Shaping the Video Conferences of Tomorrow With AI. In Anais Estendidos do Simpósio Brasileiro de Sistemas Multimídia e Web (WebMedia). SBC, 165--168. https://doi.org/10.5753/webmedia_estendido.2020.13082
[23]
Mozilla. 2021. Common Voice by Mozilla. https://commonvoice.mozilla.org/
[24]
Mozilla. 2021. mozilla/DeepSpeech. https://github.com/mozilla/DeepSpeechoriginal-date: 2016-06-02T15:04:53Z.
[25]
Nacimiento-García, Eduardo, González-González, Carina S., and Gutiérrez-Vela, Francisco L. 2020. Free software virtual assistants for designing pervasive gaming experiences to promote active aging: State of the art. CEUR-WS.org/Vol-2747 - VI Iberoamerican Conference of Computer Human Interaction (Sept. 2020), 10.
[26]
Seung-Bo Park, Kyung-Jin Oh, Heung-Nam Kim, and Geun-Sik Jo. 2008. Automatic Subtitles Localization through Speaker Identification in Multimedia System. In 2008 IEEE International Workshop on Semantic Computing and Applications. 166--172. https://doi.org/10.1109/IWSCA.2008.28
[27]
resemble ai. [n.d.]. resemble-ai/Resemblyzer. https://github.com/resemble-ai/Resemblyzer
[28]
Zoe Roupa. 2010. The use of technology by the elderly. HEALTH SCIENCE JOURNAL 4, 2 (2010), 10.
[29]
Agnieszka Szarkowska and Olivia Gerber-Morón. 2018. Viewers can keep up with fast subtitles: Evidence from eye movements. PLOS ONE 13, 6 (June 2018), e0199331. https://doi.org/10.1371/journal.pone.0199331
[30]
Nik Vaessen. 2018. Speech-to-Text in Jitsi Meet. https://doi.org/10.5446/41630
[31]
Will T. 2017. Measuring and Interpreting System Usability Scale (SUS). UIUX Trend. https://uiuxtrend.com/measuring-system-usability-scale-sus/

Cited By

View all

Index Terms

  1. Automatic captions on video calls, a must for the elderly: Using Mozilla DeepSpeech for the STT

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    Interacción '21: Proceedings of the XXI International Conference on Human Computer Interaction
    September 2021
    167 pages
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    In-Cooperation

    • Universidad de Málaga, Instituto de Salud Carlos III: Universidad de Málaga, Instituto de Salud Carlos III

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 22 September 2021

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. captions
    2. deafness
    3. elderly
    4. speech to text
    5. subtitles

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Funding Sources

    Conference

    Interacción '21

    Acceptance Rates

    Overall Acceptance Rate 109 of 163 submissions, 67%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 137
      Total Downloads
    • Downloads (Last 12 months)30
    • Downloads (Last 6 weeks)1
    Reflects downloads up to 17 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media