Skip to main content
SpringerLink
Log in

Audio rendering smart TV apps through mobile devices

  • Long Paper
  • Published:
Universal Access in the Information Society Aims and scope Submit manuscript

Abstract

TV service providers now offer a variety of features, including broadcast-related ones like electronic programme guides, catch-up and recording features, but also Internet access and a variety of TV applications. These new features turn TV devices into more versatile and interesting platforms, but also clog the screen with more content than ever. Since this content is mainly visual, this means that TVs have even more inaccessible content for visually impaired people. This paper presents the design of a solution that audio renders the TV application’s user interface through a mobile device. Resorting to a mix of accessibility experts and user studies, we compared multiple feedback versions containing different contextual information. Participants reported that the use of repetitive sentences should be avoided; concise feedback feels smoother and quicker but, for some, lacks information while extended feedback can be annoying and take too much time though is appropriate for learning phases. For menus, most participants suggested to include the position of an element and the number of elements. The results helped to identify the critical information to convey to the user and to tailor two modes differing in the amount of contextual information provided, suitable for differently skilled users. Additionally, we condensed the findings into a list of design guidelines which can be generalized to other auditory interfaces meant to be operated by a VI user.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. https://www.netflix.com/browse/audio-description

  2. https://www.youtube.com/watch?v=O7j4_aP8dWA

  3. European Blind Union, Making Information Accessible for All. URL: http://www.euroblind.org/publications-and-resources/making-information-accessible-all

  4. VEWD OS. Website: https://www.vewd.com/products-services/vewd-os/

  5. Electron Website: https://electronjs.org/

References

  1. Affi, D., Dumoulin, J., Bertini, M., Mugellini, E., Abou Khaled, O., Del Bimbo, A.: Sensitv: Smart emotional system for impaired people’s tv. In: Proceedings of the ACM International Conference on Interactive Experiences for TV and Online Video, ACM, New York, NY, USA, TVX ’15, pp. 125–130, (2015) https://doi.org/10.1145/2745197.2755512

  2. Alonso, F., Fuertes, J.L., González, Á.L., Martínez, L.: User-interface modelling for blind users. In: Miesenberger, K., Klaus, J., Zagler, W., Karshmer, A. (eds.) Computers Helping People with Special Needs, pp. 789–796. Springer, Berlin (2008)

    Chapter  Google Scholar 

  3. Andrês Sanoja, S.G.: Block-o-matic: a web page segmentation tool and its evaluation. In: 29ème journées “Base de données avancées”, BDA’13 (2013)

  4. Apple Accessibility: Tv that everyone can control. http://www.apple.com/accessibility/tv/vision/, accessed 18 September (2018)

  5. Apple Inc Vision accessibility—iphone—apple. https://www.apple.com/lae/accessibility/iphone/vision/, accessed 07 July (2019)

  6. Ariyasu, K., Kawakita, H., Handa, T., Kaneko, H.: Tactile sensibility presentation service for smart tv. In: 2014 IEEE 3rd Global Conference on Consumer Electronics (GCCE), pp. 236–237, (2014) https://doi.org/10.1109/GCCE.2014.7031287

  7. Ashok, V., Borodin, Y., Puzis, Y., Ramakrishnan, I.V.: Capti-speak: a speech-enabled web screen reader. In: Proceedings of the 12th Web for All Conference, ACM, New York, NY, USA, W4A ’15, pp. 22:1–22:10, (2015) https://doi.org/10.1145/2745555.2746660

  8. Bhowmick, A., Hazarika, S.M.: An insight into assistive technology for the visually impaired and blind people: state-of-the-art and future trends. Journal on Multimodal User Interfaces pp. 1–24, (2017). https://doi.org/10.1007/s12193-016-0235-6

  9. Bigham, J.P., Prince, C.M., Ladner, R.E.: Webanywhere: a screen reader on-the-go. In: Proceedings of the 2008 International Cross-disciplinary Conference on Web Accessibility (W4A), ACM, New York, NY, USA, W4A ’08, pp. 73–82 (2008). https://doi.org/10.1145/1368044.1368060

  10. Brewster, S.A.: Using nonspeech sounds to provide navigation cues. ACM Trans. Comput.-Hum. Interact. 5(3), 224–259 (1998). https://doi.org/10.1145/292834.292839

    Article  Google Scholar 

  11. Brewster, S.A., Crease, M.G.: Correcting menu usability problems with sound. Behav. Inf. Technol. 18(3), 165–177 (1999). https://doi.org/10.1080/014492999119066

    Article  Google Scholar 

  12. Coelho, J., Duarte, C., Biswas, P., Langdon, P.: Developing accessible tv applications. In: The Proceedings of the 13th International ACM SIGACCESS Conference on Computers and Accessibility, ACM, New York, NY, USA, ASSETS ’11, pp. 131–138, (2011). https://doi.org/10.1145/2049536.2049561

  13. Comcast: Comcast launches new accessible tv features ahead of 2018 paralympics. (2018). https://corporate.comcast.com/stories/comcast-launches-new-accessible-tv-features-ahead-of-2018-paralympics. Accessed 26 April 2019

  14. Comcast Study: Visually impaired adults tune-in to television almost as much as general public. https://corporate.comcast.com/press/releases/study-visually-impaired-adults-tune-in-to-television-almost-as-much-as-general-public. Accessed 26 April 2019 (2018)

  15. Costa, D., Duarte, C.: Visually impaired people and the emerging connected tv: a comparative study of tv and web applications’ accessibility. Univ. Access Inf. Soc. 16(1), 197–214 (2017). https://doi.org/10.1007/s10209-016-0451-6

    Article  Google Scholar 

  16. Courtois, C., D’heer, E.: Second screen applications and tablet users: constellation, awareness, experience, and interest. In: Proceedings of the 10th European Conference on Interactive Tv and Video, ACM, New York, NY, USA, EuroiTV ’12, pp. 153–156, (2012). https://doi.org/10.1145/2325616.2325646

  17. De Rosa, A., Justice, D.: Webreader: A screen reader for everyone, everywhere. In: Proceedings of the 13th Web for All Conference, ACM, New York, NY, USA, W4A ’16, pp. 10:1–10:2, (2016). https://doi.org/10.1145/2899475.2899510

  18. Duarte, C., Costa, D., Feiteira, P., Costa, D.: Building an Adaptive Multimodal Framework for Resource Constrained Systems, Springer London, London, pp. 173–191. (2015). https://doi.org/10.1007/978-1-4471-6708-2_9

  19. Epelde, G., Valencia, X., Carrasco, E., Posada, J., Abascal, J., Diaz-Orueta, U., Zinnikus, I., Husodo-Schulz, C.: Providing universally accessible interactive services through tv sets: implementation and validation with elderly users. Multimed. Tools Appl. 67(2), 497–528 (2013). https://doi.org/10.1007/s11042-011-0949-0

    Article  Google Scholar 

  20. Freedom Scientific Blindness solution: Jaws®. http://www.freedomscientific.com/Products/Blindness/JAWS. Accessed 18 September (2018)

  21. Godard, N., Pecci, I., Isokoski, P., Weslide: Gestural text entry for elderly users of interactive television. In: Proceedings of the 11th European Conference on Interactive TV and Video, ACM, New York, NY, USA, EuroITV ’13, pp. 55–58, (2013). https://doi.org/10.1145/2465958.2465963

  22. Google Android tv help: android tv accessibility features. https://support.google.com/androidtv/answer/6123320?hl=en. Accessed 18 September (2018)

  23. Google Get started on android with talkback. https://support.google.com/accessibility/android/answer/6283677?hl=en. Accessed 07 July (2019)

  24. Guerreiro, J., Gonçalves, D.: Faster text-to-speeches: enhancing blind people’s information scanning with faster concurrent speech. In: Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility, ACM, New York, NY, USA, ASSETS ’15, pp. 3–11, (2015). https://doi.org/10.1145/2700648.2809840

  25. Jassi, I.S., Ruchika, S., Pulakhandam, S., Mukherjee, S., Ashwin, T.S., Reddy, G.R.M.: Ember: a smartphone web browser interface for the blind. In: Proceedings of the 7th International Symposium on Visual Information Communication and Interaction, ACM, New York, NY, USA, VINCI ’14, pp. 106:112, (2014). https://doi.org/10.1145/2636240.2636847

  26. Johnston, M., Stent, A.J.: Epg: speech access to program guides for people with disabilities. In: Proceedings of the 12th International ACM SIGACCESS Conference on Computers and Accessibility, ACM, New York, NY, USA, ASSETS ’10, pp. 257–258, (2010). https://doi.org/10.1145/1878803.1878859

  27. Kim K.J.R.H.J: ’heart-to-feel’: A new audio description coding scheme for the visually impaired on affective cinematography and emotive vibration. In: Proceedings of the ACM International Conference on Interactive Experiences for TV and Online Video, TVX ’14 (2014)

  28. Kuber, R., Yu, W., McAllister, G.: Towards developing assistive haptic feedback for visually impaired internet users. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, New York, NY, USA, CHI ’07, pp 1525–1534, (2007). https://doi.org/10.1145/1240624.1240854

  29. Kuber, R., Yu, W., O’Modhrain, M.S.: Tactile web browsing for blind users. In: Nordahl, R., Serafin, S., Fontana, F., Brewster, S. (eds.) Haptic and Audio Interaction Design, pp. 75–84. Springer, Berlin (2010)

    Chapter  Google Scholar 

  30. Magnusson, C., Tan, C.C.S., Yu, W.: Haptic access to 3d objects on the web. In: Proceeding of the Eurohaptics 2006, p. 4 (2006)

  31. Mynatt, E.D.: Transforming graphical interfaces into auditory interfaces for blind users. Hum.-Comput. Interact. 12(1), 7–45 (1997). https://doi.org/10.1207/s15327051hci1201&2_2

    Article  Google Scholar 

  32. Nandakumar, A., Murray, J.: Companion apps for long arc tv series: Supporting new viewers in complex storyworlds with tightly synchronized context-sensitive annotations. In: Proceedings of the 2014 ACM International Conference on Interactive Experiences for TV and Online Video, ACM, New York, NY, USA, TVX ’14, pp. 3–10, (2014). https://doi.org/10.1145/2602299.2602317

  33. National Institute of Deafness and Other Communication Disorders (NIDCD) Assistive devices for people with hearing, voice, speech, or language disorders. http://www.nidcd.nih.gov/health/hearing/pages/assistive-devices.aspx, Accessed 26 February 2016, (2011)

  34. Norman, D.A., Draper, S.W.: User Centered System Design. New Perspectives on Human-Computer Interaction. L. Erlbaum Associates Inc., Hillsdale, NJ, USA (1986)

    Book  Google Scholar 

  35. NV Access (2019) Nvda. https://www.nvaccess.org/about-nvda/. Accessed 07 July (2019)

  36. Oliveira, R., Ferraz de Abreu, J., Almeida, A.M.: An approach to identify requirements for an itv audio description service. In: Proceedings of the 9th International Interactive Conference on Interactive Television, ACM, New York, NY, USA, EuroITV ’11, pp. 227–230, (2011). https://doi.org/10.1145/2000119.2000166

  37. Rajapakse, R., Dias, M., Weerasekara, K., Dharmaratne, A., Wimalaratne, P.: Audio user interface for visually impaired computer users: in a two dimensional audio environment. In: Proceedings of World Academy of Science, Engineering and Technology (2012)

  38. Samsung Accessibility settings for your tv. https://www.samsung.com/us/support/answer/ANS00062238/. Accessed 18 September (2018)

  39. Sanoja, A., Gançarski, S.: Block-o-Matic: a Web Page Segmentation Tool and its Evaluation. 29ème journées ”Base de données avancées”, BDA’13, https://hal.archives-ouvertes.fr/hal-00881693, poster (2013)

  40. Schmeidler, E., Kirchner, C.: Adding audio description. does it make a difference? J. Vis. Impairment Blind. 197–212 (2001)

  41. Silva, P., Amer, Y., Tsikerdanos, W., Shedd, J., Restrepo, I., Murray, J.: A game of thrones companion: Orienting viewers to complex storyworlds via synchronized visualizations. In: Proceedings of the ACM International Conference on Interactive Experiences for TV and Online Video, ACM, New York, NY, USA, TVX ’15, pp. 167–172, (2015). https://doi.org/10.1145/2745197.2755519

  42. Union, E.B.: Digital TV Accessibility: Report on the Current Status in European Countries. Tech. Rep, European Blind Union (2008)

    Google Scholar 

  43. Vargas, M.L.M.: Combining speech and earcons to assist menu navigation. In: Proceedings of the 9th International Conference on Auditory Display (ICAD2003) (2003)

Download references

Acknowledgements

This work was supported by FCT through the LASIGE Research Unit, ref. UIDB/00408/2020 and ref. UIDP/00408/2020 and by the Reitoria da Universidade de Lisboa through the PhD Grant BD2015. We would like also to thank Fundaç ao Raquel e Martin Sain for providing the volunteers for the studies.

Author information

Authors and Affiliations

  1. LASIGE, Departamento de Informática Faculdade de Ciências, Universidade de Lisboa, 1749-016, Lisbon, Portugal

    Daniel Costa & Carlos Duarte

Authors
  1. Daniel Costa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carlos Duarte
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Daniel Costa.

Ethics declarations

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Costa, D., Duarte, C. Audio rendering smart TV apps through mobile devices. Univ Access Inf Soc 21, 675–689 (2022). https://doi.org/10.1007/s10209-021-00796-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10209-021-00796-1

Search

Navigation