Skip to main content
SpringerLink
Log in

Alternative modalities for visually impaired users to control smart TVs

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Smart TVs offer a variety of features that increase interactivity and available services when compared to other TVs. Furthermore, remote controls have become more complex, with more and smaller buttons, making interaction difficult for people with visual impairments. To address some of these concerns, the solution described herein offers, through a mobile application, a set of alternative interaction modalities ranging from mid-air gestures to speech commands. This paper presents the results of a user study comparing user and system performance using the proposed modalities. Touch and speech were the most efficient modalities. The combined usage of the mobile application and TalkBack resulted in more user errors, consequence of the introduction of an additional interaction layer. Most participants reported they would replace the remote control with this solution. The discussion of the findings and a lessons learned section are the main contributions of this work.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Abrams M, Phanouriou C, Batongbacal AL, Williams SM, Shuster JE (1999) UIML: an appliance-independent XML user interface language. Comput Netw 31:1695–1708

    Article  Google Scholar 

  2. Ali MF and Pérez-Quiñones MA (2002) "using task models to generate multi-platform user interfaces while ensuring usability," in CHI '02 Extended Abstracts on Human Factors in Computing Systems, New York, NY, USA

  3. Azenkot S and Lee NB (2013) "exploring the use of speech input by blind people on Mobile devices," in Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility, New York, NY, USA

  4. Bailly G, Vo DB, Lecolinet E and Guiard Y (2011) "gesture-aware remote controls: guidelines and interaction technique," in Proceedings of the 13th International Conference on Multimodal Interfaces, New York, NY, USA

  5. S. Balbo, S. Goschnick, D. Tong and C. Paris (2005) "leading web usability evaluations to WAUTER," 1

  6. Bhachu AS (2011) "Inclusive Technologies for Enhancing the Accessibility of Digital Television," SIGACCESS Access. Comput., pp. 9–12, 1

  7. Brewster SA, Wright PC and Edwards ADN (1993) "an evaluation of Earcons for use in auditory human-computer interfaces," in Proceedings of the INTERACT '93 and CHI '93 Conference on Human Factors in Computing Systems, New York, NY, USA

  8. T. Bruhn Hinné and S. Keates (2011) "using motion-sensing remote controls with older adults," in Universal Access in Human-Computer Interaction. Users Diversity, Berlin

  9. Coelho J, Duarte C, Biswas P and Langdon P (2011) "developing accessible TV applications," in The Proceedings of the 13th International ACM SIGACCESS Conference on Computers and Accessibility, New York, NY, USA

  10. Costa D, Duarte C (2017) Visually Impaired People and the Emerging Connected TV: A Comparative Study of TV and Web Applications' Accessibility. Univers. Access Inf. Soc 16(3):197–214

    Article  Google Scholar 

  11. C. Courtois and E. D'heer (2012) "second screen applications and tablet users: constellation, awareness, experience, and interest," in Proceedings of the 10th European Conference on Interactive Tv and Video, New York, NY, USA

  12. Duarte C, Desart S, Costa D, Dumas B (2017) Designing multimodal Mobile interaction for a text messaging application for visually impaired users. Frontiers in ICT 4:26

    Article  Google Scholar 

  13. E. B. Union (2008) Digital TV accessibility: report on the current status in European countries

  14. E. B. Union (2013) Access to television

    Google Scholar 

  15. Epelde G, Carrasco E, Zimmermann G, Alexandersson J, Neßelrath R, Dubielzig M (2011) Universal remote console-based next-generation accessible television. Univ Access Inf Soc 12:73–87

    Article  Google Scholar 

  16. Epelde G, Valencia X, Carrasco E, Posada J, Abascal J, Diaz-Orueta U, Zinnikus I, Husodo-Schulz C (2013) Providing Universally Accessible Interactive Services Through TV Sets: Implementation and Validation with Elderly Users. Multimedia Tools Appl 67:497–528

    Article  Google Scholar 

  17. Freeman WT and Weissman CD (1994) "television control by hand gestures," 1

  18. Funes MM, Fortes RPM and Goularte R (2017) "gestural interaction for accessibility of web videos: a case study analysis with blind and visually impaired users," in Proceedings of the 23rd Brazillian Symposium on Multimedia and the Web, New York, NY, USA

  19. Gajos KZ, Wobbrock JO and Weld DS (2007) "automatically generating user interfaces adapted to Users' Motor and vision capabilities," in Proceedings of the 20th Annual ACM Symposium on User Interface Software and Technology, New York, NY, USA

  20. Godard N, Pecci I and Isokoski P (2013) "WeSlide: gestural text entry for elderly users of interactive television," in Proceedings of the 11th European Conference on Interactive TV and Video, New York, NY, USA

  21. I. Hwang, H. Kim, J. Cha, C. Ahn, K. Kim and J. Park (2015) "a gesture based TV control interface for visually impaired: initial design and user study," in 2015 21st Korea-Japan Joint Workshop on Frontiers of Computer Vision (FCV)

  22. Johnston M and Stent AJ (2010) "EPG: speech access to program guides for people with disabilities," in Proceedings of the 12th International ACM SIGACCESS Conference on Computers and Accessibility, New York, NY, USA

  23. Kearney-Volpe C, Holloway S and Hurst A (2019) "entertainment for all: understanding media streaming accessibility," in Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, New York, NY, USA

  24. Kohler M (1998) "special topics of gesture recognition applied in intelligent home environments," in In Proceedings of the Gesture Workshop

  25. Kühnel C, Westermann T, Hemmert F, Kratz S, Müller A, Möller S (2011) I'm home: defining and evaluating a gesture set for smart-home control. International Journal of Human-Computer Studies 69:693–704

    Article  Google Scholar 

  26. Luo X, Xu Y and Mullen C (2014) "Oris: enhance social self-awareness for visually impaired people," in CHI '14 Extended Abstracts on Human Factors in Computing Systems, New York, NY, USA

  27. S. Mehrotra (2018) "Potmote: a TV remote control for older adults," in Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility, New York, NY, USA

  28. Nandakumar A and Murray J (2014) "companion apps for long arc TV series: supporting new viewers in complex Storyworlds with tightly synchronized context-sensitive annotations," in Proceedings of the 2014 ACM International Conference on Interactive Experiences for TV and Online Video, New York, NY, USA

  29. T. Neate, M. Jones and M. Evans (2015) "mediating attention for second screen companion content," in Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, New York, NY, USA

  30. Oh U, Stearns L, Pradhan A, Froehlich JE and Findlater L (2017) "investigating microinteractions for people with visual impairments and the potential role of on-body interaction," in Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility, New York, NY, USA

  31. Oliveira R, Ferraz de Abreu J, Almeida AM (2011) "an approach to identify requirements for an iTV audio description service," in Proceddings of the 9th International Interactive Conference on Interactive Television, New York, NY, USA

  32. Oviatt S (2003) "The human-computer interaction handbook: Fundamentals, Evolving Technologies and Emerging Applications," J. A. Jacko and A. Sears, Eds., Hillsdale, NJ: L. Erlbaum Associates Inc., 2003, pp. 286–304

  33. V. F. Santana and M. C. Baranauskas (2014) "Log-Based Personalization Tool As an Assistive Technology," in 16th International Conference on Human-Computer Interaction. Theories, Methods, and Tools - Volume 8510, Berlin

  34. Santana VF and Silva FEF (2019) "an easy to use data logger for local user studies," in Proceedings of the 16th Web For All 2019 Personalization - Personalizing the Web, New York, NY, USA

  35. Sauro J (2011) Measuring usability with the system usability scale (SUS).

  36. Shah I (2008) Event patterns as indicators of usability problems. Journal of King Saud University - Computer and Information Sciences 20:31–43

    Article  Google Scholar 

  37. Shinohara K, Wobbrock JO (2016) "Self-Conscious or Self-Confident? A Diary Study Conceptualizing the Social Accessibility of Assistive Technology," ACM Trans. Access. Comput., vol. 8, pp. 5:1--5:31

  38. Springett MV and Griffiths RN (2007) "accessibility of interactive Television for Users with low vision: learning from the web," in Proceedings of the 5th European Conference on Interactive TV: A Shared Experience, Berlin

  39. Torres DZ (2018) "user practices for smartphone control of TV," in Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct, New York, NY, USA

  40. Trindade D, Rodrigues A, Guerreiro T and Nicolau H (2018) "hybrid-Brailler: combining physical and gestural interaction for Mobile braille input and editing," in Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, New York, NY, USA

  41. Vatavu RD (2012) "user-defined gestures for free-hand TV control," in Proceedings of the 10th European Conference on Interactive TV and Video, New York, NY, USA

  42. Vatavu RD (2012) "Point & Click Mediated Interactions for Large Home Entertainment Displays," Multimedia Tools Appl., vol. 59, pp. 113–128

  43. Vatavu RD and Pentiuc SG (2008) "interactive coffee tables: interfacing TV within an intuitive, fun and shared experience," in Changing Television Environments, Berlin

Download references

Author information

Authors and Affiliations

  1. LASIGE, Departamento de Informática, Faculdade de Ciências, Universidade de Lisboa, 1749-016, Lisbon, Portugal

    Daniel Costa & Carlos Duarte

Authors
  1. Daniel Costa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carlos Duarte
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Daniel Costa.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Costa, D., Duarte, C. Alternative modalities for visually impaired users to control smart TVs. Multimed Tools Appl 79, 31931–31955 (2020). https://doi.org/10.1007/s11042-020-09656-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09656-1

Keywords

Search

Navigation