abstract

Public Access

Mixed Reality Speaker Identification as an Accessibility Tool for Deaf and Hard of Hearing Users

Authors:

Abraham Glasser,

Edward Mason Riley,

Kaitlyn Weeks,

Raja KushalnagarAuthors Info & Claims

VRST '19: Proceedings of the 25th ACM Symposium on Virtual Reality Software and Technology

Article No.: 80, Pages 1 - 3

https://doi.org/10.1145/3359996.3364720

Published: 12 November 2019 Publication History

All formats PDF

Abstract

People who are Deaf or Hard of Hearing (DHH) benefit from text captioning to understand audio, yet captions alone are often insufficient for the complex environment of a panel presentation, with rapid and unpredictable turn-taking among multiple speakers. It is challenging and tiring for DHH individuals to view captioned panel presentations, leading to feelings of misunderstanding and exclusion. In this work, we investigate the potential of Mixed Reality (MR) head-mounted displays for providing captioning with visual cues to indicate which person on the panel is speaking. For consistency in our experimental study, we simulate a panel presentation in virtual reality (VR) with various types of MR visual cues; in a study with 18 DHH participants, visual cues made it easier to identify speakers.

References

[1]

A. Brown, J. Turner, J. Patterson, A. Schmitz, M. Armstrong, M. Glancy. 2017. Subtitles in 360-degree Video. In 2017 ACM International Conference on Interactive Experiences for TV and Online Video (TVX '17 Adjunct). ACM, New York, NY, USA, 3-8.

Digital Library

Google Scholar

[2]

K. Kurzhals, E. Cetinkaya, Y. Hu, W. Wang, D. Weiskopf. 2017. Close to the Action: Eye-Tracking Evaluation of Speaker-Following Subtitles. In Proc. 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, NY, NY, USA, 6559-6568.

Digital Library

Google Scholar

[3]

R. Kushalnagar, G. Behm, K. Wolfe, P. Yeung, B. Dingman, S. Ali, A. Glasser, C. Ryan. 2017. RTTD-ID: Tracked Captions with Multiple Speakers for Deaf Students. In Proc. 2018 ASEE Annual Conference & Exposition (ASEE ‘18). Salt Lake City, UT, USA. https://peer.asee.org/30945

Google Scholar

[4]

S. Rothe, K. Tran, H. Hußmann. 2018. Dynamic Subtitles in Cinematic Virtual Reality. In Proc. 2018 ACM International Conference on Interactive Experiences for TV and Online Video (TVX '18). ACM, New York, NY, USA, 209-214.

Digital Library

Google Scholar

Cited By

View all

McDonnell EFindlater L(2024)Envisioning Collective Communication Access: A Theoretically-Grounded Review of Captioning Literature from 2013-2023Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675649(1-18)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3663548.3675649
Nevsky ABircanin FCruice MWilson SSimperl ENeate T(2024)"I Wish You Could Make the Camera Stand Still": Envisioning Media Accessibility Interventions with People with AphasiaProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675598(1-17)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3663548.3675598
Lee DKim HPark J(2024)Design and verification of universal virtual shopping store applicationVirtual Reality10.1007/s10055-024-01063-328:4Online publication date: 9-Nov-2024
https://doi.org/10.1007/s10055-024-01063-3
Show More Cited By

Recommendations

Accessibility for Deaf and Hard of Hearing Users: Sign Language Conversational User Interfaces
CUI '20: Proceedings of the 2nd Conference on Conversational User Interfaces

With the proliferation of voice-based conversational user interfaces (CUIs) comes accessibility barriers for Deaf and Hard of Hearing (DHH) users. There has not been significant prior research on sign-language conversational interactions with ...
SoundHapticVR: Head-Based Spatial Haptic Feedback for Accessible Sounds in Virtual Reality for Deaf and Hard of Hearing Users
ASSETS '24: Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility

Virtual Reality (VR) systems use immersive spatial audio to convey critical information, but these audio cues are often inaccessible to Deaf or Hard-of-Hearing (DHH) individuals. To address this, we developed SoundHapticVR, a head-based haptic system ...
Accessibility for the Deaf and Hard-of-Hearing Introduction to the Special Thematic Session
Computers Helping People with Special Needs
Abstract
The Special Thematic Session on Accessibility for the Deaf and Hard of Hearing underscored the urgent need to address the escalating prevalence of hearing disabilities, which affected an estimated 430 million individuals worldwide. This session ...

Comments

Information & Contributors

Information

Published In

VRST '19: Proceedings of the 25th ACM Symposium on Virtual Reality Software and Technology

November 2019

498 pages

ISBN:9781450370011

DOI:10.1145/3359996

Editors:
Tomas Trescak
Western Sydney University
,
Simeon Simoff
Western Sydney University
,
Deborah Richards
Macquarie University
,
Anton Bogdanovych
Western Sydney University
,
Thierry Duval
IMT Atlantique
,
Torsten Kuhlen
RWTH Aachen University
,
Huyen Nguyen
University of New South Wales
,
Shigeo Morishima
University of Waseda
,
Yuichi Itoh
University of Osaka
,
Richard Skarbez
La Trobe
,
Anton Bogdanovych
Western Sydney University
,
Martin Masek
Edith Cowan University

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 November 2019

Check for updates

Author Tags

Qualifiers

Abstract
Research
Refereed limited

Funding Sources

Conference

VRST '19

Sponsor:

VRST '19: 25th ACM Symposium on Virtual Reality Software and Technology

November 12 - 15, 2019

NSW, Parramatta, Australia

Acceptance Rates

Overall Acceptance Rate 66 of 254 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
793
Total Downloads

Downloads (Last 12 months)165
Downloads (Last 6 weeks)25

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

McDonnell EFindlater L(2024)Envisioning Collective Communication Access: A Theoretically-Grounded Review of Captioning Literature from 2013-2023Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675649(1-18)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3663548.3675649
Nevsky ABircanin FCruice MWilson SSimperl ENeate T(2024)"I Wish You Could Make the Camera Stand Still": Envisioning Media Accessibility Interventions with People with AphasiaProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675598(1-17)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3663548.3675598
Lee DKim HPark J(2024)Design and verification of universal virtual shopping store applicationVirtual Reality10.1007/s10055-024-01063-328:4Online publication date: 9-Nov-2024
https://doi.org/10.1007/s10055-024-01063-3
Ubur SEtori NGhasemi SKing KGračanin DGini M(2024)EasyCaption: Investigating the Impact of Prolonged Exposure to Captioning on VR HMD on General PopulationUniversal Access in Human-Computer Interaction10.1007/978-3-031-60881-0_24(382-403)Online publication date: 29-Jun-2024
https://dl.acm.org/doi/10.1007/978-3-031-60881-0_24
Davis JWang HChilana PYang X(2023)"It's Not an Issue of Malice, but of Ignorance"Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36109017:3(1-31)Online publication date: 27-Sep-2023
https://dl.acm.org/doi/10.1145/3610901
Wang YLi ZChelladurai PDannels WOh TPeiris R(2023)Haptic-Captioning: Using Audio-Haptic Interfaces to Enhance Speaker Indication in Real-Time Captions for Deaf and Hard-of-Hearing ViewersProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581076(1-14)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581076
Gaggi OPerinello L(2022)Improving accessibility of web accessibility rulesProceedings of the 2022 ACM Conference on Information Technology for Social Good10.1145/3524458.3547267(167-174)Online publication date: 7-Sep-2022
https://dl.acm.org/doi/10.1145/3524458.3547267
Gaggi OPederiva V(2021)WCAG4All, a tool for making web accessibility rules accessible2021 IEEE 18th Annual Consumer Communications & Networking Conference (CCNC)10.1109/CCNC49032.2021.9369484(1-6)Online publication date: 9-Jan-2021
https://dl.acm.org/doi/10.1109/CCNC49032.2021.9369484
Amin AHassan SHuenerfauth M(2021)Effect of Occlusion on Deaf and Hard of Hearing Users’ Perception of Captioned Video QualityUniversal Access in Human-Computer Interaction. Access to Media, Learning and Assistive Environments10.1007/978-3-030-78095-1_16(202-220)Online publication date: 24-Jul-2021
https://dl.acm.org/doi/10.1007/978-3-030-78095-1_16
Kushalnagar RVogler C(2020)Teleconference Accessibility and Guidelines for Deaf and Hard of Hearing UsersProceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3373625.3417299(1-6)Online publication date: 26-Oct-2020
https://dl.acm.org/doi/10.1145/3373625.3417299

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Recommendations

Accessibility for Deaf and Hard of Hearing Users: Sign Language Conversational User Interfaces

SoundHapticVR: Head-Based Spatial Haptic Feedback for Accessible Sounds in Virtual Reality for Deaf and Hard of Hearing Users

Accessibility for the Deaf and Hard-of-Hearing Introduction to the Special Thematic Session