skip to main content
10.1145/3430263.3452428acmotherconferencesArticle/Chapter ViewAbstractPublication Pagesw4aConference Proceedingsconference-collections
research-article

Understanding deaf and hard-of-hearing users' interest in sign-language interaction with personal-assistant devices

Published: 20 May 2021 Publication History

Abstract

Proliferation of voice-controlled personal-assistant devices poses accessibility barriers for Deaf and Hard of Hearing (DHH) users. In this mixed interview (N=21) and survey (N=86) study, DHH American Sign Language (ASL) signers reported little experience with such devices yet strong interest if these devices could understand ASL commands. Beyond traditional uses, e.g. weather information, participants were interested in uses specific to the DHH experience, e.g. receiving alerts about sounds, triggering video calls, or requesting ASL interpretation. Participants gave examples of commands they would issue in these categories, where they would use these devices in their home, and how this interaction could be structured. This study contributes empirical knowledge of DHH signers' interest in ASL interaction with these devices, providing motivation and foundation for future research, including what to include in future datasets of videos of users giving commands in ASL, needed to create future sign-recognition technology.

References

[1]
Ali Abdolrahmani, Ravi Kuber, and Stacy M. Branham. 2018. "Siri Talks at You": An Empirical Investigation of Voice-Activated Personal Assistant (VAPA) Usage by Individuals Who Are Blind. In Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility (Galway, Ireland) (ASSETS '18). Association for Computing Machinery, New York, NY, USA, 249--258.
[2]
BBC. 2018. Sign-language hack lets Amazon Alexa respond to gestures. https://www.bbc.com/news/technology-44891054
[3]
Jeffrey P. Bigham, Raja Kushalnagar, Ting-Hao Kenneth Huang, Juan Pablo Flores, and Saiph Savage. 2017. On How Deaf People Might Use Speech to Control Devices. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (Baltimore, Maryland, USA) (ASSETS '17). Association for Computing Machinery, New York, NY, USA, 383--384.
[4]
Danielle Bragg, Oscar Koller, Mary Bellard, Larwan Berke, Patrick Boudreault, Annelies Braffort, Naomi Caselli, Matt Huenerfauth, Hernisa Kacorri, Tessa Verhoef, Christian Vogler, and Meredith Ringel Morris. 2019. Sign Language Recognition, Generation, and Translation: An Interdisciplinary Perspective. In The 21st International ACM SIGACCESS Conference on Computers and Accessibility (Pittsburgh, PA, USA) (ASSETS '19). Association for Computing Machinery, New York, NY, USA, 16--31.
[5]
Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative Research in Psychology 3, 2 (2006), 77--101.
[6]
Devin Coldewey. 2018. SignAll is slowly but surely building a sign language translation platform. https://techcrunch.com/2018/02/14/signall-is-slowly-but-surely-building-a-sign-language-translation-platform/
[7]
Julie Doyle, Emma Murphy, Janneke Kuiper, Suzanne Smith, Caoimhe Hannigan, An Jacobs, and John Dinsmore. 2019. Managing Multimorbidity: Identifying Design Requirements for a Digital Self-Management Tool to Support Older Adults with Multiple Chronic Conditions. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19). Association for Computing Machinery, New York, NY, USA, 1--14.
[8]
Eleni Efthimiou, Stavroula-Evita Fotinea, Thomas Hanke, Julie A. Hochgesang, Jette Kristoffersen, and Johanna Mesch. 2020. Proceedings of the 9th Workshop on the Representation and Processing of Sign Languages: Sign Language Resources in the Service of the Language Community, Technological Challenges and Application Perspectives. (2020). https://lrec2020.lrec-conf.org/media/proceedings/Workshops/Books/SIGN2020book.pdf
[9]
Leah Findlater, Bonnie Chinh, Dhruv Jain, Jon Froehlich, Raja Kushalnagar, and Angela Carey Lin. 2019. Deaf and Hard-of-Hearing Individuals' Preferences for Wearable and Mobile Sound Awareness Technologies. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19). Association for Computing Machinery, New York, NY, USA, 1--13.
[10]
Abraham Glasser. 2019. Automatic Speech Recognition Services: Deaf and Hard-of-Hearing Usability. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI EA '19). Association for Computing Machinery, New York, NY, USA, 1--6.
[11]
Abraham T. Glasser, Kesavan R. Kushalnagar, and Raja S. Kushalnagar. 2017. Feasibility of Using Automatic Speech Recognition with Voices of Deaf and Hard-of-Hearing Individuals. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (Baltimore, Maryland, USA) (ASSETS '17). Association for Computing Machinery, New York, NY, USA, 373--374.
[12]
Matthew B. Hoy. 2018. Alexa, Siri, Cortana, and More: An Introduction to Voice Assistants. Medical Reference Services Quarterly 37, 1 (2018), 81--88.
[13]
David Isbitski. 2017. How to Build Alexa Skills for Echo Show. https://developer.amazon.com/blogs/alexa/post/12826e9e-e06a-4ab4-a583-8e074709a9f3/how-to-build-alexa-skills-for-echo-show
[14]
Dhruv Jain, Angela Lin, Rose Guttman, Marcus Amalachandran, Aileen Zeng, Leah Findlater, and Jon Froehlich. 2019. Exploring Sound Awareness in the Home for People Who Are Deaf or Hard of Hearing. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19). Association for Computing Machinery, New York, NY, USA, 1--13.
[15]
Jiepu Jiang, Ahmed Hassan Awadallah, Rosie Jones, Umut Ozertem, Imed Zitouni, Ranjitha Gurunath Kulkarni, and Omar Zia Khan. 2015. Automatic Online Evaluation of Intelligent Assistants. In Proceedings of the 24th International Conference on World Wide Web (Florence, Italy) (WWW '15). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 506--516.
[16]
Irene Lopatovska, Katrina Rink, Ian Knight, Kieran Raines, Kevin Cosenza, Harriet Williams, Perachya Sorsche, David Hirsch, Qi Li, Adrianna Martinez, and et al. 2018. Talk to me: Exploring user interactions with the Amazon Alexa. Journal of Librarianship and Information Science 51, 4 (2018), 984--997.
[17]
Tara Matthews, Janette Fong, F. Wai-Ling Ho-Ching, and Jennifer Mankoff. 2006. Evaluating non-speech sound visualizations for the deaf. Behaviour & Information Technology 25, 4 (2006), 333--351.
[18]
Microsoft. 2019. AI for Accessibility Hackathon 2019. https://blogs.partner.microsoft.com/mpn-apac/ai-for-accessibility-hackathon-2019/
[19]
Christi Olson and Kelli Kemery. 2019. 2019 Microsoft Voice report. https://about.ads.microsoft.com/en-us/insights/2019-voice-report
[20]
Alisha Pradhan, Kanika Mehta, and Leah Findlater. 2018. "Accessibility Came by Accident": Use of Voice-Controlled Intelligent Personal Assistants by People with Disabilities. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI '18). Association for Computing Machinery, New York, NY, USA, 1--13.
[21]
Jason Rodolitz, Evan Gambill, Brittany Willis, Christian Vogler, and Raja Kushalnagar. 2019. Accessibility of Voice-Activated Agents for People who are Deaf or Hard of Hearing. Journal on Technology and Persons with Disabilities 7 (2019), 144--156. http://hdl.handle.net/10211.3/210397
[22]
Kaleigh Rogers. 2018. Augmented Reality App Can Translate Sign Language Into Spoken English, and Vice Versa. https://www.vice.com/en_us/article/zmgnd9/app-to-translate-sign-language
[23]
Alex Sciuto, Arnita Saini, Jodi Forlizzi, and Jason I. Hong. 2018. "Hey Alexa, What's Up?": A Mixed-Methods Studies of In-Home Conversational Agent Usage. In Proceedings of the 2018 Designing Interactive Systems Conference (Hong Kong, China) (DIS '18). Association for Computing Machinery, New York, NY, USA, 857--868.
[24]
SignGenius. 2020. Do's & Don'ts - Getting Attention in the Deaf Community. https://www.signgenius.com/info-do's&don'ts.shtml
[25]
Sorenson. 2020. Sorenson VRS. https://www.sorensonvrs.com/
[26]
Jason Ward. 2018. Microsoft's Windows and Cortana absolutely must learn sign language. https://www.windowscentral.com/microsoft-must-bring-sign-language-recognition-windows-and-cortana

Cited By

View all
  • (2024)Voice Assistant Utilization among the Disability Community for Independent Living: A Rapid Review of Recent EvidenceHuman Behavior and Emerging Technologies10.1155/2024/64949442024(1-39)Online publication date: 12-Apr-2024
  • (2024)Sign Language-Based versus Touch-Based Input for Deaf Users with Interactive Personal Assistants in Simulated Kitchen EnvironmentsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3651075(1-9)Online publication date: 11-May-2024
  • (2024)What Factors Motivate Culturally Deaf People to Want Assistive Technologies?Extended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650871(1-7)Online publication date: 11-May-2024
  • Show More Cited By

Index Terms

  1. Understanding deaf and hard-of-hearing users' interest in sign-language interaction with personal-assistant devices

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Other conferences
      W4A '21: Proceedings of the 18th International Web for All Conference
      April 2021
      224 pages
      ISBN:9781450382120
      DOI:10.1145/3430263
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 20 May 2021

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. accessibility
      2. deaf and hard of hearing
      3. personal assistants
      4. sign language

      Qualifiers

      • Research-article

      Funding Sources

      Conference

      W4A '21
      W4A '21: 18th Web for All Conference
      April 19 - 20, 2021
      Ljubljana, Slovenia

      Acceptance Rates

      Overall Acceptance Rate 171 of 371 submissions, 46%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)81
      • Downloads (Last 6 weeks)1
      Reflects downloads up to 30 Jan 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Voice Assistant Utilization among the Disability Community for Independent Living: A Rapid Review of Recent EvidenceHuman Behavior and Emerging Technologies10.1155/2024/64949442024(1-39)Online publication date: 12-Apr-2024
      • (2024)Sign Language-Based versus Touch-Based Input for Deaf Users with Interactive Personal Assistants in Simulated Kitchen EnvironmentsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3651075(1-9)Online publication date: 11-May-2024
      • (2024)What Factors Motivate Culturally Deaf People to Want Assistive Technologies?Extended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650871(1-7)Online publication date: 11-May-2024
      • (2024)Assessment of Sign Language-Based versus Touch-Based Input for Deaf Users Interacting with Intelligent Personal AssistantsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642094(1-15)Online publication date: 11-May-2024
      • (2024)Deaf and Hard of Hearing People’s Perspectives on Augmented Reality Interfaces for Improving the Accessibility of Smart SpeakersUniversal Access in Human-Computer Interaction10.1007/978-3-031-60881-0_21(334-357)Online publication date: 29-Jun-2024
      • (2023)FingerSpeller: Camera-Free Text Entry Using Smart Rings for American Sign Language Fingerspelling RecognitionProceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3597638.3614491(1-5)Online publication date: 22-Oct-2023
      • (2023)User Perceptions and Preferences for Online Surveys in American Sign Language: An Exploratory StudyProceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3597638.3608444(1-17)Online publication date: 22-Oct-2023
      • (2023)‘Auslan Alexa’: A case study of VR Wizard of Oz prototyping for requirements elicitation with Deaf participants2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)10.1109/VRW58643.2023.00298(911-912)Online publication date: Mar-2023
      • (2022)Exploring Smart Speaker User Experience for People Who StammerProceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3517428.3544823(1-10)Online publication date: 23-Oct-2022
      • (2022)Analyzing Deaf and Hard-of-Hearing Users’ Behavior, Usage, and Interaction with a Personal Assistant Device that Understands Sign-Language InputProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501987(1-12)Online publication date: 29-Apr-2022

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media