skip to main content
10.1145/3439231.3440603acmotherconferencesArticle/Chapter ViewAbstractPublication PagesdsaiConference Proceedingsconference-collections
research-article

Apple Siri (input) + Voice Over (output) = a de facto marriage: An exploratory case study with blind people

Published: 09 June 2021 Publication History

Abstract

People who are blind or have severe low vision (BLVP) often rely on synthesized voice (output) to interact with computers. Thanks to Voice Assistants (VAs), BLVP can now use voice commands to interact (input) with a range of devices. Yet, very little is known about how they use VAs. This exploratory paper reports on semi-structured, face-to-face interviews with (N=10) legally blind adults, including typhlotechnicians, who teach other BLVP to use digital technologies and may themselves be blind people. Whilst the current impact of VAs on our everyday lives focuses on aiding in the completion of simple day-to-day activities, the results show that the ‘couple’ Apple Siri and Voice Over has a strong, positive impact on the everyday lives of our participants. They reported using VAs mostly as a tool, not as a social actor, and that productivity was more important for them than privacy in their everyday use of Siri. Implications for design and research are outlined.

References

[1]
[n.d.]. The Principles of Service Design Thinking Building Better Services. https://www.interactiondesign.org/literature/article/the-principles-of-service-design-thinking-buildingbetter-services
[2]
Ali Abdolrahmani, Ravi Kuber, and Stacy M. Branham. 2018. ”Siri Talks at You”: An Empirical Investigation of Voice-Activated Personal Assistant (VAPA) Usage by Individuals Who Are Blind. In Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility (Galway, Ireland) (ASSETS ’18). Association for Computing Machinery, New York, NY, USA, 249–258. https://doi.org/10.1145/3234695.3236344
[3]
Ali Abdolrahmani, Kevin M. Storer, Antony Rishin Mukkath Roy, Ravi Kuber, and Stacy M. Branham. 2020. Blind Leading the Sighted: Drawing Design Insights from Blind Users towards More Productivity-Oriented Voice Interfaces. ACM Trans. Access. Comput. 12, 4, Article 18 (Jan. 2020), 35 pages. https://doi.org/10.1145/3368426
[4]
Erin Beneteau, Ashley Boone, Yuxing Wu, Julie A. Kientz, Jason Yip, and Alexis Hiniker. 2020. Parenting with Alexa: Exploring the Introduction of Smart Speakers on Family Dynamics. Association for Computing Machinery, New York, NY, USA, 1–13. https://doi-org.sire.ub.edu/10.1145/3313831.3376344
[5]
Erin Beneteau, Olivia K. Richards, Mingrui Zhang, Julie A. Kientz, Jason Yip, and Alexis Hiniker. 2019. Communication Breakdowns Between Families and Alexa. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3290605.3300473
[6]
Stacy M. Branham and Antony Rishin Mukkath Roy. 2019. Reading Between the Guidelines: How Commercial Voice Assistant Guidelines Hinder Accessibility for Blind Users. In The 21st International ACM SIGACCESS Conference on Computers and Accessibility (Pittsburgh, PA, USA) (ASSETS ’19). Association for Computing Machinery, New York, NY, USA, 446–458. https://doi.org/10.1145/3308561.3353797
[7]
Virginia Braun and Victoria Clarke. 2019. Reflecting on reflexive thematic analysis. Qualitative Research in Sport, Exercise and Health 11, 4 (2019), 589–597. https://doi.org/10.1080/2159676X.2019.1628806 arXiv:https://doi.org/10.1080/2159676X.2019.1628806
[8]
Heloisa Candello, Cosmin Munteanu, Leigh Clark, Jaisie Sin, María Inés Torres, Martin Porcheron, Chelsea M. Myers, Benjamin Cowan, Joel Fischer, Stephan Schlögl, Christine Murad, and Stuart Reeves. 2020. CUI@CHI: Mapping Grand Challenges for the Conversational User Interface Community. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI EA ’20). Association for Computing Machinery, New York, NY, USA, 1–8. https://doi.org/10.1145/3334480.3375152
[9]
Heloisa Candello, Claudio Pinhanez, Mauro Pichiliani, Paulo Cavalin, Flavio Figueiredo, Marisa Vasconcelos, and Haylla Do Carmo. 2019. The Effect of Audiences on the User Experience with Conversational Interfaces in Physical Spaces. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3290605.3300320
[10]
Hyunji Chung, Michaela Iorga, Jeffrey Voas, and Sangjin Lee. 2017. “Alexa, Can I Trust You?”. Computer 50, 9 (Sep 2017), 100–104. https://doi.org/10.1109/mc.2017.3571053
[11]
Leigh Clark, Nadia Pantidi, Orla Cooney, Philip Doyle, Diego Garaialde, Justin Edwards, Brendan Spillane, Emer Gilmartin, Christine Murad, Cosmin Munteanu, Vincent Wade, and Benjamin R. Cowan. 2019. What Makes a Good Conversation? Challenges in Designing Truly Conversational Agents. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3290605.3300705
[12]
Peter Conradie, Lieven De Couvreur, Jelle Saldien, and Lieven De Marez. 2014. Disabled Users as Lead Users in Product Innovation: A Literature Overview. DS 81: Proceedings of NordDesign 2014, Espoo, Finland 27-29th August 2014 (2014).
[13]
Stefania Druga, Randi Williams, Cynthia Breazeal, and Mitchel Resnick. 2017. ”Hey Google is It OK If I Eat You?”: Initial Explorations in Child-Agent Interaction. In Proceedings of the 2017 Conference on Interaction Design and Children (Stanford, California, USA) (IDC ’17). Association for Computing Machinery, New York, NY, USA, 595–600. https://doi.org/10.1145/3078072.3084330
[14]
Nathaniel Fruchter and Ilaria Liccardi. 2018. Consumer Attitudes Towards Privacy and Security in Home Assistants. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI EA ’18). Association for Computing Machinery, New York, NY, USA, 1–6. https://doi.org/10.1145/3170427.3188448
[15]
Sangyeal Han and Heetae Yang. 2018. Understanding adoption of intelligent personal assistants. Industrial Management & Data Systems(2018). https://doi.org/10.1108/IMDS-05-2017-0214
[16]
Arne Holst. 2021. Number of voice assistants in use worldwide 2019-2024. https://www.statista.com/statistics/973815/worldwide-digital-voice-assistant-in-use/
[17]
Yi-Chieh Lee, Naomi Yamashita, Yun Huang, and Wai Fu. 2020. ”I Hear You, I Feel You”: Encouraging Deep Self-Disclosure through a Chatbot. Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3313831.3376175
[18]
Irene Lopatovska and Harriet Williams. 2018. Personification of the Amazon Alexa: BFF or a Mindless Companion. In Proceedings of the 2018 Conference on Human Information Interaction and Retrieval (New Brunswick, NJ, USA) (CHIIR ’18). Association for Computing Machinery, New York, NY, USA, 265–268. https://doi.org/10.1145/3176349.3176868
[19]
Ewa Luger and Abigail Sellen. 2016. ”Like Having a Really Bad PA”: The Gulf between User Expectation and Experience of Conversational Agents. Association for Computing Machinery, New York, NY, USA, 5286–5297. https://doi.org/10.1145/2858036.2858288
[20]
Aqueasha Martin-Hammond, Sravani Vemireddy, and Kartik Rao. 2019. Exploring Older Adults’ Beliefs About the Use of Intelligent Assistants for Consumer Health Information Management: A Participatory Design Study. Jmir Aging 2, 2 (2019), e15381.
[21]
Christine Murad, Cosmin Munteanu, Leigh Clark, and Benjamin R. Cowan. 2018. Design Guidelines for Hands-Free Speech Interaction. In Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct (Barcelona, Spain) (MobileHCI ’18). Association for Computing Machinery, New York, NY, USA, 269–276. https://doi.org/10.1145/3236112.3236149
[22]
Martin Porcheron, Joel E. Fischer, Moira McGregor, Barry Brown, Ewa Luger, Heloisa Candello, and Kenton O’Hara. 2017. Talking with Conversational Agents in Collaborative Action. In Companion of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (Portland, Oregon, USA) (CSCW ’17 Companion). Association for Computing Machinery, New York, NY, USA, 431–436. https://doi.org/10.1145/3022198.3022666
[23]
Alisha Pradhan, Leah Findlater, and Amanda Lazar. 2019. ”Phantom Friend” or ”Just a Box with Information”: Personification and Ontological Categorization of Smart Speaker-Based Voice Assistants by Older Adults. Proc. ACM Hum.-Comput. Interact. 3, CSCW, Article 214 (Nov. 2019), 21 pages. https://doi.org/10.1145/3359316
[24]
Alisha Pradhan, Kanika Mehta, and Leah Findlater. 2018. ”Accessibility Came by Accident”: Use of Voice-Controlled Intelligent Personal Assistants by People with Disabilities. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3173574.3174033
[25]
Amanda Purington, Jessie G. Taft, Shruti Sannon, Natalya N. Bazarova, and Samuel Hardman Taylor. 2017. ”Alexa is My New BFF”: Social Roles, User Satisfaction, and Personification of the Amazon Echo. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI EA ’17). Association for Computing Machinery, New York, NY, USA, 2853–2859. https://doi.org/10.1145/3027063.3053246
[26]
Aung Pyae and Paul Scifleet. 2019. Investigating the Role of User’s English Language Proficiency in Using a Voice User Interface: A Case of Google Home Smart Speaker. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI EA ’19). Association for Computing Machinery, New York, NY, USA, 1–6. https://doi.org/10.1145/3290607.3313038
[27]
Byron Reeves and Clifford Nass. 2006. The media equation: how people treat computers, television, and new media like real people and places. CSLI Publications.
[28]
Sergio Sayago. 2020. MobileHCI: Voice Assistants as Learning Companions: An Initial Exploration With Computer Science Students. In 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services (Oldenburg, Germany). Association for Computing Machinery, New York, NY, USA, 3 pages. https://doi.org/10.1145/3406324.3410707
[29]
Alex Sciuto, Arnita Saini, Jodi Forlizzi, and Jason I. Hong. 2018. ”Hey Alexa, What’s Up?”: A Mixed-Methods Studies of In-Home Conversational Agent Usage. In Proceedings of the 2018 Designing Interactive Systems Conference (Hong Kong, China) (DIS ’18). Association for Computing Machinery, New York, NY, USA, 857–868. https://doi.org/10.1145/3196709.3196772
[30]
Yang Shi, Xin Yan, Xiaojuan Ma, Yongqi Lou, and Nan Cao. 2018. Designing Emotional Expressions of Conversational States for Voice Assistants: Modality and Engagement. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI EA ’18). Association for Computing Machinery, New York, NY, USA, 1–6. https://doi.org/10.1145/3170427.3188560
[31]
Kevin M. Storer, Tejinder K. Judge, and Stacy M. Branham. 2020. ”All in the Same Boat”: Tradeoffs of Voice Assistant Ownership for Mixed-Visual-Ability Families. Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3313831.3376225
[32]
Richard S. Sutton and Andrew G. Barto. 2018. Reinforcement learning: an introduction. The MIT Press.
[33]
Alexandra Vtyurina, Adam Fourney, Meredith Ringel Morris, Leah Findlater, and Ryen W. White. 2019. VERSE: Bridging Screen Readers and Voice Assistants for Enhanced Eyes-Free Web Search. In The 21st International ACM SIGACCESS Conference on Computers and Accessibility(Pittsburgh, PA, USA) (ASSETS ’19). Association for Computing Machinery, New York, NY, USA, 414–426. https://doi.org/10.1145/3308561.3353773

Cited By

View all
  • (2025)A Comprehensive Survey of Retrieval-Augmented Large Language Models for Decision Making in Agriculture: Unsolved Problems and Research OpportunitiesJournal of Artificial Intelligence and Soft Computing Research10.2478/jaiscr-2025-000715:2(115-146)Online publication date: 5-Feb-2025
  • (2025)Breaking down barriersInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2024.103403194:COnline publication date: 1-Feb-2025
  • (2024)Enhancing Productivity Applications for People who are Blind using AI AssistantsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3638180(1-6)Online publication date: 11-May-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
DSAI '20: Proceedings of the 9th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion
December 2020
245 pages
ISBN:9781450389372
DOI:10.1145/3439231
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 June 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. datasets
  2. gaze detection
  3. neural networks
  4. text tagging

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

  • MINECO Grant
  • MINECO/FEDER

Conference

DSAI 2020

Acceptance Rates

Overall Acceptance Rate 17 of 23 submissions, 74%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)37
  • Downloads (Last 6 weeks)3
Reflects downloads up to 13 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)A Comprehensive Survey of Retrieval-Augmented Large Language Models for Decision Making in Agriculture: Unsolved Problems and Research OpportunitiesJournal of Artificial Intelligence and Soft Computing Research10.2478/jaiscr-2025-000715:2(115-146)Online publication date: 5-Feb-2025
  • (2025)Breaking down barriersInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2024.103403194:COnline publication date: 1-Feb-2025
  • (2024)Enhancing Productivity Applications for People who are Blind using AI AssistantsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3638180(1-6)Online publication date: 11-May-2024
  • (2023)A Systematic Review of Ethical Concerns with Voice AssistantsProceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society10.1145/3600211.3604679(131-145)Online publication date: 8-Aug-2023
  • (2023)Who are CUIs Really For? Representation and Accessibility in the Conversational User Interface LiteratureProceedings of the 5th International Conference on Conversational User Interfaces10.1145/3571884.3603760(1-5)Online publication date: 19-Jul-2023

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media