research-article

Empirical Analysis of Bias in Voice-based Personal Assistants

Authors:

Lanna Lima,

Vasco Furtado,

Elizabeth Furtado,

Virgilio AlmeidaAuthors Info & Claims

WWW '19: Companion Proceedings of The 2019 World Wide Web Conference

Pages 533 - 538

https://doi.org/10.1145/3308560.3317597

Published: 13 May 2019 Publication History

Get Access

Abstract

Voice-based assistants are becoming increasingly widespread all over the world. However, the performance of these assistants in the interaction with users of languages and accents of developing countries is not clear yet. Eventual bias against specific language or accent of different groups of people in developing countries is maybe a factor to increase the digital gap in these countries. Our research aims at analysing the presence of bias in the interaction via audio. We carried out experiments to verify the quality of the recognition of phrases spoken by different groups of people. We evaluated the behaviour of Google Assistant and Siri for groups of people formed according to gender and regions that have different accents. Preliminary results indicate that accent and mispronunciation due to regional differences are not being properly considered by the assistants we have analyzed.

References

[1]

Ricardo Baeza-Yates. 2018. Bias on the web. Commun. ACM 61, 6 (2018), 54–61.

Digital Library

Google Scholar

[2]

Jerome R Bellegarda. 2014. Spoken language understanding for natural interaction: The siri experience. In Natural Interaction with Robots, Knowbots and Smartphones. Springer, 3–14.

Google Scholar

[3]

Estado de São Paulo Newspaper. 2017. WhatsApp chega a 120 milhões de usuarios no Brasil.

Google Scholar

[4]

Adriana de Souza e Silva, Daniel M Sutko, Fernando A Salis, and Claudio de Souza e Silva. 2011. Mobile phone appropriation in the favelas of Rio de Janeiro, Brazil. New Media & Society 13, 3 (2011), 411–426.

Crossref

Google Scholar

[5]

Anthony G Greenwald. 1976. Within-subjects designs: To use or not to use?Psychological Bulletin 83, 2 (1976), 314.

Google Scholar

[6]

Reuters Institute. 2018. The Future of Voice and the Implications for News.

Google Scholar

[7]

The Intercept. 2018. Amazon’s accent recognition technology could tell the government where you’re from.

Google Scholar

[8]

Wired Magazine. 2017. Voice Is the Next Big Platform, Unless You Have an Accent.

Google Scholar

[9]

Rami Matarneh, Svitlana Maksymova, Vyacheslav Lyashenko, and N Belova. 2017. Speech recognition systems: A comparative review. (2017).

Google Scholar

[10]

Tyler K Perrachione. 2017. Speaker recognition across languages. Oxford University Press.

Google Scholar

[11]

Moirangthem Tiken Singh, Abdur Razzaq Fayjie, and Biswajeet Kachari. 2015. Speech Recognition System for North-East Indian Accent. In International Journal of Applied Information Systems (IJAIS). Vol. 9. Citeseer.

Google Scholar

[12]

Yanli Zheng, Richard Sproat, Liang Gu, Izhak Shafran, Haolang Zhou, Yi Su, Daniel Jurafsky, Rebecca Starr, and Su-Youn Yoon. 2005. Accent detection and speech recognition for shanghai-accented mandarin. In Ninth European Conference on Speech Communication and Technology.

Crossref

Google Scholar

Cited By

View all

Cibrian FChen YAnderson KAbrahamsson CMotti V(2025)Limitations in speech recognition for young adults with down syndromeUniversal Access in the Information Society10.1007/s10209-025-01197-4Online publication date: 15-Feb-2025
https://doi.org/10.1007/s10209-025-01197-4
Losada Maestre RSánchez Medero R(2024)Implementation of smart devices in health crisis scenarios: risks and opportunitiesFrontiers in Political Science10.3389/fpos.2024.15180676Online publication date: 17-Dec-2024
https://doi.org/10.3389/fpos.2024.1518067
Pathiyan Cherumanal SScholer FTrippas JSpina D(2024)Towards Investigating Biases in Spoken Conversational SearchCompanion Proceedings of the 26th International Conference on Multimodal Interaction10.1145/3686215.3690156(61-66)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3686215.3690156
Show More Cited By

Index Terms

Empirical Analysis of Bias in Voice-based Personal Assistants

Index terms have been assigned to the content through auto-classification.

Recommendations

Mixed-initiative Personal Assistants
SIGCSE '17: Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education

Specification and implementation of flexible human-computer dialogs is challenging because of the complexity involved in rendering the dialog responsive to a vast number of varied paths through which users might desire to complete the dialog. To address ...
Bias in Automated Speaker Recognition
FAccT '22: Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency

Automated speaker recognition uses data processing to identify speakers by their voice. Today, automated speaker recognition is deployed on billions of smart devices and in services such as call centres. Despite their wide-scale deployment and known ...
Understanding the Adoption of Voice Activated Personal Assistants

This study aims to investigate the factors that affect the usage of voice-activated personal assistants VAPA which are mobile device applications such as Siri, Google Now, S Voice, Cortana, Alexa, etc. A theoretical framework is proposed based on the ...

Comments

Information & Contributors

Information

Published In

WWW '19: Companion Proceedings of The 2019 World Wide Web Conference

May 2019

1331 pages

ISBN:9781450366755

DOI:10.1145/3308560

Editors:
Ling Liu
Georgia Tech, USA
,
Ryen White
Microsoft Research, USA

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

IW3C2: International World Wide Web Conference Committee

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 May 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

WWW '19

WWW '19: The Web Conference

May 13 - 17, 2019

San Francisco, USA

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

25
Total Citations
View Citations
634
Total Downloads

Downloads (Last 12 months)84
Downloads (Last 6 weeks)8

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Cibrian FChen YAnderson KAbrahamsson CMotti V(2025)Limitations in speech recognition for young adults with down syndromeUniversal Access in the Information Society10.1007/s10209-025-01197-4Online publication date: 15-Feb-2025
https://doi.org/10.1007/s10209-025-01197-4
Losada Maestre RSánchez Medero R(2024)Implementation of smart devices in health crisis scenarios: risks and opportunitiesFrontiers in Political Science10.3389/fpos.2024.15180676Online publication date: 17-Dec-2024
https://doi.org/10.3389/fpos.2024.1518067
Pathiyan Cherumanal SScholer FTrippas JSpina D(2024)Towards Investigating Biases in Spoken Conversational SearchCompanion Proceedings of the 26th International Conference on Multimodal Interaction10.1145/3686215.3690156(61-66)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3686215.3690156
Decker TKoebler ALebacher MThon ITresp VBuettner FBaeza-Yates RBonchi F(2024)Explanatory Model Monitoring to Understand the Effects of Feature Shifts on PerformanceProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671959(550-561)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671959
Wenzel KKaufman G(2024)Designing for Harm Reduction: Communication Repair for Multicultural Users' Voice InteractionsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642900(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642900
Leschanowsky AGaznepoglu ÜPeters N(2024)Voice Anonymization for All-Bias Evaluation of the Voice Privacy Challenge Baseline SystemsICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP48485.2024.10447137(4785-4789)Online publication date: 14-Apr-2024
https://doi.org/10.1109/ICASSP48485.2024.10447137
Bergman SMarchal NMellor JMohamed SGabriel IIsaac W(2024)STELA: a community-centred approach to norm elicitation for AI alignmentScientific Reports10.1038/s41598-024-56648-414:1Online publication date: 19-Mar-2024
https://doi.org/10.1038/s41598-024-56648-4
Furtado EOliveira FPinheiro V(2024)Konversationelle Assistenten und ihre Anwendungen in Gesundheit und NephrologieInnovationen in der Nephrologie10.1007/978-3-031-65236-3_18(317-340)Online publication date: 27-Oct-2024
https://doi.org/10.1007/978-3-031-65236-3_18
Harrington L(2023)Incorporating automatic speech recognition methods into the transcription of police-suspect interviews: factors affecting automatic performanceFrontiers in Communication10.3389/fcomm.2023.11652338Online publication date: 13-Jul-2023
https://doi.org/10.3389/fcomm.2023.1165233
Siegle RPuchalski JCraig S(2023)Learning from accented virtual humans: Can quality of voice and globalization overcome accent familiarity?Proceedings of the Human Factors and Ergonomics Society Annual Meeting10.1177/2169506723119241767:1(1811-1816)Online publication date: 20-Nov-2023
https://doi.org/10.1177/21695067231192417
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Abstract

References

Cited By

Index Terms

Recommendations

Mixed-initiative Personal Assistants

Bias in Automated Speaker Recognition

Understanding the Adoption of Voice Activated Personal Assistants

Comments

Information

Published In

In-Cooperation

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

HTML Format

Share

Share this Publication link

Share on social media

Affiliations