skip to main content
10.1145/3517428.3550360acmconferencesArticle/Chapter ViewAbstractPublication PagesassetsConference Proceedingsconference-collections
poster

“What Makes Sonification User-Friendly?” Exploring Usability and User-Friendliness of Sonified Responses

Published: 22 October 2022 Publication History

Abstract

Sonification is a commonly used technique to make online data visualizations accessible to screen-reader users through auditory means. While current sonification solutions provide plausible utility (usefulness) to screen-reader users in exploring data visualizations, they are limited in exploring the quality (usability) of the sonified responses. In this preliminary exploration, we investigated the usability and user-friendliness of data visualization sonification for screen-reader users. Specifically, we evaluated the Pleasantness, Clarity, Confidence, and Overall Score of discrete and continuous sonified responses generated using various oscillator waveforms and synthesizers through user studies with 10 screen-reader users. Additionally, we examined these factors using both simple and complex trends. Our results show that screen-reader users preferred distinct non-continuous responses generated using oscillators with square waveforms. We utilized our findings to extend the functionality of Sonifier—an open-source JavaScript library that enables developers to sonify online data visualizations. Our follow-up interviews with screen-reader users identified the need to personalize the sonified responses per their individualized preferences.

References

[1]
Dragan Ahmetovic, Federico Avanzini, Adriano Baratè, Cristian Bernareggi, Gabriele Galimberti, Luca A Ludovico, Sergio Mascetti, and Giorgio Presti. 2019. Sonification of rotation instructions to support navigation of people with visual impairment. In 2019 IEEE International Conference on Pervasive Computing and Communications (PerCom. IEEE, 1–10.
[2]
Dragan Ahmetovic, Cristian Bernareggi, João Guerreiro, Sergio Mascetti, and Anna Capietto. 2019. Audiofunctions. web: Multimodal exploration of mathematical function graphs. In Proceedings of the 16th International Web for All Conference. 1–10.
[3]
Apple. n.d. Audio Graphs | Apple Developer Documentation. https://developer.apple.com/documentation/accessibility/audio_graphs. (Accessed on 08/01/2021).
[4]
Michael Bostock, Vadim Ogievetsky, and Jeffrey Heer. 2011. D3 data-driven documents. IEEE transactions on visualization and computer graphics 17, 12(2011), 2301–2309.
[5]
Nils Dahlbäck, Arne Jönsson, and Lars Ahrenberg. 1993. Wizard of Oz studies—why and how. Knowledge-based systems 6, 4 (1993), 258–266.
[6]
Joel J Davis. 2002. Disenfranchising the Disabled: The Inaccessibility of Internet-Based Health Information. Journal of Health Communication 7, 4 (2002), 355–367. https://doi.org/10.1080/10810730290001701
[7]
Danyang Fan, Alexa Fay Siu, Wing-Sum Adrienne Law, Raymond Ruihong Zhen, Sile O’Modhrain, and Sean Follmer. 2022. Slide-Tone and Tilt-Tone: 1-DOF Haptic Techniques for Conveying Shape Characteristics of Graphs to Blind Users. In CHI Conference on Human Factors in Computing Systems. 1–19.
[8]
Connor Geddes, David R Flatla, Garreth W Tigwell, and Roshan L Peiris. 2022. Improving Colour Patterns to Assist People with Colour Vision Deficiency. In CHI Conference on Human Factors in Computing Systems. 1–17.
[9]
Andrea Gerino, Lorenzo Picinali, Cristian Bernareggi, Nicolò Alabastro, and Sergio Mascetti. 2015. Towards large scale evaluation of novel sonification techniques for non visual shape exploration. In Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility. 13–21.
[10]
Nicholas A. Giudice, Hari Prasath Palani, Eric Brenner, and Kevin M. Kramer. 2012. Learning Non-Visual Graphical Information Using a Touch-Based Vibro-Audio Interface. In Proceedings of the 14th International ACM SIGACCESS Conference on Computers and Accessibility (Boulder, Colorado, USA) (ASSETS ’12). Association for Computing Machinery, New York, NY, USA, 103–110. https://doi.org/10.1145/2384916.2384935
[11]
Pedro Antonio Gutiérrez, Maria Perez-Ortiz, Javier Sanchez-Monedero, Francisco Fernandez-Navarro, and Cesar Hervas-Martinez. 2015. Ordinal regression methods: survey and experimental study. IEEE Transactions on Knowledge and Data Engineering 28, 1(2015), 127–146.
[12]
Melita Hajdinjak and France Mihelic. 2004. Conducting the Wizard-of-Oz Experiment.Informatica (Slovenia) 28, 4 (2004), 425–429.
[13]
Highcharts. n.d. Sonification | Highcharts. https://www.highcharts.com/docs/accessibility/sonification. (Accessed on 08/01/2021).
[14]
Leona M Holloway, Cagatay Goncu, Alon Ilsar, Matthew Butler, and Kim Marriott. 2022. Infosonics: Accessible Infographics for People who are Blind using Sonification and Voice. In CHI Conference on Human Factors in Computing Systems. 1–13.
[15]
Edward Kim and Kathleen F McCoy. 2018. Multimodal deep learning using images and text for information graphic classification. In Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility. 143–148.
[16]
Bongshin Lee, Arjun Srinivasan, Petra Isenberg, John Stasko, 2021. Post-WIMP Interaction for Information Visualization. Foundations and Trends® in Human-Computer Interaction 14, 1(2021), 1–95.
[17]
Yotam Mann. n.d. Tone.js. https://tonejs.github.io/. (Accessed on 08/02/2021).
[18]
Kim Marriott, Bongshin Lee, Matthew Butler, Ed Cutrell, Kirsten Ellis, Cagatay Goncu, Marti Hearst, Kathleen McCoy, and Danielle Albers Szafir. 2021. Inclusive data visualization for people with disabilities: a call to action. Interactions 28, 3 (2021), 47–51.
[19]
David K. McGookin and Stephen A. Brewster. 2006. SoundBar: Exploiting Multiple Views in Multimodal Graph Browsing. In Proceedings of the 4th Nordic Conference on Human-Computer Interaction: Changing Roles (Oslo, Norway) (NordiCHI ’06). Association for Computing Machinery, New York, NY, USA, 145–154. https://doi.org/10.1145/1182475.1182491
[20]
Giorgio Presti, Dragan Ahmetovic, Mattia Ducci, Cristian Bernareggi, Luca A. Ludovico, Adriano Baratè, Federico Avanzini, and Sergio Mascetti. 2021. Iterative Design of Sonification Techniques to Support People with Visual Impairments in Obstacle Avoidance. ACM Trans. Access. Comput. 14, 4, Article 19 (oct 2021), 27 pages. https://doi.org/10.1145/3470649
[21]
Ather Sharif, Sanjana Shivani Chintalapati, Jacob O Wobbrock, and Katharina Reinecke. 2021. Understanding Screen-Reader Users’ Experiences with Online Data Visualizations. In The 23rd International ACM SIGACCESS Conference on Computers and Accessibility. 1–16.
[22]
Ather Sharif and Babak Forouraghi. 2018. evoGraphs — A jQuery plugin to create web accessible graphs. In 2018 15th IEEE Annual Consumer Communications Networking Conference (CCNC). IEEE, Las Vegas, NV, USA, 1–4. https://doi.org/10.1109/CCNC.2018.8319239
[23]
Ather Sharif, Olivia H. Wang, Alida T. Muongchan, Katharina Reinecke, and Jacob O. Wobbrock. 2022. Sonifier: JavaScript library that converts a two-dimensional data into a sonified response. https://github.com/athersharif/sonifier. (Accessed on 06/12/2022).
[24]
Ather Sharif, Olivia H. Wang, Alida T. Muongchan, Katharina Reinecke, and Jacob O. Wobbrock. 2022. VoxLens: Making Online Data Visualizations Accessible with an Interactive JavaScript Plug-In. In CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 478, 19 pages. https://doi.org/10.1145/3491102.3517431
[25]
Lei Shi, Idan Zelzer, Catherine Feng, and Shiri Azenkot. 2016. Tickers and Talker: An Accessible Labeling Toolkit for 3D Printed Models. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 4896–4907. https://doi.org/10.1145/2858036.2858507
[26]
Alexa Siu, Gene SH Kim, Sile O’Modhrain, and Sean Follmer. 2022. Supporting Accessible Data Visualization Through Audio Data Narratives. In CHI Conference on Human Factors in Computing Systems. 1–19.
[27]
Ruobin Wang, Crescentia Jung, and Yea-Seul Kim. 2022. Seeing Through Sounds: Mapping Auditory Dimensions to Data and Charts for People with Visual Impairments. In The 24th EG/VGTC Conference on Visualization (EuroVis’ 22), Rome, Italy, 13-17 June, 2022. Eurographics-European Association for Computer Graphics.
[28]
Christopher Winship and Robert D Mare. 1984. Regression models with ordinal variables. American sociological review(1984), 512–525.
[29]
Haixia Zhao, Catherine Plaisant, Ben Shneiderman, and Jonathan Lazar. 2008. Data sonification for users with visual impairment: a case study with georeferenced data. ACM Transactions on Computer-Human Interaction (TOCHI) 15, 1(2008), 1–28.

Cited By

View all
  • (2024)Grouping Effect for Bar Graph Summarization for People with Visual ImpairmentsProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3688534(1-6)Online publication date: 27-Oct-2024
  • (2024)Understanding the Visualization and Analytics Needs of Blind and Low-Vision ProfessionalsProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3688496(1-5)Online publication date: 27-Oct-2024
  • (2024)Understanding and Reducing the Challenges Faced by Creators of Accessible Online Data VisualizationsProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675625(1-20)Online publication date: 27-Oct-2024
  • Show More Cited By

Index Terms

  1. “What Makes Sonification User-Friendly?” Exploring Usability and User-Friendliness of Sonified Responses

        Recommendations

        Comments

        Information & Contributors

        Information

        Published In

        cover image ACM Conferences
        ASSETS '22: Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility
        October 2022
        902 pages
        ISBN:9781450392587
        DOI:10.1145/3517428
        Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

        Sponsors

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 22 October 2022

        Check for updates

        Author Tags

        1. audio graphs
        2. screen-reader users
        3. sonification
        4. visualizations
        5. waveforms

        Qualifiers

        • Poster
        • Research
        • Refereed limited

        Conference

        ASSETS '22
        Sponsor:

        Acceptance Rates

        ASSETS '22 Paper Acceptance Rate 35 of 132 submissions, 27%;
        Overall Acceptance Rate 436 of 1,556 submissions, 28%

        Upcoming Conference

        ASSETS '25

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • Downloads (Last 12 months)107
        • Downloads (Last 6 weeks)7
        Reflects downloads up to 14 Feb 2025

        Other Metrics

        Citations

        Cited By

        View all
        • (2024)Grouping Effect for Bar Graph Summarization for People with Visual ImpairmentsProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3688534(1-6)Online publication date: 27-Oct-2024
        • (2024)Understanding the Visualization and Analytics Needs of Blind and Low-Vision ProfessionalsProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3688496(1-5)Online publication date: 27-Oct-2024
        • (2024)Understanding and Reducing the Challenges Faced by Creators of Accessible Online Data VisualizationsProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675625(1-20)Online publication date: 27-Oct-2024
        • (2024)ChartA11y: Designing Accessible Touch Experiences of Visualizations with Blind Smartphone UsersProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675611(1-15)Online publication date: 27-Oct-2024
        • (2024)Charting Beyond Sight with DataStory: Sensory Substitution and Storytelling in Visual Literacy Education for Visually Impaired ChildrenExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650800(1-8)Online publication date: 11-May-2024
        • (2024)Erie: A Declarative Grammar for Data SonificationProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642442(1-19)Online publication date: 11-May-2024
        • (2024)AutoVizuA11y: A Tool to Automate Screen Reader Accessibility in ChartsComputer Graphics Forum10.1111/cgf.1509943:3Online publication date: 10-Jun-2024
        • (2024)Beyond Vision Impairments: Redefining the Scope of Accessible Data RepresentationsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.335656630:12(7619-7636)Online publication date: 1-Dec-2024
        • (2023)ShapeSonic: Sonifying Fingertip Interactions for Non-Visual Virtual Shape PerceptionSIGGRAPH Asia 2023 Conference Papers10.1145/3610548.3618246(1-9)Online publication date: 10-Dec-2023
        • (2023)Conveying Uncertainty in Data Visualizations to Screen-Reader Users Through Non-Visual MeansProceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3597638.3614502(1-6)Online publication date: 22-Oct-2023
        • Show More Cited By

        View Options

        Login options

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        HTML Format

        View this article in HTML Format.

        HTML Format

        Figures

        Tables

        Media

        Share

        Share

        Share this Publication link

        Share on social media