skip to main content
10.1145/3526114.3558695acmconferencesArticle/Chapter ViewAbstractPublication PagesuistConference Proceedingsconference-collections
poster

A11yBoard: Using Multimodal Input and Output to Make Digital Artboards Accessible to Blind Users

Published: 28 October 2022 Publication History

Abstract

We present A11yBoard, an interactive multimodal system that makes interpreting and authoring digital artboards, such as presentation slides or vector drawings, accessible to blind and low-vision (BLV) users. A11yBoard combines a web-based application with a mobile touch screen device such as a smartphone or tablet. The artboard is mirrored from the PC onto the touch screen, enabling spatial exploration of the artboard via touch and gesture. In addition, speech recognition and non-speech audio are used for input and output, respectively. Finally, keyboard input is used with a custom search-driven command line interface to access various commands and properties. These modalities combine into a rich, accessible system in which artboard contents, such as shapes, lines, text boxes, and images, can be interpreted, generated, and manipulated with ease. With A11yBoard, BLV users can not only consume accessible content, but create their own as well.

References

[1]
Jens Bornschein and Gerhard Weber. 2017. Digital Drawing Tools for Blind Users: A State-of-the-Art and Requirement Analysis. In Proceedings of the 10th International Conference on PErvasive Technologies Related to Assistive Environments(PETRA ’17). Association for Computing Machinery, New York, NY, USA, 21–28. https://doi.org/10.1145/3056540.3056542
[2]
L. M. Brown, S. A. Brewster, S. A. Ramloll, R. Burton, and B. Riedel. 2003. Design guidelines for audio presentation of graphs and tables. https://eprints.gla.ac.uk/3196/
[3]
William Grussenmeyer and Eelke Folmer. 2016. AudioDraw: user preferences in non-visual diagram drawing for touchscreens. In Proceedings of the 13th International Web for All Conference(W4A ’16). Association for Computing Machinery, New York, NY, USA, 1–8. https://doi.org/10.1145/2899475.2899483
[4]
Hesham M. Kamel and James A. Landay. 2000. A study of blind drawing practice: creating graphical information without the visual channel. In Proceedings of the fourth international ACM conference on Assistive technologies(Assets ’00). Association for Computing Machinery, New York, NY, USA, 34–41. https://doi.org/10.1145/354324.354334
[5]
Shaun K. Kane, Jeffrey P. Bigham, and Jacob O. Wobbrock. 2008. Slide rule: making mobile touch screens accessible to blind people using multi-touch interaction techniques. In Proceedings of the 10th international ACM SIGACCESS conference on Computers and accessibility(Assets ’08). Association for Computing Machinery, New York, NY, USA, 73–80. https://doi.org/10.1145/1414471.1414487
[6]
Shaun K. Kane, Meredith Ringel Morris, Annuska Z. Perkins, Daniel Wigdor, Richard E. Ladner, and Jacob O. Wobbrock. 2011. Access overlays: improving non-visual access to large touch screens for blind users. In Proceedings of the 24th annual ACM symposium on User interface software and technology - UIST ’11. ACM Press, Santa Barbara, California, USA, 273. https://doi.org/10.1145/2047196.2047232
[7]
Shaun K. Kane, Meredith Ringel Morris, and Jacob O. Wobbrock. 2013. Touchplates: low-cost tactile overlays for visually impaired touch screen users. In Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility(ASSETS ’13). Association for Computing Machinery, New York, NY, USA, 1–8. https://doi.org/10.1145/2513383.2513442
[8]
Martin Kurze. 1996. TDraw: a computer-based tactile drawing tool for blind people. In Proceedings of the second annual ACM conference on Assistive technologies(Assets ’96). Association for Computing Machinery, New York, NY, USA, 131–138. https://doi.org/10.1145/228347.228368
[9]
Denise Prescher, Gerhard Weber, and Martin Spindler. 2010. A tactile windowing system for blind users. In Proceedings of the 12th international ACM SIGACCESS conference on Computers and accessibility(ASSETS ’10). Association for Computing Machinery, New York, NY, USA, 91–98. https://doi.org/10.1145/1878803.1878821
[10]
Anastasia Schaadhardt, Alexis Hiniker, and Jacob O Wobbrock. 2021. Understanding Blind Screen-Reader Users’ Experiences of Digital Artboards. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–19.
[11]
Carsten Schäfer. 2022. DrawerJS: a platform-independent, web-based JavaScript WYSIWYG HTML canvas editor. https://www.drawerjs.com/
[12]
Ather Sharif and Babak Forouraghi. 2018. evoGraphs — A jQuery plugin to create web accessible graphs. In 2018 15th IEEE Annual Consumer Communications & Networking Conference (CCNC). 1–4. https://doi.org/10.1109/CCNC.2018.8319239 ISSN: 2331-9860.
[13]
Steven Wall and Stephen Brewster. 2006. Feeling what you hear: tactile feedback for navigation of audio graphs. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’06). Association for Computing Machinery, New York, NY, USA, 1123–1132. https://doi.org/10.1145/1124772.1124941
[14]
Tsubasa Yoshida, Kris M. Kitani, Hideki Koike, Serge Belongie, and Kevin Schlei. 2011. EdgeSonic: image feature sonification for the visually impaired. In Proceedings of the 2nd Augmented Human International Conference(AH ’11). Association for Computing Machinery, New York, NY, USA, 1–4. https://doi.org/10.1145/1959826.1959837

Cited By

View all
  • (2024)Speech-based Mark for Data SonificationProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3688514(1-5)Online publication date: 27-Oct-2024
  • (2024)"It's like Goldilocks:" Bespoke Slides for Fluctuating Audience Access NeedsProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675640(1-15)Online publication date: 27-Oct-2024
  • (2024)"We Musicians Know How to Divide and Conquer": Exploring Multimodal Interactions To Improve Music Reading and Memorization for Blind and Low Vision LearnersProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675604(1-14)Online publication date: 27-Oct-2024
  • Show More Cited By

Index Terms

  1. A11yBoard: Using Multimodal Input and Output to Make Digital Artboards Accessible to Blind Users

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      UIST '22 Adjunct: Adjunct Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology
      October 2022
      413 pages
      ISBN:9781450393218
      DOI:10.1145/3526114
      Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 28 October 2022

      Check for updates

      Qualifiers

      • Poster
      • Research
      • Refereed limited

      Conference

      UIST '22

      Acceptance Rates

      Overall Acceptance Rate 355 of 1,733 submissions, 20%

      Upcoming Conference

      UIST '25
      The 38th Annual ACM Symposium on User Interface Software and Technology
      September 28 - October 1, 2025
      Busan , Republic of Korea

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)77
      • Downloads (Last 6 weeks)5
      Reflects downloads up to 20 Jan 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Speech-based Mark for Data SonificationProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3688514(1-5)Online publication date: 27-Oct-2024
      • (2024)"It's like Goldilocks:" Bespoke Slides for Fluctuating Audience Access NeedsProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675640(1-15)Online publication date: 27-Oct-2024
      • (2024)"We Musicians Know How to Divide and Conquer": Exploring Multimodal Interactions To Improve Music Reading and Memorization for Blind and Low Vision LearnersProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675604(1-14)Online publication date: 27-Oct-2024
      • (2024)Designing Accessible Content Creation Support with Blind and Low Vision CreatorsACM SIGACCESS Accessibility and Computing10.1145/3654768.3654775(1-1)Online publication date: 26-Mar-2024
      • (2024)Understanding Expert Crafting Practices of Blind and Low Vision CreativesExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650960(1-8)Online publication date: 11-May-2024
      • (2024)TADA: Making Node-link Diagrams Accessible to Blind and Low-Vision PeopleProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642222(1-20)Online publication date: 11-May-2024
      • (2023)Developing and Deploying a Real-World Solution for Accessible Slide Reading and Authoring for Blind UsersProceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3597638.3608418(1-15)Online publication date: 22-Oct-2023
      • (2023)Understanding Digital Content Creation Needs of Blind and Low Vision PeopleProceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3597638.3608387(1-15)Online publication date: 22-Oct-2023

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format.

      HTML Format

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media