Deploying VizLens: Characterizing User Needs, Preferences, and Challenges of Physical Interfaces Usage in the Wild
Article No.: 89, Pages 1 - 4
Abstract
Blind or Visually Impaired (BVI) people often encounter flat, inaccessible interfaces. Current solutions lack cost-effectiveness, portability, and robustness in real-world settings. We introduce VizLens, a fully-automated, full-stack mobile application powered by computer vision algorithms. The system is deployed and publicly available through the Apple App Store (https://vizlens.org/). From May to August 2023, we had 665 users, who uploaded 1,320 interface images. We aim to use it to study usage patterns and possible challenges BVI users may encounter with flat interfaces through a large-scale study in real-world settings. With in-depth analysis of user data and activity logs, our study will provide insights into BVI users’ interface interests, preferred assistance modes, and potential challenges due to system limitations or users’ diverse abilities. Our goal is to enhance the understanding of how BVI users interact with inaccessible, flat interfaces, and inform future assistive technology design.
References
[1]
Herbert Bay, Tinne Tuytelaars, and Luc Van Gool. 2006. SURF: Speeded Up Robust Features. In Computer Vision – ECCV 2006, Aleš Leonardis, Horst Bischof, and Axel Pinz (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 404–417.
[2]
Google Cloud. 2023. Google Cloud Vision. https://cloud.google.com/vision
[3]
Aira Tech Corp. 2022. Aira. https://aira.io/
[4]
Be My Eyes. 2023. Be My Eyes. https://www.bemyeyes.com/
[5]
Giovanni Fusco, Ender Tekin, R.E. Ladner, and James Coughlan. 2014. Using Computer Vision to Access Appliance Displays. ASSETS / Association for Computing Machinery. ACM Conference on Assistive Technologies 2014. https://doi.org/10.1145/2661334.2661404
[6]
Anhong Guo, Xiang ‘Anthony’ Chen, Haoran Qi, Samuel White, Suman Ghosh, Chieko Asakawa, and Jeffrey P. Bigham. 2016. VizLens: A Robust and Interactive Screen Reader for Interfaces in the Real World. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (Tokyo, Japan) (UIST ’16). Association for Computing Machinery, New York, NY, USA, 651–664. https://doi.org/10.1145/2984511.2984518
[7]
Anhong Guo, Jeeeun Kim, Xiang ‘Anthony’ Chen, Tom Yeh, Scott E. Hudson, Jennifer Mankoff, and Jeffrey P. Bigham. 2017. Facade: Auto-Generating Tactile Interfaces to Appliances. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’17). Association for Computing Machinery, New York, NY, USA, 5826–5838. https://doi.org/10.1145/3025453.3025845
[8]
Anhong Guo, Junhan Kong, Michael Rivera, Frank F. Xu, and Jeffrey P. Bigham. 2019. StateLens: A Reverse Engineering Solution for Making Existing Dynamic Touchscreens Accessible. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology (New Orleans, LA, USA) (UIST ’19). Association for Computing Machinery, New York, NY, USA, 371–385. https://doi.org/10.1145/3332165.3347873
[9]
Apple Machine Learning. 2023. Apple Machine Learning Vision. https://developer.apple.com/documentation/vision/
[10]
Chen Liang, Yasha Iravantchi, Thomas Krolikowski, Ruijie Geng, Alanson P. Sample, and Anhong Guo. 2023. BrushLens: Hardware Interaction Proxies for Accessible Touchscreen Interface Actuation. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (San Francisco, CA, USA) (UIST ’23). Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3586183.3606730
[11]
Tim Morris, Paul Blenkhorn, Luke Crossey, Quang Ngo, Martin Ross, David Werner, and Christina Wong. 2006. Clearspeech: A Display Reader for the Visually Handicapped. IEEE Transactions on Neural Systems and Rehabilitation Engineering 14, 4 (2006), 492–500. https://doi.org/10.1109/TNSRE.2006.881538
[12]
Vladimir Vezhnevets, Vassili Sazonov, and Alla Andreeva. 2004. A Survey on Pixel-Based Skin Color Detection Techniques. (03 2004).
Index Terms
- Deploying VizLens: Characterizing User Needs, Preferences, and Challenges of Physical Interfaces Usage in the Wild
Recommendations
VizLens: A Robust and Interactive Screen Reader for Interfaces in the Real World
UIST '16: Proceedings of the 29th Annual Symposium on User Interface Software and TechnologyThe world is full of physical interfaces that are inaccessible to blind people, from microwaves and information kiosks to thermostats and checkout terminals. Blind people cannot independently use such devices without at least first learning their layout,...
Making Real-World Interfaces Accessible Through Crowdsourcing, Computer Vision, and Fabrication
W4A '17: Proceedings of the 14th International Web for All ConferenceThe world is full of physical interfaces that are inaccessible to blind people, from microwaves and information kiosks to thermostats and checkout terminals. Blind people cannot independently use such devices without at least first learning their layout,...
User friendly system for the visually impaired in learning Al-Quran
This study presents a method to enable the visually impaired Muslim to learn and read the Al-Quran using Braille Display with software help. The system reads the database which contains all verses of Al-Quran and user will need to select the verse and ...
Comments
Information & Contributors
Information
Published In
October 2023
1163 pages
ISBN:9798400702204
DOI:10.1145/3597638
Copyright © 2023 Owner/Author.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 22 October 2023
Check for updates
Author Tags
Qualifiers
- Poster
- Research
- Refereed limited
Conference
ASSETS '23
Sponsor:
ASSETS '23: The 25th International ACM SIGACCESS Conference on Computers and Accessibility
October 22 - 25, 2023
NY, New York, USA
Acceptance Rates
ASSETS '23 Paper Acceptance Rate 55 of 182 submissions, 30%;
Overall Acceptance Rate 436 of 1,556 submissions, 28%
Upcoming Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 98Total Downloads
- Downloads (Last 12 months)48
- Downloads (Last 6 weeks)7
Reflects downloads up to 05 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format