Md Touhidul Islam
Donald E Porter
Syed Masum Billah
ABSTRACT
Perceived accessibility of an application is a subjective measure of how well an individual with a particular disability, skills, and goals experiences the application via assistive technology. This paper first presents a study with 11 blind users to report how they perceive the accessibility of desktop applications while interacting via assistive technology such as screen readers and a keyboard. The study identifies the low navigational complexity of the user interface (UI) elements as the primary contributor to higher perceived accessibility of different applications. Informed by this study, we develop a probabilistic model that accounts for the number of user actions needed to navigate between any two arbitrary UI elements within an application. This model contributes to the area of computational interaction for non-visual interaction. Next, we derive three metrics from this model: complexity, coverage, and reachability, which reveal important statistical characteristics of an application indicative of its perceived accessibility. The proposed metrics are appropriate for comparing similar applications and can be fine-tuned for individual users to cater to their skills and goals. Finally, we present five use cases, demonstrating how blind users, application developers, and accessibility practitioners can benefit from our model and metrics.
Footnotes
1 freelists.org/list/program-l and nvda.groups.io/g/nvda
Footnote2 https://accessibilityinsights.io/docs/en/windows/overview/
Footnote
Supplemental Material
- Amaia Aizpurua, Myriam Arrue, Simon Harper, and Markel Vigo. 2014. Are users the gold standard for accessibility evaluation?. In Proceedings of the 11th Web for All Conference. 1–4.Google Scholar
Digital Library
- Cynthia L Bennett and Daniela K Rosner. 2019. The Promise of Empathy: Design, Disability, and Knowing the" Other". In Proceedings of the 2019 CHI conference on human factors in computing systems. 1–13.Google ScholarDigital Library
- Jeffrey P. Bigham, Irene Lin, and Saiph Savage. 2017. The Effects of "Not Knowing What You Don’t Know" on Web Accessibility for Blind Web Users. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (Baltimore, Maryland, USA) (ASSETS ’17). Association for Computing Machinery, New York, NY, USA, 101–109. https://doi.org/10.1145/3132525.3132533Google ScholarDigital Library
- Syed Masum Billah, Vikas Ashok, Donald E. Porter, and I.V. Ramakrishnan. 2017. Ubiquitous Accessibility for People with Visual Impairments: Are We There Yet?. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 5862–5868. https://doi.org/10.1145/3025453.3025731Google ScholarDigital Library
- Syed Masum Billah, Donald E. Porter, and I. V. Ramakrishnan. 2016. Sinter: low-bandwidth remote access for the visually-impaired. In Proceedings of the Eleventh European Conference on Computer Systems. ACM, 2901335, 1–16. https://doi.org/10.1145/2901318.2901335Google ScholarDigital Library
- Shirin Feiz, Syed Masum Billah, Vikas Ashok, Roy Shilkrot, and I. V. Ramakrishnan. 2019. Towards Enabling Blind People to Independently Write on Printed Forms. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, 1–12. https://doi.org/10.1145/3290605.3300530Google ScholarDigital Library
- Syed Masum Billah, Yu-Jung Ko, Vikas Ashok, Xiaojun Bi, and I. V. Ramakrishnan. 2019. Accessible Gesture Typing for Non-Visual Text Entry on Smartphones. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, 1–10. https://doi.org/10.1145/3290605.3300606Google ScholarDigital Library
- Farhani Momotaz, Md Touhidul Islam, Md Ehtesham-Ul-Haque, and Syed Masum Billah. 2021. Understanding Screen Readers’ Plugins. In The 23rd International ACM SIGACCESS Conference on Computers and Accessibility. ACM, 1–10. https://doi.org/10.1145/3441852.3471205Google ScholarDigital Library
- Pradipta Biswas and Peter Robinson. 2008. Automatic Evaluation of Assistive Interfaces. In Proceedings of the 13th International Conference on Intelligent User Interfaces (Gran Canaria, Spain) (IUI ’08). Association for Computing Machinery, New York, NY, USA, 247–256. https://doi.org/10.1145/1378773.1378806Google ScholarDigital Library
- Pradipta Biswas and Peter Robinson. 2010. Evaluating the design of inclusive interfaces by simulation. In Proceedings of the 15th international conference on intelligent user interfaces. 277–280.Google ScholarDigital Library
- Yevgen Borodin, Jeffrey P. Bigham, Glenn Dausch, and I. V. Ramakrishnan. 2010. More than meets the eye: a survey of screen-reader browsing strategies. In Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A). ACM, 1806005, 1–10. https://doi.org/10.1145/1805986.1806005Google ScholarDigital Library
- Giorgio Brajnik. 2008. Beyond conformance: the role of accessibility evaluation methods. In International Conference on Web Information Systems Engineering. Springer, 63–80.Google ScholarDigital Library
- Giorgio Brajnik. 2008. A comparative test of web accessibility evaluation methods. In Proceedings of the 10th international ACM SIGACCESS Conference on Computers and Accessibility. 113–120.Google ScholarDigital Library
- Dan Brown. 2010. Eight principles of information architecture. Bulletin of the American Society for Information Science and Technology 36, 6 (2010), 30–34.Google ScholarCross Ref
- A. Bryman and R.G. Burgess. 1994. Analyzing Qualitative Data. Routledge. https://books.google.com/books?id=KQkotSd9YWkCGoogle Scholar
- Shiya Cao and Eleanor Loiacono. 2019. The state of the awareness of web accessibility guidelines of student website and app developers. In International Conference on Human-Computer Interaction. Springer, 32–42.Google ScholarDigital Library
- SK Card, TP Moran, and A Newell. 1983. The Psychology of Human Computer Interaction Lawrence Erlbaum. Associates, NJ (1983).Google Scholar
- Stuart K Card, Thomas P Moran, and Allen Newell. 1980. The keystroke-level model for user performance time with interactive systems. Commun. ACM 23, 7 (1980), 396–410.Google ScholarDigital Library
- Stuart K Card, Thomas P Moran, and Allen Newell. 1983. The psychology of human-computer interaction. Crc Press.Google ScholarDigital Library
- Lucas Pedroso Carvalho, Bruno Piovesan Melchiori Peruzza, Flávia Santos, Lucas Pereira Ferreira, and André Pimenta Freire. 2016. Accessible smart cities? Inspecting the accessibility of Brazilian municipalities’ mobile applications. In Proceedings of the 15th Brazilian Symposium on Human Factors in Computing Systems. 1–10.Google ScholarDigital Library
- Raphael Clegg-Vinell, Christopher Bailey, and Voula Gkatzidou. 2014. Investigating the appropriateness and relevance of mobile web accessibility guidelines. In Proceedings of the 11th Web for All Conference. 1–4.Google ScholarDigital Library
- Mark Colley, Taras Kränzle, and Enrico Rukzio. 2022. Accessibility-Related Publication Distribution in HCI Based on a Meta-Analysis. In CHI Conference on Human Factors in Computing Systems Extended Abstracts. 1–28.Google ScholarDigital Library
- Giliam de Carpentier. 2012. Discrete staircase probability distribution. https://www.decarpentier.nl/staircase-distributionGoogle Scholar
- Biplab Deka, Zifeng Huang, and Ranjitha Kumar. 2016. ERICA: Interaction Mining Mobile Apps. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (Tokyo, Japan) (UIST ’16). ACM, New York, NY, USA, 767–776. https://doi.org/10.1145/2984511.2984581Google ScholarDigital Library
- Catherine D’Ignazio, Alexis Hope, Becky Michelson, Robyn Churchill, and Ethan Zuckerman. 2016. A Feminist HCI Approach to Designing Postpartum Technologies: " When I first saw a breast pump I was wondering if it was a joke". In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 2612–2622.Google ScholarDigital Library
- Morgan Dixon and James Fogarty. 2010. Prefab: Implementing Advanced Behaviors Using Pixel-based Reverse Engineering of Interface Structure. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Atlanta, Georgia, USA) (CHI ’10). ACM, New York, NY, USA, 1525–1534. https://doi.org/10.1145/1753326.1753554Google ScholarDigital Library
- Gavin Doherty and Mieke Massink. 1999. Continuous interaction and human control. In Proceedings of the XVIII European annual conference on human decision making and manual control. 80–96.Google Scholar
- Paul M Fitts. 1954. The information capacity of the human motor system in controlling the amplitude of movement.Journal of experimental psychology 47, 6 (1954), 381.Google Scholar
- International Organization for Standardization. 2010. Ergonomics of Human-system Interaction: Part 210: Human-centred Design for Interactive Systems. ISO.Google Scholar
- Martin Fowler and Matthew Foemmel. 2006. Continuous integration.Google Scholar
- SBL Ferreira, RC dos Santos, and DS Silveira. 2007. Panorama of Brazilian web accessibility, Proceedings of the XXXI ANPAD Meeting-EnANPAD, page 17pGoogle Scholar
- Richard Gong and David Kieras. 1994. A validation of the GOMS model methodology in the development of a specialized, commercial software application. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 351–357.Google ScholarDigital Library
- Google. 2021. Expresso | Use Espresso to write concise, beautiful, and reliable Android UI tests. https://developer.android.com/training/testing/espresso. Online; accessed 23 January 2023.Google Scholar
- Google. 2021. iOS UI Automation Test Framework. https://github.com/google/EarlGrey. Online; accessed 23 January 2023.Google Scholar
- Web Accessibility Initiative. 2017. Accessibility, usability, and inclusion: related aspects of a web for all.Google Scholar
- Richard J Jagacinski and John M Flach. 2018. Control theory for humans: Quantitative approaches to modeling performance. CRC press.Google Scholar
- Bonnie E John. 1994. Toward a deeper comparison of methods: A reaction to Nielsen & Phillips and new data. In Conference Companion on Human Factors in Computing Systems. 285–286.Google ScholarDigital Library
- Bonnie E John and David E Kieras. 1996. The GOMS family of user interface analysis techniques: Comparison and contrast. ACM Transactions on Computer-Human Interaction (TOCHI) 3, 4(1996), 320–351.Google ScholarDigital Library
- Bonnie E John and David E Kieras. 1996. Using GOMS for user interface design and evaluation: Which technique?ACM Transactions on Computer-Human Interaction (TOCHI) 3, 4(1996), 287–319.Google Scholar
- David Kieras. 1997. A guide to GOMS model usability evaluation using NGOMSL. In Handbook of human-computer interaction. Elsevier, 733–766.Google Scholar
- David Kieras 2001. Using the keystroke-level model to estimate execution times. University of Michigan 555 (2001).Google Scholar
- KIF. 2021. KIF iOS Integration Testing Framework. https://github.com/kif-framework/KIF. Online; accessed 23 January 2023.Google Scholar
- John Brooke. 1996. SUS-A quick and dirty usability scale. Usability evaluation in industry 189, (1996), 194, 194.Google Scholar
- Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in psychology. Elsevier, 52, 139–183.Google Scholar
- Sooyeon Lee, Madison Reddie, and John M Carroll. 2021. Designing for Independence for People with Visual Impairments. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1(2021), 1–19.Google ScholarDigital Library
- Barbara Leporini and Fabio Paternò. 2004. Increasing usability when interacting through screen readers. Universal access in the information society 3, 1 (2004), 57–70.Google Scholar
- Jennifer Mankoff, Gillian R Hayes, and Devva Kasnitz. 2010. Disability studies as a source of critical inquiry for the field of assistive technology. In Proceedings of the 12th international ACM SIGACCESS conference on Computers and accessibility. 3–10.Google ScholarDigital Library
- Beatriz Martins and Carlos Duarte. 2022. Large-scale study of web accessibility metrics. Universal Access in the Information Society(2022), 1–24.Google Scholar
- Atif Memon, Ishan Banerjee, and Adithya Nagarajan. 2003. GUI ripping: Reverse engineering of graphical user interfaces for testing. In 10th Working Conference on Reverse Engineering, 2003. WCRE 2003. Proceedings. IEEE, 260–269.Google ScholarCross Ref
- Microsoft. 2021. Accessibility tools - AccChecker (UI Accessibility Checker). https://docs.microsoft.com/en-us/windows/win32/winauto/ui-accessibility-checker. Online; accessed 23 January 2023.Google Scholar
- Microsoft. 2021. Accessibility tools - Inspect. https://learn.microsoft.com/en-us/windows/win32/winauto/inspect-objects. Online; accessed 23 January 2023.Google Scholar
- Microsoft Inc.2020. UI Automation Overview. http://msdn.microsoft.com/en-us/library/ms747327.aspxGoogle Scholar
- Lauren R Milne, Cynthia L Bennett, and Richard E Ladner. 2014. The accessibility of mobile health sensors for blind users. In International Technology and Persons with Disabilities Conference Scientific/Research Proceedings (CSUN 2014). 166–175.Google Scholar
- Antti Oulasvirta, Per Ola Kristensson, Xiaojun Bi, and Andrew Howes. 2018. Computational interaction. Oxford University Press.Google Scholar
- Helen Petrie and Omar Kheir. 2007. The Relationship between Accessibility and Usability of Websites. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’07). Association for Computing Machinery, New York, NY, USA, 397–406. https://doi.org/10.1145/1240624.1240688Google ScholarDigital Library
- Christopher Power, Paul Cairns, and Mark Barlet. 2018. Inclusion in the third wave: access to experience. In New Directions in Third Wave Human-Computer Interaction: Volume 1-Technologies. Springer, 163–181.Google Scholar
- Dudekula Mohammad Rafi, Katam Reddy Kiran Moses, Kai Petersen, and Mika V Mäntylä. 2012. Benefits and limitations of automated software testing: Systematic literature review and practitioner survey. In 2012 7th International Workshop on Automation of Software Test (AST). IEEE, 36–42.Google ScholarCross Ref
- Robolectric. 2021. Robolectric is the industry-standard unit testing framework for Android. http://robolectric.org/getting-started/. Online; accessed 23 January 2023.Google Scholar
- Anne Spencer Ross, Xiaoyi Zhang, James Fogarty, and Jacob O. Wobbrock. 2017. Epidemiology As a Framework for Large-Scale Mobile Application Accessibility Assessment. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (Baltimore, Maryland, USA) (ASSETS ’17). ACM, New York, NY, USA, 2–11. https://doi.org/10.1145/3132525.3132547Google ScholarDigital Library
- Martin Schrepp. 2010. GOMS analysis as a tool to investigate the usability of web units for disabled users. Universal Access in the Information Society 9, 1 (2010), 77–86.Google ScholarDigital Library
- Leandro Coelho Serra, Lucas Pedroso Carvalho, Lucas Pereira Ferreira, Jorge Belimar Silva Vaz, and André Pimenta Freire. 2015. Accessibility evaluation of e-government mobile applications in Brazil. Procedia Computer Science 67 (2015), 348–357.Google ScholarCross Ref
- Ben Shneiderman. 2000. Universal usability. Commun. ACM 43, 5 (2000), 84–91.Google ScholarDigital Library
- Ben Shneiderman. 2002. Promoting universal usability with multi-layer interface design. ACM SIGCAPH Computers and the Physically Handicapped73-74 (2002), 1–8.Google Scholar
- Hironobu Takagi, Chieko Asakawa, Kentarou Fukuda, and Junji Maeda. 2003. Accessibility designer: visualizing usability for the blind. ACM SIGACCESS accessibility and computing77-78 (2003), 177–184.Google ScholarDigital Library
- TalkBack. [n. d.]. TalkBack: An Open Source Screenreader For Android. https://support.google.com/accessibility/android/answer/6283677Google Scholar
- The World Wide Web Consortium (W3C). 2018. Web Content Accessibility Guidelines (WCAG) 2.1. https://www.w3.org/TR/WCAG21/Google Scholar
- [67] Section 508 of the Rehabilitation Act.2019. https://www.fcc.gov/general/section-508-rehabilitation-actGoogle Scholar
- Mary Frances Theofanos and Janice Redish. 2003. Bridging the gap: between accessibility and usability. interactions 10, 6 (2003), 36–51.Google Scholar
- Shannon M. Tomlinson. 2016. Perceptions of Accessibility and Usability by Blind or Visually Impaired Persons: A Pilot Study. In Proceedings of the 79th ASIST Annual Meeting: Creating Knowledge, Enhancing Lives through Information & Technology(Copenhagen, Denmark) (ASIST ’16). American Society for Information Science, USA, Article 120, 4 pages.Google ScholarDigital Library
- Henrik Tonn-Eichstädt. 2006. Measuring website usability for visually impaired people-a modified GOMS analysis. In Proceedings of the 8th international ACM SIGACCESS conference on Computers and accessibility. 55–62.Google ScholarDigital Library
- Shari Trewin, Bonnie E. John, John Richards, Cal Swart, Jonathan Brezin, Rachel Bellamy, and John Thomas. 2010. Towards a Tool for Keystroke Level Modeling of Skilled Screen Reading. In Proceedings of the 12th International ACM SIGACCESS Conference on Computers and Accessibility(Orlando, Florida, USA) (ASSETS ’10). Association for Computing Machinery, New York, NY, USA, 27–34. https://doi.org/10.1145/1878803.1878811Google ScholarDigital Library
- Daniel W Turner III and Nicole Hagstrom-Schmidt. 2022. Qualitative interview design. Howdy or Hello? Technical and Professional Communication (2022).Google Scholar
- Andries Van Dam. 1997. Post-WIMP user interfaces. Commun. ACM 40, 2 (1997), 63–67.Google ScholarDigital Library
- Markel Vigo, Justin Brown, and Vivienne Conway. 2013. Benchmarking web accessibility evaluation tools: measuring the harm of sole reliance on automated tests. In Proceedings of the 10th International Cross-Disciplinary Conference on Web Accessibility. 1–10.Google ScholarDigital Library
- Beat Vollenwyder, Glena H Iten, Florian Brühlmann, Klaus Opwis, and Elisa D Mekler. 2019. Salient beliefs influencing the intention to consider Web Accessibility. Computers in Human Behavior 92 (2019), 352–360.Google ScholarDigital Library
- Beat Vollenwyder, Serge Petralito, Glena H. Iten, Florian Brühlmann, Klaus Opwis, and Elisa D. Mekler. 2023. How compliance with web accessibility standards shapes the experiences of users with and without disabilities. International Journal of Human-Computer Studies 170 (2023), 102956. https://doi.org/10.1016/j.ijhcs.2022.102956Google ScholarDigital Library
- W3C Web Accessibility Initiative (WAI). 2016. Accessibility, Usability, and Inclusion. https://www.w3.org/WAI/fundamentals/accessibility-usability-inclusion/Google Scholar
- Cynthia Waddell, Bob Regan, Shawn Lawton Henry, Michael R Burks, Jim Thatcher, Mark D Urban, and Paul Bohman. 2003. Constructing accessible web sites. Apress.Google Scholar
- Jonathan Lazar, Alfreda Dudley-Sponaugle, and Kisha-Dawn Greenidge.2004. Improving web accessibility: a study of webmaster perceptions. Computers in human behavior, 20, 2, 269–288ElsevierGoogle Scholar
- Jacob O Wobbrock, Shaun K Kane, Krzysztof Z Gajos, Susumu Harada, and Jon Froehlich. 2011. Ability-based design: Concept, principles and examples. ACM Transactions on Accessible Computing (TACCESS) 3, 3 (2011), 1–27.Google ScholarDigital Library
- Mulong Xie, Sidong Feng, Zhenchang Xing, Jieshan Chen, and Chunyang Chen. 2020. UIED: a hybrid tool for GUI element detection. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 1655–1659.Google ScholarDigital Library
- Jian Xu, Syed Masum Billah, Roy Shilkrot, and Aruna Balasubramanian. 2019. DarkReader: Bridging the Gap Between Perception and Reality of Power Consumption in Smartphones for Blind Users. In The 21st International ACM SIGACCESS Conference on Computers and Accessibility (Pittsburgh, PA, USA) (ASSETS ’19). Association for Computing Machinery, New York, NY, USA, 96–104. https://doi.org/10.1145/3308561.3353806Google ScholarDigital Library
- Shunguo Yan and P. G. Ramachandran. 2019. The Current Status of Accessibility in Mobile Apps. ACM Trans. Access. Comput. 12, 1, Article 3 (Feb. 2019), 31 pages. https://doi.org/10.1145/3300176Google ScholarDigital Library
- Yeliz Yesilada, Giorgio Brajnik, Markel Vigo, and Simon Harper. 2012. Understanding web accessibility and its drivers. In Proceedings of the international cross-disciplinary conference on web accessibility. 1–9.Google ScholarDigital Library
- William E Hick. 1952. On the rate of gain of information. Quarterly Journal of experimental psychology, Taylor & Francis, 4, 1, 11–26.Google Scholar
- Yeliz Yesilada, Giorgio Brajnik, Markel Vigo, and Simon Harper. 2015. Exploring perceptions of web accessibility: a survey approach. Behaviour & Information Technology 34, 2 (2015), 119–134.Google ScholarDigital Library
- Amaia Aizpurua, Simon Harper, and Markel Vigo. 2016. Exploring the relationship between web accessibility and user experienceInternational Journal of Human-Computer Studies, Elsevier, Vol. 91, 13–23 pages.Google Scholar
- Krzysztof Z. Gajos, Daniel Weld, and Jacob Wobbrock. 2010. Automatically generating personalized user interfaces with SUPPLE. Vol. 174. 910–950 pages. https://doi.org/10.1016/j.artint.2010.05.005Google ScholarDigital Library
Index Terms
- A Probabilistic Model and Metrics for Estimating Perceived Accessibility of Desktop Applications in Keystroke-Based Non-Visual Interactions
Recommendations
WAI-ARIA live regions: eBuddy IM as a case example
W4A '10: Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A)Rich Internet Applications (RIAs) offer new levels of user interactivity through a Web browser. By combining semantics, style and behavior it is possible to create a RIA that can rival a traditional desktop application. Unfortunately, much of the ...
Are users the gold standard for accessibility evaluation?
W4A '14: Proceedings of the 11th Web for All ConferenceUser testing is considered a key part of web accessibility evaluation. However, little is known about how effective is for identifying accessibility problems. Our experience, informed by a series of studies with blind users, corroborates that a website ...
Interacting with mobile devices via VoiceOver: usability and accessibility issues
OzCHI '12: Proceedings of the 24th Australian Computer-Human Interaction ConferenceIn this paper we analyze the interaction of blind users with Apple touchscreen devices iPad, iPhone and iPod touch, accessible to the visually-impaired thanks to their pre-installed VoiceOver screen reader or magnifier. Specifically, we focus on the ...
Comments