short-paper

Reliability Aware Web Accessibility Experience Metric

Authors:

Chengchao Shen,

Andreas Artmeier,

Qin ZhouAuthors Info & Claims

W4A '18: Proceedings of the 15th International Web for All Conference

Article No.: 24, Pages 1 - 4

https://doi.org/10.1145/3192714.3192836

Published: 23 April 2018 Publication History

Abstract

Web accessibility metrics can measure the accessibility levels of websites. Although many metrics with different motivations have been proposed, current metrics are limited in their applicability when considering user experience. This study proposes Reliability Aware Web Accessibility Experience Metric (RA-WAEM), a novel Web accessibility metric which considers the user experience of people with disabilities and their reliability in objectively assessing the severity of accessibility barriers. We present an optimization algorithm based on Expectation Maximization to derive the parameters of RA-WAEM efficiently. Moreover, we conduct an extensive accessibility study on 46 websites with 323,098 Web pages and collect the user experience of 122 people. An evaluation on this dataset shows that RA-WAEM outperforms state of the art accessibility metrics in reflecting the user experience.

References

[1]

Becker, S. R., Candès, E. J., and Grant, M. C. 2011. Templates for convex cone problems with applications to sparse signal recovery. Mathematical programming computation, 3(3), pp. 165--218.

[2]

Brajnik, G., Yesilada, Y., and Harper, S. 2010. Testability and validity of WCAG 2.0: the expertise effect. In Proceedings of the 12th international ACM SIGACCESS conference on Computers and accessibility. pp. 43--50. ACM.

Digital Library

[3]

Brajnik, G., Yesilada, Y., and Harper, S. 2011. The expertise effect on web accessibility evaluation methods. Human--Computer Interaction, 26(3), pp. 246--283.

[4]

Bühler, C., Heck, H., Perlick, O., Nietzio, A., and Ulltveit-Moe, N. 2006. Interpreting results from large scale automatic evaluation of web accessibility, In International Conference on Computers for Handicapped Persons, Springer, Berlin, Heidelberg, pp. 184--191.

Digital Library

[5]

China Disabled Persons' Federation. 2012. Statics on the population of people from different disability groups in China. http://www.cdpf.org.cn/sjzx/cjrgk/201206/t20120626_387581.shtml.

[6]

Cooper, M. 2011. "Web accessibility metrics" -- What are they for then. https://martyncooper.wordpress.com/2011/12/06/web-accessibility-metrics/.

[7]

Dempster, A. P., Laird, N. M., and Rubin, D. B. 1977. Maximum likelihood from incomplete data via the EM algorithm. Journal of the royal statistical society. Series B (methodological), pp.1--38.

[8]

González, J., Macías, M., Rodríguez, R., Sánchez, F. 2003. Accessibility metrics of web pages for blind end-users, In International Conference on Web Engineering, pp. 661--707.

Digital Library

[9]

Lopes, R., Gomes, D., and Carriço, L. 2010. Web not for all: a large scale study of web accessibility. In Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A). pp. 10. ACM.

Digital Library

[10]

Song, S., Wang, C., Li, L., Yu, Z., Lin, X., Bu, J. 2017. WAEM: A Web Accessibility Evaluation Metric Based on Partial User Experience Order. In Proceedings of the 14th Web for All Conference on The Future of Accessible Work.

Digital Library

[11]

Sullivan, T., Matson, R. 2000. Barriers to use: Usability and content accessibility on the web's most popular sites, Conference on Universal Usability, ACM, pp. 139--144.

Digital Library

[12]

Vigo, M., Arrue, M., and Brajnik, G. 2007. Quantitative metrics for measuring web accessibility. In Proceedings of the 2007 international cross-disciplinary conference on Web accessibility (W4A) pp. 99--107. ACM.

Digital Library

[13]

Web Accessibility Benchmarking Cluster. 2006. D-WAB4 unified web evaluation methodology (UWEM 1.0), 2006. http://www.wabcluster.org/uwem1/.

[14]

World Wide Web Consortium. 1999. Web Content Accessibility Guidelines 1.0, W3C Recommendation. https://www.w3.org/TR/WAI-WEBCONTENT/.

[15]

Zeng, X. 2004. Evaluation and enhancement of web content accessibility for persons with disabilities. PhD thesis, University of Pittsburgh.

Cited By

Martins BDuarte C(2024)Large-scale study of web accessibility metricsUniversal Access in the Information Society10.1007/s10209-022-00956-x23:1(411-434)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1007/s10209-022-00956-x
Soon GYew JChua NSaputra JLeng NCheong W(2023)The effect of interface quality, system quality, and perceived usefulness of an automated pronunciation scoring system on student satisfactionInternational Journal of ADVANCED AND APPLIED SCIENCES10.21833/ijaas.2023.07.00410:7(23-32)Online publication date: Jul-2023
https://doi.org/10.21833/ijaas.2023.07.004
Onay Durdu PSoydemir Ö(2022)A Systematic Review of Web Accessibility MetricsApp and Website Accessibility Developments and Compliance Strategies10.4018/978-1-7998-7848-3.ch004(77-108)Online publication date: 2022
https://doi.org/10.4018/978-1-7998-7848-3.ch004
Show More Cited By

Index Terms

Reliability Aware Web Accessibility Experience Metric
1. Human-centered computing
  1. Accessibility
    1. Accessibility design and evaluation methods

Recommendations

WAEM: A Web Accessibility Evaluation Metric Based on Partial User Experience Order
W4A '17: Proceedings of the 14th International Web for All Conference

Quantitative accessibility metrics are widely used in accessibility evaluation, which synthesize a summative value to represent the accessibility level of a website. Many of these metrics are the results of a two-step process. The first step is the ...
Investigating accessibility on web-based maps

This paper presents results of an accessibility evaluation carried out with web-based map applications. Three points of view were considered: experts on accessibility, evaluation tools and final users (partially or totally blind people). The document ...
An evaluation of web accessibility metrics based on their attributes
SIGDOC '08: Proceedings of the 26th annual ACM international conference on Design of communication

Accessibility is a concept related to providing access to Web content to people with different abilities and people using different devices. A number of metrics have been proposed in the last years to help to obtain quantitative Web accessibility ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

W4A '18: Proceedings of the 15th International Web for All Conference

April 2018

223 pages

ISBN:9781450356510

DOI:10.1145/3192714

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 April 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper
Research
Refereed limited

Funding Sources

the National Key Technology R&D Program of China
Zhejiang Provincial Natural Science Foundation of China
the National Natural Science Foundation of China

Conference

W4A '18

W4A '18: The Internet of Accessible Things

April 23 - 25, 2018

Lyon, France

Acceptance Rates

W4A '18 Paper Acceptance Rate 25 of 43 submissions, 58%;

Overall Acceptance Rate 171 of 371 submissions, 46%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
232
Total Downloads

Downloads (Last 12 months)18
Downloads (Last 6 weeks)2

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Martins BDuarte C(2024)Large-scale study of web accessibility metricsUniversal Access in the Information Society10.1007/s10209-022-00956-x23:1(411-434)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1007/s10209-022-00956-x
Soon GYew JChua NSaputra JLeng NCheong W(2023)The effect of interface quality, system quality, and perceived usefulness of an automated pronunciation scoring system on student satisfactionInternational Journal of ADVANCED AND APPLIED SCIENCES10.21833/ijaas.2023.07.00410:7(23-32)Online publication date: Jul-2023
https://doi.org/10.21833/ijaas.2023.07.004
Onay Durdu PSoydemir Ö(2022)A Systematic Review of Web Accessibility MetricsApp and Website Accessibility Developments and Compliance Strategies10.4018/978-1-7998-7848-3.ch004(77-108)Online publication date: 2022
https://doi.org/10.4018/978-1-7998-7848-3.ch004
Alsaeedi A(2020)Comparing Web Accessibility Evaluation Tools and Evaluating the Accessibility of Webpages: Proposed FrameworksInformation10.3390/info1101004011:1(40)Online publication date: 12-Jan-2020
https://doi.org/10.3390/info11010040
García-Santiago LOlvera-Lobo M(2020)How accessibility guidelines are used in Spanish World Heritage websites: an exploratory studyLibrary Hi Tech10.1108/LHT-05-2019-0113ahead-of-print:ahead-of-printOnline publication date: 18-Mar-2020
https://doi.org/10.1108/LHT-05-2019-0113
Brajnik GVigo M(2019)Automatic Web Accessibility MetricsWeb Accessibility10.1007/978-1-4471-7440-0_27(505-521)Online publication date: 4-Jun-2019
https://doi.org/10.1007/978-1-4471-7440-0_27
Song SBu JArtmeier AShi KWang YYu ZWang C(2018)Crowdsourcing-Based Web Accessibility Evaluation with Golden Maximum Likelihood InferenceProceedings of the ACM on Human-Computer Interaction10.1145/32744322:CSCW(1-21)Online publication date: 1-Nov-2018
https://dl.acm.org/doi/10.1145/3274432

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten