Skip to main content
SpringerLink
Log in
Book cover

Web Accessibility pp 505–521Cite as

Automatic Web Accessibility Metrics

Where We Were and Where We Went

  • Chapter
  • First Online:

Part of the book series: Human–Computer Interaction Series ((HCIS))

Abstract

This chapter starts by depicting how the topic of web accessibility metrics was in 2011, analyses what has happened since, and discusses the challenges that accessibility metrics face today. More specifically, we review a variety of metrics and a quality framework for metrics based on validity, reliability, sensitivity, adequacy and complexity. We then describe what new metrics were defined and how metrics have been used in the last 7 years, which range from assessments of accessibility awareness, of accessibility progress, to in-depth analyses within the banking sector, in country-based and continent-based assessments. We illustrate metrics that use new kinds of data, like human judgements or questionnaires, that in some case are used to deal with validity and reliability of metrics. The chapter ends with a discussion of the challenges ahead.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   229.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   299.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   299.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    Note that this list is not exhaustive.

References

  • Abascal J, Arrue M, Valencia X (2018) Tools for web accessibility evaluation. In: Yesilada Y, Harper S (eds) Web accessibility. Springer. This volume

    Google Scholar 

  • Al-Khalifa HS, Baazeem I, Alamer R (2017) Revisiting the accessibility of Saudi Arabia government websites. Univers Access Inf Soc 16(4):1027–1039 Nov

    Article  Google Scholar 

  • Bailey J, Burd E (2005) Tree-map visualization for web accessibility. In: COMPSAC ’05: proceedings of the 29th annual international computer software and applications conference (COMPSAC’05) vol 1. IEEE Computer Society, Washington, pp 275–280

    Google Scholar 

  • Bailey J, Burd E (2007) Towards more mature web maintenance practices for accessibility. In: IEEE workshop on web site evolution, WSE07. IEEE Press, pp 81–87

    Google Scholar 

  • Benavidez C, Cardoso C, Fernandes J, Gutiérrez y Restrepo E, Gutiérrez H, Martínez-Normand L (2014) Iberoamerican observatory of web accessibility. In: Miesenberger K, Fels D, Archambault D, Peňáz P, Zagler W (eds) Computers helping people with special needs. Springer International Publishing, pp 101–108

    Google Scholar 

  • Brajnik G (2004) Comparing accessibility evaluation tools: a method for tool effectiveness. Univers Access Inf Soc 3(3):252–263

    Article  Google Scholar 

  • Brajnik G, Graca S (2018) On accessibility policies for higher education institutions. In: Proceedings of the 15th web for all conference, ACM. https://doi.org/10.1145/3192714.3192833

  • Brajnik G, Lomuscio R (2007) SAMBA: a semi-automatic method for measuring barriers of accessibility. In: Trewin S, Pontelli E (eds) 9th international ACM SIGACCESS conference on computers and accessibility, ASSETS, Tempe, AZ, ACM Press

    Google Scholar 

  • Brajnik G, Yesilada Y, Harper S (2011) The expertise effect on web accessibility evaluation methods. Hum Comput Interact 26(3):246–283

    Google Scholar 

  • BS and ISO (2011) Systems and software engineering systems and software quality requirements and evaluation (square) system and software quality models - BS ISO/IEC 25010:2011

    Google Scholar 

  • Bühler C, Heck H, Perlick O, Nietzio A, Ullveit-Moe, N (2006) Interpreting results from large scale automatic evaluation of web accessibility. In: Computers helping people with special needs, ICCHP06, LNCS 4061. Springer, pp 184–191

    Google Scholar 

  • Eraslan S, Bailey C (2018) End user evaluations. In: Yesilada Y, Harper S (eds) Web accessibility. Springer. This volume

    Google Scholar 

  • European Parliament (2016) Directive (eu) 2016/2102 of the European parliament and of the council of 26 October 2016 on the accessibility of the websites and mobile applications of public sector bodies. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv:OJ.L_.2016.327.01.0001.01.ENG

  • Fernandes J, Benavidez C (2011) A zero in echecker equals a 10 in examinator: a comparison between two metrics by their scores. In: Connor JO, Vigo M, Brajnik G (eds) W3C symposium on website accessibility metrics, page article 8

    Google Scholar 

  • Fukuda K, Saito S, Takagi H, Asakawa C (2005) Proposing new metrics to evaluate web usability for the blind. In: Extended abstracts of proceedings of the SIGCHI conference on human factors in computing systems, CHI’05. ACM Press, pp 1387–1390

    Google Scholar 

  • G3ict (2016) Convention on the rights of persons with disabilties - 2016 ICT accessibility progress report. Technical report, G3ict. www.g3ict.org

  • Gambino O, Pirrone R, Di Giorgio F (2016) Accessibility of the italian institutional web pages: a survey on the compliance of the italian public administration web pages to the stanca act and its 22 technical requirements for web accessibility. Univers Access Inf Soc 15(2):305–312 Jun

    Article  Google Scholar 

  • González J, Macías M, Rodríguez R, Sànchez F (2003) Accessibility metrics of web pages for blind end–users. In: Cueva Lovelle JM et al (ed) Web engineering, number 2722 in Lecture notes in computer science. Springer, pp 661–707

    Google Scholar 

  • Hackett S, Parmanto B, Zeng X (2004) Accessibility of internet web sites through time. In: Proceedings of ACM SIGACCESS conference on computers and accessibility. ACM Press, pp 32–39

    Google Scholar 

  • Hanson VL, Richards JT (2013) Progress on website accessibility? ACM Trans Web 7(1). https://doi.org/10.1145/2435215.2435217

    Article  Google Scholar 

  • Ismailova R (2017) Web site accessibility, usability and security: a survey of government web sites in Kyrgyz Republic. Univers Access Inf Soc 16(1):257–264

    Article  Google Scholar 

  • Ivory MY, Yu S, Gronemyer K (2004) Search result exploration: a preliminary study of blind and sighted users’ decision making and performance. In: CHI ’04 extended abstracts on human factors in computing systems, CHI EA ’04. ACM, New York, pp 1453–1456

    Google Scholar 

  • Lopes R, Carriço L (2008) The impact of accessibility assessment in macro scale universal usability studies of the web. In: International cross-disciplinary conference on web accessibility, W4A08. ACM Press, pp 5–14

    Google Scholar 

  • Lorca P, de Andrés J, Belén Martínez A (2016) Does web accessibility differ among banks? World Wide Web 19:351–373. https://doi.org/10.1007/s11280-014-0314-0

    Article  Google Scholar 

  • Mich L, Franch M, Gaio L (2003) Evaluating and designing web site quality. IEEE MultiMed 10(1):34–43

    Article  Google Scholar 

  • Mucha J, Snaprud M, Nietzio A (2016) Web page clustering for more efficient website accessibility evaluations. In: Miesenberger K, Bühler C, Penaz P (eds) Computers helping people with special needs. Springer International Publishing, Cham, pp 259–266

    Google Scholar 

  • Nietzio A, Eibegger M, Goodwin M, Snaprud M (2011) Towards a score function for wcag 2.0 benchmarking. In: Connor JO, Vigo M, Brajnik G (eds) W3C symposium on website accessibility metrics, page article 11

    Google Scholar 

  • Nurmela K, Pirhonen A, Salminen A (2013) Accessibility of public web services: a distant dream? In: INTERACT 2013. IFIP international federation for information processing

    Google Scholar 

  • Ochoa RL, Crovi DM (2018) Evaluation of accessibility in mexican cybermedia. Univers Access Inf Soc

    Google Scholar 

  • Olsina L, Rossi G (2002) Measuring web application quality with WebQEM. IEEE Multimed 9(4):20–29

    Article  Google Scholar 

  • Olsina L, Papa F, Molina H (2008) How to measure and evaluate web applications in a consistent way. Springer, London, pp 385–420

    Chapter  Google Scholar 

  • Power C, Petrie H (2018) Working with participants. In: Yesilada Y, Harper S (eds) Web accessibility. Springer. This volume

    Google Scholar 

  • Sirithumgul P, Suchato A, Punyabukkana P (2009) Quantitative evaluation for web accessibility with respect to disabled groups. In: International cross-disciplinary conference on web accessibility, W4A09. ACM Press, pp 136–141

    Google Scholar 

  • Song S, Wang C, Li L, Yu Z, Lin X, Bu J (2017) WAEM: a web accessibility evaluation metric based on partial user experience order. In: Proceedings of the 14th web for all conference. ACM, 21 p

    Google Scholar 

  • Song S, Bu J, Shen C, Artmeier A, Yu Z, Zhou Q (2018a) Reliability aware web accessibility experience metric. In: Proceedings of the 15th web for all conference, ACM. https://doi.org/10.1145/3192714.3192836

  • Song S, Bu J, Wang Y, Yu Z, Artmeier A, Dai L, Wang C (2018b) Web accessibility evaluation in a crowdsourcing-based system with expertise-based decision strategy. In: Proceedings of the 15th web for all conference, ACM. https://doi.org/10.1145/3192714.3192827

  • Sullivan T, Matson R (2000) Barriers to use: usability and content accessibility on the web’s most popular sites. In: Proceedings of the 2000 conference on universal usability, CUU ’00. ACM, New York, pp 139–144

    Google Scholar 

  • Takagi H, Kawanaka S, Kobayashi M, Sato D, Asakawa C (2009) Collaborative web accessibility improvement: challenges and possibilities. In: Proceedings of the 11th international ACM SIGACCESS conference on Computers and accessibility, Assets ’09. ACM, New York, pp 195–202

    Google Scholar 

  • Velleman E, Strobbe C, Koch J, Velasco CA, Snaprud M, Nietzio A (2007) D-WAB4: unified web evaluation methodology (UWEM)1.2. Technical report, WAB Cluster

    Google Scholar 

  • Vigo M, Brajnik G (2011) Automatic web accessibility metrics: where we are and where we can go. Interact Comput 23(2):137–155

    Article  Google Scholar 

  • Vigo M, Harper S (2013) Evaluating accessibility-in-use. In: Proceedings of the 10th international cross-disciplinary conference on web accessibility, W4A ’13. ACM, New York, pp 7:1–7:4

    Google Scholar 

  • Vigo M, Arrue M, Brajnik G, Lomuscio R, Abascal J (2007) Quantitative metrics for measuring web accessibility. In: W4A ’07: proceedings of the 2007 international cross-disciplinary workshop on web accessibility (W4A). ACM Press, New York, pp 99–107

    Google Scholar 

  • Vigo M, Brown J, Conway V (2013) Benchmarking web accessibility evaluation tools: measuring the harm of sole reliance on automated tests. In: Proceedings of the 10th international cross-disciplinary conference on web accessibility, W4A ’13. ACM, New York, pp 1:1–1:10

    Google Scholar 

  • Wentz B, Pham D, Feaser E, Smith D, Smith J, Wilson A (2018) Documenting the accessibility of 100 US bank and finance websites. Univers Access Inf Soc

    Google Scholar 

  • Zeng X (2004) Evaluation and enhancement of web content accessibility for persons with disabilities. Ph.D. thesis, University of Pittsburgh

    Google Scholar 

  • Zhang M, Wang C, Bu J, Yu Z, Lu Y, Zhang R, Chen C (2015) An optimal sampling method for web accessibility quantitative metric. In: Proceedings of the 12th web for all conference, ACM. https://doi.org/10.1145/2745555.2746663

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, Informatics and Physics, University of Udine, Udine, Italy

    Giorgio Brajnik

  2. School of Computer Science, University of Manchester, Manchester, UK

    Markel Vigo

Authors
  1. Giorgio Brajnik
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Markel Vigo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Giorgio Brajnik .

Editor information

Editors and Affiliations

  1. Middle East Technical University Northern Cyprus Campus, Güzelyurt, Turkey

    Yeliz Yesilada

  2. School of Computer Science, University of Manchester, Manchester, UK

    Simon Harper

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer-Verlag London Ltd., part of Springer Nature

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Brajnik, G., Vigo, M. (2019). Automatic Web Accessibility Metrics. In: Yesilada, Y., Harper, S. (eds) Web Accessibility. Human–Computer Interaction Series. Springer, London. https://doi.org/10.1007/978-1-4471-7440-0_27

Download citation

  • DOI: https://doi.org/10.1007/978-1-4471-7440-0_27

  • Published:

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-4471-7439-4

  • Online ISBN: 978-1-4471-7440-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation