Abstract
While automatic tools are not intended to replace human judgment, they are crucial in order to develop accessible websites. The release of WCAG 2.0 has caused great expectation, as it is supposed to be precisely testable with automated review tools. Therefore, more effective tools could be developed. However, so far few tools applying WCAG 2.0 have been developed. This paper presents an evaluation framework which has been updated in order to evaluate the new tests. In addition, it describes a validation process carried out in order to verify the effectiveness of the new version of the evaluation tool. The effectiveness is validated by conducting a quantitative and qualitative analysis of the results obtained by applying both versions of the tool (the one implementing WCAG 1.0 and the one implementing WCAG 2.0) to a set of selected web pages, as well as by manual evaluation of an expert for detecting the possible false positives and false negatives produced by each tool.












Similar content being viewed by others
Notes
XML-Schema of UGL: http://sipt07.si.ehu.es/evalaccess3/ugl.xsd. Its representation: http://sipt07.si.ehu.es/evalaccess3/ugl.png.
References
Abascal, J., Nicolle, C.: Why inclusive design guidelines? In: Abascal, J., Nicolle, C. (eds.) Inclusive Design Guidelines for HCI, Chapter 1, pp. 3–13. Taylor & Francis, London (2001)
Abascal, J., Arrue, M., Fajardo, I., Garay, N., Tomás, J.: Use of guidelines to automatically verify web accessibility. Univ. Access Inf. Soc. 3(1), 71–79. (2004). (Springer)
Abou-Zahra, S. (ed.): Web Accessibility Evaluation Tools: Overview. Available at http://www.w3.org/WAI/ER/tools/ (2006)
Arrue, M., Vigo, M., Abascal, J.: Including heterogeneous web accessibility guidelines in the development process. In: Engineering Interactive Systems, EIS 2007, LNCS 4940, pp. 620–637. Springer (2008)
Brajnik, G.: Towards valid quality models for websites. In: Proceedings of the 7th Conference on Human Factors and the Web (2001)
Brajnik, G.: Comparing accessibility evaluation tools: a method for tool effectiveness. Univ. Access Inf. Soc. 3(3–4), 252–263 (2004). (Springer)
Brajnik, G.: Validity and reliability of web accessibility guidelines. In: Proceedings of the 11th international ACM SIGACCESS Conference on Computers and Accessibility, ASSETS’09, pp. 131–138. ACM Press (2009)
Caldwell, B., Cooper, M., Guarino Reid, L., Vanderheiden, G.: Web Content Accessibility Guidelines 2.0 (W3C Recommendation). Available at http://www.w3.org/TR/WCAG20/ (2008)
Chisholm, W., Vanderheiden, G., Jacobs, I.: Web Content Accessibility Guidelines 1.0. (W3C Recommendation). Available at http://www.w3.org/TR/WAI-WEBCONTENT/ (1999)
Clark, J.: To Hell with WCAG 2.0. A List Apart 217. Available at http://www.alistapart.com/articles/tohellwithwcag2 (2006)
Diaper, D., Worman, L.: Two falls out of three in the automated accessibility assessment of world wide web sites: a-prompt v. bobby. In: People and Computers, vol. XVII, pp. 349–363. Springer (2003)
Freed, G., Rothberg, M., Wlodkowski, T.: Making Educational Software and Web Sites Accessible. Available at http://ncam.wgbh.org/cdrom/guideline/ (2003)
Fuertes, J.L., González, R., Gutiérrez, E., Martínez, L.: Hera-FFX: a Firefox add-on for semi-automatic web accessibility evaluation. In: Proceedings of the 2009 International Cross-Disciplinary Conference on Web Accessibility, W4A’09, pp. 26–35. ACM Press (2009)
IBM Accessibility Center: Developer guidelines for Web Accessibility. Available at http://www-306.ibm.com/able/guidelines/web/accessweb.html. Accessed Dec 2009
Ivory, M.Y., Mankoff, J., Le, A.: Using automated tools to improve web site usage by users with diverse abilities. Inf. Technol. Soc. 1(3), 195–236 (2003)
Ivory, M.Y., Hearst, M.A.: The state of art in automating usability evaluations of user interfaces. ACM Comput. Surv. 33(4), 470–516. (2001) (ACM Press)
Kurniawan, S., Zaphiris, P.: Research-derived web design guidelines for older people. In: Proceedings of the ACM SIGACCESS Conference on Computers and Accessibility, ASSETS’05, pp. 129–135. ACM Press (2005)
Leporini, B., Paternò, F., Scorcia, A.: Flexible tool support for accessibility evaluation. Interact. Comput. 18(5), 869–890 (2006). (Elsevier)
Luque, V., Delgado, C., Gaedke, M., Nussbaumer, M.: WCAG formalization with W3C standards. In: Special Interest Tracks and Posters of the 14th International Conference On World Wide Web, WWW’05, pp. 1146–1147. ACM Press (2005)
Mbipom, G., Harper, S.: The transition from web content accessibility guidelines 1.0 to 2.0: what this means for evaluation and repair. In: Proceedings of the 27th ACM International Conference on Design of Communication, SIGDOC’09, pp. 37–44. ACM Press (2009)
Rabin, J., McCathieNevile, C.: Mobile Web Best Practices (W3C Recommendation). http://www.w3.org/TR/mobile-bp/ (2008)
Takata, Y., Nakamura, T., Seki, H.: Accessibility verification of WWW documents by an automatic guideline verification tool. In: Proceedings of the 37th Hawaii International Conference on System Sciences, HICSS’04, Track 4, vol. 4. IEEE Computer Society (2004)
Vanderdonckt, J.: Development milestones towards a tool for working with guidelines. Interact. Comput. 12(2), 81–118 (1999). (Elsevier)
Vanderdonckt, J., Bereikdar, A.: Automated web evaluation by guideline review. J. Web Eng. 4(2), 102–117. (Rinton Press) (2005)
Vigo, M., Arrue, M., Brajnik, G., Lomuscio, R., Abascal, J.: Quantitative metrics for measuring web accessibility. In: Proceedings of the International Cross-Disciplinary Conference on Web accessibility, W4A’07, pp. 99–107. ACM Press (2007)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Aizpurua, A., Arrue, M., Vigo, M. et al. Validating the effectiveness of EvalAccess when deploying WCAG 2.0 tests. Univ Access Inf Soc 10, 425–441 (2011). https://doi.org/10.1007/s10209-011-0226-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10209-011-0226-z