Christopher Bailey
ABSTRACT
The Accessibility Evaluation Assistant (AEA) is a web accessibility knowledge management tool designed specifically to assist novice auditors in conducting an accessibility evaluation. The software incorporates a bespoke structured walkthrough approach designed to guide the auditor through a series of checks based on established accessibility principles with the goal of identifying accessibility barriers. A previous trial examined the effectiveness of the AEA and explored the pedagogical potential of the tool when incorporated into the undergraduate computing curriculum. The results of the evaluations carried out by the novices yielded promising levels of validity and reliability. This paper presents the results of a second experiment designed to test the overall efficacy of the AEA when compared to a WCAG 2.0 conformance review. The results of evaluations produced using both AEA and Conformance Review methods were examined and comparisons made of quality factors such as effectiveness, reliability, efficiency and usefulness. Quantitative and qualitative data from the experiment support continued use of the AEA in an educational context, highlighting the benefits compared to WCAG 2.0 and gives further insight into the complex nature of developing accessibility evaluation skills in novices.
- Accessibility Evaluation Assistant. Accessibility Research Centre. http://arc.tees.ac.uk/aea/. Accessed: 27/11/10.Google Scholar
- ACM. Current Curricula. Available Online: http://www.acm.org/education/curricula/. Accessed: 14/11/11.Google Scholar
- Al-Khalifa, A. S. & Al-Khalifa, H. S. 2011. An educational tool for generating inaccessible page examples based on WCAG 2.0 failures. In Proceedings of the International Cross-Disciplinary Conference on Web Accessibility (W4A '11). ACM, New York, NY, USA. Article 30, 4 pages. DOI=10.1145/1969289.1969328. http://doi.acm.org/10.1145/1969289.1969328 Google ScholarDigital Library
- Alonso, F., Fuertes, J. L., Gonzalez, L. A. and Martinez, L. 2010. On the testability of WCAG 2.0 for beginners. In Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A) (W4A '10). ACM, New York, NY, USA. DOI=10.1145/1805986.1806000 http://doi.acm.org/10.1145/1805986.1806000 Google ScholarDigital Library
- Arrue, M., Vigo, M., Aizpurua, A. and Abascal, J. 2007. Accessibility Guidelines Management Framework. In C. Stephanidis (Ed.). Universal Access in HCI, Part III, HCI International 2007 (Beijing, China, July 22--27, 2007). LNCS 4556, 3--10, Springer, 2007. Google ScholarDigital Library
- Baguma, R., Stone, R. G., Lugega, J. T., and van der Weide, T. P. 2009. A framework for filtering web accessibility guidelines. In Proceedings of the 2009 international Cross-Disciplinary Conference on Web Accessibililty (W4a) (Madrid, Spain, April 20 - 21, 2009). W4A '09. ACM, New York, NY, 46--49. DOI= http://doi.acm.org/10.1145/1535654.1535663 Google ScholarDigital Library
- Bailey, C., and Pearson, E. 2010. An educational tool to support the accessibility evaluation process. In Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A) (W4A '10). ACM, New York, NY, USA. DOI=10.1145/1805986.1806003 http://doi.acm.org/10.1145/1805986.1806003 Google ScholarDigital Library
- Bailey, C. & Pearson, E. 2011. Development and trial of an educational tool to support the accessibility evaluation process. In Proceedings of the International Cross-Disciplinary Conference on Web Accessibility (W4A '11). ACM, New York, NY, USA, Article 2, 10 pages. DOI=10.1145/1969289.1969293. http://doi.acm.org/10.1145/1969289.1969293 Google ScholarDigital Library
- Benavídez, C., Fuertes, J. L., Gutiérrez, E., & Martíínez, L. 2006. Teaching Web Accessibility with "Contramano" and Hera. In: Miesenberger, K., Klaus, J., Zagler, W. L., Karshmer, A. I. (eds.) ICCHP 2006. LNCS, Vol. 4061, pp. 341--348. Springer, Heidelberg. Google ScholarDigital Library
- Brajnik, G. 2008. A comparative test of web accessibility evaluation methods. In Proceedings of the 10th international ACM SIGACCESS Conference on Computers and Accessibility (Halifax, Nova Scotia, Canada, October 13 - 15, 2008). Assets '08. ACM, New York, NY, 113--120. DOI= http://doi.acm.org/10.1145/1414471.1414494 Google ScholarDigital Library
- Brajnik, G. 2009. Validity and reliability of web accessibility guidelines. In Proceedings of the 11th international ACM SIGACCESS conference on Computers and accessibility (Assets '09). ACM, New York, NY, USA, 131--138. DOI=10.1145/1639642.1639666 http://doi.acm.org/10.1145/1639642.1639666 Google ScholarDigital Library
- Brajnik, G., Yesilada, Y. and Harper, S. 2010. Testability and validity of WCAG 2.0: the expertise effect. In Proceedings of the 12th international ACM SIGACCESS conference on Computers and accessibility (ASSETS '10). ACM, New York, NY, USA, 43--50. DOI=10.1145/1878803.1878813 http://doi.acm.org/10.1145/1878803.1878813 Google ScholarDigital Library
- Fuertes, J. L., Gutiérrez, E., & Martíínez, L. 2011. Developing Hera-FFX for WCAG 2.0. In Proceedings of the International Cross-Disciplinary Conference on Web Accessibility (W4A '11). ACM, New York, NY, USA, Article 3, 9 pages. DOI=10.1145/1969289.1969294. http://doi.acm.org/10.1145/1969289.1969294 Google ScholarDigital Library
- Gellenbeck, E. 2005. Integrating accessibility into the computer science curriculum. J. Comput. Small Coll. 21, 1 (October 2005), 267--273. Google ScholarDigital Library
- Kelly, B., Sloan, D., Brown, S., Seale, J., Petrie, H., Lauke, P and Ball, S. 2007. Accessibility 2.0: people, policies and processes. In Proceedings of the 2007 international cross-disciplinary conference on Web accessibility (W4A) (W4A '07). ACM, New York, NY, USA, 138--147. DOI=10.1145/1243441.1243471 http://doi.acm.org/10.1145/1243441.1243471 Google ScholarDigital Library
- Hertzum, M., & Jacobsen, N. E. 2001. The evaluator effect: A chilling fact about usability evaluation methods. International Journal of Human-Computer Interaction, 13, 421--443.Google ScholarCross Ref
- Law, C., Jacko, J., and Edwards, P. 2005. Programmer-focused website accessibility evaluations. In Proceedings of the 7th international ACM SIGACCESS Conference on Computers and Accessibility (Baltimore, MD, USA, October 09 - 12, 2005). Assets '05. ACM, New York, NY, 20--27. DOI= http://doi.acm.org/10.1145/1090785.1090792 Google ScholarDigital Library
- Leporini, B., Paternò, F., Scorcia, A. 2006. Flexible tool support for accessibility evaluation, Interacting with Computers, v.18 n.5, p.869--890, September, 2006 DOI=10.1016/j.intcom.2006.03.001 Google ScholarDigital Library
- Naftali, M. 2010. Analysis and integration of web accessibility metrics. In Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A) (W4A '10). ACM, New York, NY, USA. DOI=10.1145/1805986.1805996 http://doi.acm.org/10.1145/1805986.1805996 Google ScholarDigital Library
- Pearson, E., Bailey, C., & Green, S. 2011. A tool to support the web accessibility evaluation process for novices. In Proceedings of the 16th annual joint conference on Innovation and technology in computer science education (ITiCSE '11). ACM, New York, NY, USA, 28--32. DOI=10.1145/1999747.1999758. http://doi.acm.org/10.1145/1999747.1999758 Google ScholarDigital Library
- Sloan, D. 2006. The Effectiveness of the Web Accessibility Audit as a Motivational and Educational Tool in Inclusive Web Design. Ph.D. Thesis, University of Dundee, Scotland. June, 2006.Google Scholar
- Velleman, E., Meerveld, C., Strobbe, C., Koch, J., Velasco, C. A., Snaprud, M. and Nietzio, A. 2007. D-WAB4 Unified Web Evaluation Methodology. Web Accessibility Benchmarking Cluster. Retrieved 14th February 2010: http://www.wabcluster.org/uwem1_2/UWEM_1_2_CORE.pdfGoogle Scholar
- Waller, A., Hanson, V., and Sloan, D. 2009. Including accessibility within and beyond undergraduate computing courses. In Proceedings of the 11th international ACM SIGACCESS conference on Computers and accessibility (ASSETS '09). ACM, New York, NY, USA, 155--162. DOI=10.1145/1639642.1639670. http://doi.acm.org/10.1145/1639642.1639670 Google ScholarDigital Library
- WCAG 2.0 Assessment Tool. ACCESSIBLE Applications Design and Development Project. Accessed 14th February 2011: http://www.iti.gr/accessible/WCAG2.0_WebAssessmentTool_v2.0.zipGoogle Scholar
- Web Accessibility Toolbar 2.0. The Paciello Group. http://www.paciellogroup.com/resources/wat-ie-about.html. Accessed: 27/11/10.Google Scholar
- Yesilada, Y. Brajnik, G. and Harper, S. 2009. How much does expertise matter?: a barrier walkthrough study with experts and non-experts. In Proceedings of the 11th international ACM SIGACCESS conference on Computers and accessibility (Assets '09). ACM, New York, NY, USA, 203--210. DOI=10.1145/1639642.1639678 http://doi.acm.org/10.1145/1639642.1639678 Google ScholarDigital Library
Index Terms
- Evaluation of the effectiveness of a tool to support novice auditors
Recommendations
Measuring and comparing the reliability of the structured walkthrough evaluation method with novices and experts
W4A '14: Proceedings of the 11th Web for All ConferenceEffective evaluation of websites for accessibility remains problematic. Automated evaluation tools still require a significant manual element. There is also a significant expertise and evaluator effect. The Structured Walkthrough method is the ...
A tool to support the web accessibility evaluation process for novices
ITiCSE '11: Proceedings of the 16th annual joint conference on Innovation and technology in computer science educationThe Accessibility Evaluation Assistant (AEA) is designed to assist novice auditors in the process of an accessibility evaluation of websites. The tool has been incorporated into an undergraduate computing module in Accessibility and Adaptive ...
Development and trial of an educational tool to support the accessibility evaluation process
W4A '11: Proceedings of the International Cross-Disciplinary Conference on Web AccessibilityThis paper describes the design and development of a web accessibility knowledge management tool, known as the Accessibility Evaluation Assistant (AEA) designed to assist novice auditors in the process of an accessibility evaluation. The software ...
Comments