Skip to main content
SpringerLink
Log in

Web accessibility guideline aggregation for older users and its validation

  • Long paper
  • Published:
Universal Access in the Information Society Aims and scope Submit manuscript

Abstract

Web site-evaluation methodologies and validation engines take the view that all accessibility guidelines must be met to gain compliance. Problems exist in this regard, as contradictions within the rule set may arise, and the type of impairment or its severity is not isolated. The Barrier Walkthrough (BW) method goes someway to addressing these issues, by enabling barrier types derived from guidelines to be applied to different user categories such as motor or visual impairment, etc. However, the problem remains of combinatorial explosion of possibilities when one has to consider users with multiple disabilities. In this paper, a simple set theory operation is used to create a validation scheme for older users by aggregating barrier types specific to motor impaired and low-vision users, thereby creating a new “older users” category from the results of this set union. To evaluate the feasibility and validity of this aggregation approach, two BW experiments were conducted. The first experiment evaluated the aggregated results by focusing on quality attributes and showed that aggregation generates data whose quality is comparable to the original one. However, this first experiment could not test for validity, as the older users category was not included. To remedy this deficiency, another BW experiment was conducted with expert judges who evaluated a web page in the context of older users. In this second experiment, it was found that there is no significant difference between the aggregated and the manually evaluated (by experts) barrier scores, and that the same barriers are identified using experts and aggregation, even though there are differences in how severity scores are distributed. From these results, it is concluded that the aggregation of barriers is a viable alternative to expert evaluation, when the target of that aggregation could not be evaluated manually or it would not be feasible to do so. It is also argued that aggregation is a technique that can be used in combination with other evaluation methods, like user testing or subjective assessments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. This term has been defined in numerous ways; in [42] it is defined as “over 58”.

  2. http://www.bbc.co.uk/commissioning/.

  3. http://www.statistics.gov.uk/.

  4. WCAG, http://www.w3.org/TR/WAI-WEBCONTENT/.

  5. Global data refers here to data for both experts and non-experts evaluators.

  6. http://www.alexa.com.

  7. http://hcw.cs.manchester.ac.uk/research/riam/experiments/riam-samba.php.

  8. http://hcw.cs.manchester.ac.uk/research/riam/experiments/riam-samba-info.php.

  9. Demographics questionnaire, http://hcw.cs.manchester.ac.uk/research/riam/experiments/Samba_riam_demographics.doc.

  10. Post evaluation questionnaire, http://hcw.cs.manchester.ac.uk/research/riam/experiments/Samba_riam_post_evaluation.doc.

  11. Tenth ACM Conference on Computers and Accessibility, see http://www.sigaccess.org/assets08.

  12. In doing this we are assuming that a critical barrier is 3 times more severe than a minor one, and is 1.5 times more severe than a significant one.

  13. http://hcw.cs.manchester.ac.uk/research/riam/experiments/riam-samba-older.php.

References

  1. Anderson, C.: The Long Tail—How Endless Choice is Creating Unlimited Demand. Random House Business Book, New York (2006)

    Google Scholar 

  2. Arch, A.: Web Accessibility for Older Users: A Literature Review. W3c working draft, World Wide Web Consortium (W3C), Web Accessibility Initiative, MIT/CSAIL, Building 32-G530, 32 Vassar St, Cambridge, MA 02139, USA, May 2008a. http://www.w3.org/TR/2008/WD-wai-age-literature-20080514/

  3. Arch, A.: Web accessibility for older users: A literature review. W3C. http://www.w3.org/TR/wai-age-literature (2008b)

  4. Aula, A., Kaki, M.: Less is more in web search interfaces for older adults. First Monday (online), 7(10), July 2005. http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/issue/view/186

  5. Brajnik, G.: Web accessibility testing: when the method is the culprit. In: Proceedings of ICCHP 2006 (2006)

  6. Brajnik, G.: A comparative test of web accessibility evaluation methods. In: Assets ‘08: Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility, Halifax, Canada, Oct 2008. ACM, New York

  7. Brajnik, G.: Barrier walkthrough: heuristic evaluation guided by accessibility barriers. http://www.dimi.uniud.it/giorgio/projects/bw/bw.html (2009)

  8. Brajnik, G.: Validity of web accessibility guidelines. In: Assets ‘09: Proceedings of the 11th International ACM SIGACCESS Conference on Computers and Accessibility, Pittsburgh, USA. ACM, New York (2009)

  9. Brajnik, G., Lomuscio, R.: Samba: a semi-automatic method for measuring barriers of accessibility. In: Assets ‘07: Proceedings of the 9th International ACM SIGACCESS Conference on Computers and Accessibility, pp. 43–50, New York, NY, USA, 2007. ACM, New York (2009). ISBN: 978-1-59593-573-1. http://doi.acm.org/10.1145/1296843.1296853

  10. Brajnik, G., Yesilada, Y., Harper, S.: Guideline aggregation: web accessibility evaluation for older users. In: W4A ‘09: Proceedings of the 2009 International Cross-Disciplinary Conference on Web Accessibility (W4A), pp. 127–135, New York, NY, USA. ACM, New York (2009)

  11. Brajnik, G.: Beyond conformance: the role of accessibility evaluation methods. In: WISE ‘08: Proceedings of the 2008 International Workshops on Web Information Systems Engineering, pp. 63–80 (2008)

  12. Chadwick-Dias, A., McNulty, M., Tullis, T.: Web usability and age: how design changes can improve performance. In: CUU ‘03: Proceedings of the 2003 Conference on Universal Usability, pp. 30–37, New York, NY, USA. ACM, New York (2003). ISBN: 1-58113-701-X. http://doi.acm.org/10.1145/957205.957212. http://portal.acm.org/citation.cfm? doid = 957205.957212

  13. Chadwick-Dias, A., Bergel, M., Tullis, T.: Senior surfers 2.0: a re-examination of the older web user and the dynamic web. In: Universal Access in Human Computer Interaction: Coping with Diversity—UAHCI 2007: 4th International Conference on Universal Access in Human-Computer Interaction, vol. 4554/2007 of Lecture Notes in Computer Science, pp. 868–876. Springer, Berlin/Heidelberg (2007). 10.1007/978-3-540-73279-2\s\do5(9)7

  14. Chisnell, D., Lee, A., Redish, J.: Designing Web Sites for Older Users: Comparing AARP’s Studies to Earlier Findings. Online Publication, March 2004. http://www.aarp.org/olderwiserwired/oww-features/Articles/a2004-03-03-comparison-studies.html (2004)

  15. Cockton, G., Woolrych, A.: Understanding inspection methods: lessons from an assessment of heuristic evaluation. In: Blandford, A., Vanderdonckt, J., Gray, P. (eds.) People & Computers XV, pp. 171–192. Springer, New York (2001)

  16. Pernice Coyne, K., Nielsen, J.: Web usability for senior citizens: 46 design guidelines based on usability studies with people age 65 and older. Nielsen Norman Group Report (2002)

  17. Czaja, S.J., Chin Lee, C.: The impact of aging on access to technology. Univers. Access Inf. Soc. 5(4), 341–349 (2007)

    Article  Google Scholar 

  18. Czaja, S.J., Sharit, J.: Ability-performance relationships as a function of age and task experience for a data entry task. J. Exp. Psychol. 4(4), 332–351 (1998)

    Article  Google Scholar 

  19. Czaja, S.J., Sharit, J., Ownby, R., Roth, D.L., Nair, S.: Examining age differences in performance of a complex information search and retrieval task. Psychol. Aging 16(4), 564–579 (2001). ISSN: 0882-7974

    Google Scholar 

  20. Disability Rights Commission (DRC): The web: access and inclusion for disabled people. (2004)

  21. Fairweather, P.: How older and younger adults differ in their approach to problem solving on a complex website. In: Assets ‘08: Proceedings of the 10th International ACM SIGACCESS Conference on Computers and Accessibility, pp. 67–72, New York, NY, USA, Oct 2008. ACM, New York. ISBN: 978-1-59593-976-0. http://doi.acm.org/10.1145/1414471.0401

  22. Fisk, D., Rogers, W.A., Charness, N., Czaja, S.J., Sharit, J.: Designing for Older Adults. CRC press, Boca Raton (2004)

    Book  Google Scholar 

  23. Fozard, J.: Chapter Vision and hearing in aging. In: Birren, J.E., Schaie, W. (eds.) The Handbook of the Psychology of Ageing, pp. 150–170. Academic, New York (1990)

  24. Gray, W., Salzman, M.: Damaged merchandise: a review of experiments that compare usability evaluation methods. Human–Comput. Interact. 13(3), 203–261 (1998)

    Article  Google Scholar 

  25. Hartley, A.A.: Chapter 1: Attention. In: Craik, F.I.M., Salthouse, T.A. (eds.) The Handbook of Aging and Cognition, pp. 3–53. Laurence Erlbaum Associates, Hillsdale (1992). ISBN: 0–8058-0713-6

  26. Hartson, H.R., Andre, T.S., Williges, R.C.: Criteria for evaluating usability evaluation methods. Int. J. Human–Comp Interact 15(1), 145–183 (2003)

    Article  Google Scholar 

  27. Hawthorn, D.: Possible implications of aging for interface designers. Interact. Comput. 12(5), 507–528 (2000). ISSN: 0953-5438. doi: 10.1016/S0953-5438(99)00021-1

  28. Hawthorn, D.: How Universal Is Good Design for Older Users? In: ACM SIGCAPH Computers and the Physically Handicapped, 73–74(June–Sept), pp. 38–45 (2002). ISSN: 0163-5727. http://doi.acm.org/10.1145/960201.0407

  29. Henry, S.: Just Ask: Integrating Accessibility Throughout Design. Georgia Tech Research Corporation (2004)

  30. Hertzum, M., Jacobsen, N., Molich, R.: Usability inspections by groups of specialists: perceived agreement in spite of disparate observations. In: CHI ‘02 extended abstracts on Human factors in computing systems, pp. 662–663. ACM, New York (2002). ISBN: 1-58113-454-1. http://doi.acm.org/10.1145/506443.506534

  31. Hodes, R.J., Lindberg, D.A.B.: Making your website senior friendly. Technical report, National Institute on Aging, Building 31, Room 5C27 31 Center Drive, MSC 2292 Bethesda, MD 20892, USA, Oct 2002. http://www.nlm.nih.gov/pubs/staffpubs/od/ocpl/agingchecklist.html

  32. Kelly, B., Sloan, D., Brown, S., Seale, J., Petrie, H., Lauke, P., Ball, S.: Accessibility 2.0: people, policies and processes. In: W4A ‘07: Proceedings of the 2007 International Cross-Disciplinary Conference on Web Accessibility (W4A), pp. 138–147, New York, NY, USA. ACM Press, New York (2007). ISBN: 1-59593-590-X. http://doi.acm.org/10.1145/1243441.1243471

  33. Kurniawan, S.H.: Ageing. In: Harper, S., Yesilada, Y. (eds.) Web Accessibility: A Foundation for Research, Human–Computer Interaction Series, Chap. 5, pp. 47–58, 1st edn. Springer, London (2008). ISBN: 978-1-84800-049-0. http://dx.doi.org/10.1007/978-1-84800-050-6\s\do5(5)

  34. Lunn, D., Yesilada, Y., Harper, S.: Barriers faced by older users on static web pages: criteria used in the barrier walkthrough method. Technical Report 108, University of Manchester–School of Computer Science. http://hcw-eprints.cs.manchester.ac.uk/108/(2009)

  35. Lustig, C., Hasher, L., Tonev, S.T.: Inhibitory control over the present and the past. Eur. J. Cogn. Psychol. 13(1&2), 107–122 (2001). 10.1080/09541440126215. http://www.informaworld.com/10.1080/09541440126215

  36. Meyer, B., Sit, R.A., Spaulding, V.A., Mead, S.E., Walker, N.: Age group differences in world wide web navigation. In: CHI ‘97: CHI ‘97 extended abstracts on Human factors in computing systems, pp. 295–296, New York, NY, USA. ACM, New York (1997). ISBN: 0-89791-926-2. http://doi.acm.org/10.1145/1120212.1120401

  37. Milne, S., Dickinson, A., Carmichael, A., Sloan, D., Eisma, R., Gregor, P.: Are guidelines enough? An introduction to designing web sites accessible to older people. IBM Syst. J. 44(3), 557–571 (2005)

    Article  Google Scholar 

  38. Mitchell, J., Adkins, R., Kemp, B.: The effects of aging on employment of people with and without disabilities. Rehabil. Couns. Bull. 49(3), (2006)

  39. Molich, R., Bevan, N., Curson, I., Butler, S., Kindlund, E., Miller, D., Kirakowski, J.: Comparative evaluation of usability tests. In: Proceedings of the Usability Professionals Association Conference (1998)

  40. Morris, A., Goodman, J., Brading, H.: Internet use and non-use: views of older adults. Univers. Access Inf. Soc. 61(1), 43–57 (2007)

    Article  Google Scholar 

  41. Newell, F., Dickinson, A., Smith, M.J., Gregor, P.: Designing a portal for older users: a case study of an industrial/academic collaboration. ACM Trans. Comput. Human Interact. (TOCHI) 13(3), 347–375 (2006). ISSN: 1073-0516. http://doi.acm.org/10.1145/1183456.1183459. http://portal.acm.org/citation.cfm? doid = 1183456.1183459

  42. Nichols, T., Rogers, W., Fisk, A., West, L.: How old are your participants? An investigation of age classifications as reported in human factors. In: Human Factors and Ergonomics Society Annual Meeting Proceedings, Aging, pp. 260–261 (2001)

  43. Nielsen, J.: Ten usability heuristics. Alertbox (2005)

  44. Pan, B., Hembrooke, H.A., Gay, G.K., Granka, L.A., Feusner, M.K., Newman, J.K.: The determinants of web page viewing behavior: an eye-tracking study. In ETRA ‘04: Proceedings of the 2004 Symposium on Eye Tracking Research & Applications, pp. 147–154, New York, NY, USA. ACM, New York (2004). ISBN: 1-58113-825-3. http://doi.acm.org/10.1145/968363.968391. http://portal.acm.org/citation.cfm? id = 968391

  45. Perlmutter, M., Mitchell, D.B.: Chapter 7: The appearance and disappearance of age differences in adult memory. In: Craik, F.I.M (ed.) Aging and Cognitive Processes. Advances in the Study of Communication and Affect, pp. 127–144. Plenum, New York (1982). ISBN: 0-3064-09461

  46. R Development Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria (2008). http://www.R-project.org. ISBN 3-900051-07-0

  47. Rayner, K.: Eye movements in reading and information processing: 20 years of research. Psychol. Bull. 124(3), 327–422 (1998)

    Article  Google Scholar 

  48. Rogers, W.A., Fisk, A.D.: Chapter 11: Understanding the Role of Attention in Cognitive Aging Research. In: Birren, J.E., Schaie, K.W. (eds.) The Handbook of Psychology and Aging, pp. 267–287, 5th edn. Academic Press Inc, New York (2001)

  49. Sears, A.: Heuristic walkthroughs: finding the problems without the noise. Int. J. Human–Comput. Interact. 9, 213–234 (1997)

    Article  Google Scholar 

  50. Sloan, D., Dickinson, A.: Accessibility statements are inaccessible to older web users. In: HCI, the Web and the Older Population—British HCI 2006 (2006)

  51. Tobias, C.A.: Computers and the elderly: a review of the literature and directions for future research. In: Proceedings of the Human Factors Society 31st Annual Meeting, pp. 866–870, Santa Monica, CA, 1987

  52. Trochim, W.M.: The research methods knowledge base. http://www.socialresearchmethods.net/kb/(2006)

  53. Tullis, T.S.: Older adults and the web: lessons learned from eye-tracking. In: Stephanidis, C. (ed.) Universal Access in Human Computer Interaction. Coping with Diversity, vol. 4554, pp. 1030–1039. Springer, New York (2007). ISBN: 978-3-540-73278-5. 10.1007/978-3-540-73279-2\s\do5(1)15

  54. Vigo, M., Kobsa, A., Arrue, M., Abascal, J.: User-tailored web accessibility evaluations. In: Turner, S., Turner, P. (eds.) HyperText 2007, pp. 95–104, Manchester, UK, Sept 2007. ACM, New York

  55. Woolrych, A., Cockton, G.: Assessing heuristic evaluation: mind the quality, not just the percentages. In: Proceedings of HCI 2000, pp. 35–36 (2000)

  56. Yesilada, Y., Brajnik, G., Harper, S.: How much does expertise matter? A barrier walkthrough study with experts and non-experts. In: Proceedings of the 11th ACM Conference on Computers and Accessibility (2009)

  57. Yesilada, Y., Jay, C., Stevens, R., Harper, S.: Validating the use and role of visual elements of web pages in navigation with an eye-tracking study. In: 17th International World Wide Web Conference, Beijing, China (2008)

  58. Yesilada, Y., Brajnik, G. and Harper, S.: A barrier walkthrough study with expert and non-expert judges. HCW-RIAM Technical Report 4.1, March 2009. University of Manchester, School of Computer Science (2009). http://hcw-eprints.cs.man.ac.uk/view/subjects/riam.html

  59. Zajicek, M.: Web 2.0: hype or happiness? In: W4A ‘07: Proceedings of the 2007 International Cross-Disciplinary Conference on Web Accessibility (W4A), pp. 35–39, New York, NY, USA. ACM, New York (2007). ISBN: 1-59593-590-X. http://doi.acm.org/10.1145/1243441.0384

  60. Zhao, H.: Universal usability web design guidelines for the elderly (Age 65 and older). Technical report, The University of Maryland, Department of Computer Science, College Park, Maryland, 20742, USA, April 2001. http://www.co-bw.com/DMS_Web_the_elderly_on_the_web.htm

Download references

Acknowledgments

This work is part of a collaboration between the UK EPSRC funded RIAM project (EP/E002218/1), and the University of Udine. As such the authors would like to thank both organizations for their continued support. We would also like to thank all our participants for their valuable effort and time (320 h in total). We would like to thank also two anonymous reviewers for the quality of their reviews and help in improving the paper. Statistical analysis was performed using the R system [46]. The data of this experiment are publicly available; see [58] for details.

Author information

Authors and Affiliations

  1. Dipartimento di Matematica e Informatica, Università di Udine, Udine, Italy

    Giorgio Brajnik

  2. Middle East Technical University, Northern Cyprus Campus, Guzelyurt, Mersin 10, Turkey

    Yeliz Yesilada

  3. School of Computer Science, University of Manchester, Manchester, UK

    Simon Harper

Authors
  1. Giorgio Brajnik
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yeliz Yesilada
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Simon Harper
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Giorgio Brajnik.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Brajnik, G., Yesilada, Y. & Harper, S. Web accessibility guideline aggregation for older users and its validation. Univ Access Inf Soc 10, 403–423 (2011). https://doi.org/10.1007/s10209-011-0220-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10209-011-0220-5

Keywords

Search

Navigation