Skip to main content
SpringerLink
Log in

A systematic examination of universal design resources: part 1, heuristic evaluation

  • Long Paper
  • Published:
Universal Access in the Information Society Aims and scope Submit manuscript

Abstract

This paper presents the evaluation of eight published Universal Design Resources (UDRs) to measure how effectively they support typical design processes and design psychology. New heuristics and principles to evaluate the UDRs from the point of view of designers who were universal design novices were created. Established methodologies for heuristic evaluation were used with the new heuristics. The evaluators found numerous problems in seven of the eight UDRs, providing evidence in support of the hypothesis that the content UDRs does not facilitate the design process and is not commensurate with what is known about typical design psychology.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. ITTATC’s list was avaialble at the time of writing, but a note on their site states that their funding expired in May 2006 (http://www.ittatc.org).

  2. See URL in acknowledgments section for the UDiP website with information on all papers relating to this project.

  3. In future studies under the UDiP project we plan to investigate how designers utilize multiple resources.

  4. There were a number of iterations of the list of heuristics. We conducted pilot studies with a higher number of heuristics. The final list is as described herein. We employed the same format of Nielsen’s list of heuristics, with a title followed by a short paragraph of explanatory text (see Appendix A).

References

  1. ABA : Industry Standards on Accessibility of Electronic Banking. Available: http://www.bankers.asn.au/ (2003)

  2. Architectural and Transportation Barriers Compliance Board: Telecommunication Act accessibility guidelines. Available: http://www.access-board.gov/telecomm/rule.htm (1998)

  3. Architectural and Transportation Barriers Compliance Board: Electronic and Information Technology Accessibility Standards. Available: http://www.access-board.gov/sec508/standards.htm (2000)

  4. Ball, L., Evans, B., Dennis, I.: Cognitive processes in engineering design: a longitudinal study. Ergonomics 37(11), 1753–1786 (1994)

    Article  Google Scholar 

  5. Barker T.T.: Writing Software Documentation: A Task-oriented Approach. Allyn & Bacon, Boston (1998)

    Google Scholar 

  6. Bonnardel, N., Summer, T.: Supporting evaluation in design. Acta Psychologica 91, 221–244 (1996)

    Article  Google Scholar 

  7. Buchanan R.: Wicked problems in design thinking. In: Margolin V., Buchanan R. (eds) The Idea of Design. pp. 3–20. MIT, Cambridge (1995)

    Google Scholar 

  8. Burns, C.M., Vicente, K.J., Christoffersen, K., Pawlak, W.S.: Towards viable, useful and usable human factors design guidance. Appl. Ergon. 28(5/6), 311–322 (1997)

    Article  Google Scholar 

  9. Carrol, J.M.: Reconstructing minimalism. In: Carrol J.M.: (eds) Minimalism beyond the Nurnberg Funnel. MIT, Cambridge (1998)

    Google Scholar 

  10. Cassim, J., Dong, H.: Critical users in design innovation. In: Clarkson, J., Coleman, R., Keates, S., Lebbon, C. (eds) Inclusive Design: Design for the Whole Population. pp. 533–553. Springer, London (2003)

    Google Scholar 

  11. Connel, B., Jones, M., Mace, R., Mueller, J., Mullick, A., Ostroff, E., Sanford, J., Steinfield, E., Story, M., Vanderheiden, G.: Principles of Universal Design. Available: http://www.design.ncsu.edu/cud/about_ud/udprincipleshtmlformat.html (1997)

  12. Cooper, A.: The Inmates Are Running the Asylum: Why High Tech Products Drive Us Crazy and How To Restore The Sanity. SAMS (1999)

  13. COST219 Accessibility for All to Services and Terminals for Next Generation Networks. Available: http://www.tiresias.org/cost219ter/ (2003)

  14. CUD Universal Design Education Online. Available: http://www.udeducation.org/ (2003)

  15. Dahl, D.W., Moreau, P.: The influence and value of analogical thinking during new product ideation. J. Mark. Res. 39(1), 47–60 (2002)

    Article  Google Scholar 

  16. HHRC The Inclusive Design Education Resource. Available: http://www.designcouncil.info/inclusivedesignresource/ (2003)

  17. INDA Accessibility Guidelines for Public Access Terminals. Available: http://accessit.nda.ie/it-accessibility-guidelines/public-access-terminals (2003)

  18. Kantner, L., Shroyer, R., Rosenbaum, S.: Structured heuristic evaluation of online documentation. Presented at IEEE International Professional Communication Conference (IPCC), Portland, USA, (2002)

  19. Keates, S., Lebbon, C., Clarkson, J.: Investigating industry attitudes to Universal Design. Presented at Rehabilitation Engineering and assistive technology Society of North America (RESNA), Annual Conference, Orlando, USA, (2000)

  20. Koubek, R.J., Benysh, D., Buck, M., Harvey, C.M., Reynolds, M.: The development of a theoretical framework and design tool for process usability assessment. Ergonomics 46(1–3), 220–241 (2003)

    Article  Google Scholar 

  21. Law, C.M., Vanderheiden, G.C.: EZ Access strategies for cross-disability access to kiosks, telephones and VCRs. Presented at CSUN’98 (California State University at Northridge) Conference on Technology and Persons with Disabilities, Los Angeles, USA, (1998)

  22. Lawson, B.: How designers think: the design process demystified. 3rd. Architectural press (1997)

  23. Lebbon, C., Coleman, J.: A designer-centred approach. In: Clarkson, P.J., Coleman, R., Keates, S., Lebbon, C. (eds.) Inclusive Design: Design for the Whole Population. pp. 500–518. Springer, Heidelberg (2003)

  24. Lidwell W., Holden K., Butler J.: Universal Principles of Design: A Cross-disciplinary Reference. Rockport Publishers, Gloucester (2003)

    Google Scholar 

  25. Lindblom, C.E.: The science of “muddling through”. Public Admin. Rev. 19(2), 79–88 (1959)

    Article  Google Scholar 

  26. Mehlenbacher, B.: Documentation: not yet implemented, but coming soon! In: Jacko, J.A., Sears, A. (eds.) The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies and Emerging Applications. Lawrence Erlbaum Associates (2003)

  27. Nicolle, C., Abascal, J.: Inclusive Design Guidelines for Human-Computer Interaction. Taylor & Francis, London (2001)

    Google Scholar 

  28. Nielsen, J.: Usability Engineering. Academic, London (1993)

    MATH  Google Scholar 

  29. Nielsen J.: Heuristic Evaluation. In: Nielsen J., Mack R.L. (eds.) Usability inspection methods. Wiley, London (1994)

    Google Scholar 

  30. Norman, D.A.: The design of everyday things. 1st Doubleday/Currency. Doubleday (1990)

  31. Papanek, V.: Design for the Real World. Random House, New York (1971)

    Google Scholar 

  32. Petroski H.: Invention by design: how designers get from thought to thing. Harvard University Press, Cambridge (1996)

    Google Scholar 

  33. Potok A.: A Matter of Dignity: Changing the World of the Disabled. Bantam, New York (2002)

    Google Scholar 

  34. Purho, V.: Heuristic inspection for documentation: 10 recommended documentation heuristics. STC Usability SIG Newsletter 6(4) (2000)

  35. Roe, P.: Bridging the gap. COST 219 (Cooperation in Science and Technology #219, Telecommunications: Access for Disabled and Elderly People), Available: http://www.tiresias.org/phoneability/bridging_the_gap/ (2001)

  36. Royal National Institute of the Blind: Which Button? Available: http://www.tiresias.org/publications/controls/ (2001)

  37. Royal National Institute of the Blind: Access-Ability: Making technology more useable by people with disabilities. Available: http://www.tiresias.org/guidelines/access-ability/ (2004)

  38. Schriver, K.A.: Dynamics in document design. Wiley, London (1997)

    Google Scholar 

  39. Schulz, Y.: Fuzzy Goals Lead to Foundering Projects. Computing Canada, September 13 (1995)

  40. Stephanidis, C., Akoumianakis, D., Vernardakis, N., Emiliani, P.L., Vanderheiden, G., Ekberg, J., Ziegler, J., Faehnrich, K.P., Galetsas, A., Haataja, S., Iakovidis, I., Kemppainen, E., Jenkins, P., Korn, P., Maybury, M., Murphy, H.J., Ueda, H.: Industrial Policy Issues. In: Stephanidis, C. (ed.) User interfaces for all: Concepts, Methods, and Tools. Lawrence Erlbaum Associates (2001)

  41. Stephanidis, C., Salvendy, G.: Toward an Information Society for All: An International R&D Agenda. Int. J. Hum. Comput. Interact. 10(2), 107–134 (1998)

    Article  Google Scholar 

  42. Stewart, T., Travis, D.: Guidelines, Standards, and Style Guides. In: Jacko, J.A., Sears, A. (eds.) The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies and Emerging Applications. Lawrence Erlbaum Associates (2003)

  43. Tobias, J., Vanderheiden, G., Vanderheiden, K.: Why companies might adopt universal design: a report from the Universal Design Research Project. Presented at RESNA (Rehabilitation Engineering and assistive technology Society of North America) Annual Conference, Minneapolis (1998)

  44. Van der Meij H., Carrol J.M.: Principles and heuristics for designing minimalist instruction. In: Carrol J.M. (ed.) Minimalism beyond the Nurnberg Funnel. MIT, Cambridge (1998)

    Google Scholar 

  45. Vanderheiden, G.C.: Thirty-something million: should they be exceptions? Hum. Factors 32(4), 383–396 (1990)

    Google Scholar 

  46. Vicente, K.J., Burns, C.M., Pawlak, W.S.: Muddling through wicked design problems. Ergon. Des. 5(1), 25–30 (1997)

    Google Scholar 

  47. W3C (World Wide Web Consortium) Web Accessibility Initiative, Web content accessibility guidelines 1.0. Available: http://www.w3.org/TR/WCAG10/ (1999)

  48. W3C (World Wide Web Consortium) Web Accessibility Initiative, Web content accessibility guidelines 2.0 Working Draft. Available: http://www.w3.org/TR/WCAG20/ (2006)

  49. Wixon, D.: Evaluating usability methods: why the current literature fails the practitioner. Interactions 10(4), 28–34 (2003)

    Article  Google Scholar 

Download references

Acknowledgments

The authors wish to express their thanks to Jimmy Ginn for his comments on a draft of this paper, Erin Kinzel for her contribution to the heuristic evaluation, and to the members of the project’s advisory committee who provided comments on the project plans.

This work is part of the UDiP Project in the Wallace H. Coulter, Department of Biomedical Engineering at the Georgia Institute of Technology funded by the National Institute on Disability and Rehabilitation Research (NIDRR), US Department of Education (grant number: H133G040151). The opinions and content are those of the grantees. They do not necessarily represent the policy of the Department of Education. More information about the UDiP Project can be found on the project website at http://www.hsi.gatech.edu/cise/udip/ or http://udprojects.org/

Author information

Authors and Affiliations

  1. Department of Information Systems, UMBC (University of Maryland, Baltimore County), 1000 Hilltop Circle, Baltimore, MD, 21250, USA

    Chris M. Law

  2. Health Systems Institute, Georgia Institute of Technology, 313 Ferst Drive NW, Atlanta, GA, 30332, USA

    Ji Soo Yi, Young Sang Choi & Julie A. Jacko

  3. School of Business Information Technology, RMIT University, Level 3, Building 112, 255 Bourke Street, Melbourne, VIC, 3001, Australia

    Chris M. Law

Authors
  1. Chris M. Law
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ji Soo Yi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Young Sang Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Julie A. Jacko
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chris M. Law.

Appendix A: Full set of heuristics and sub-points

Appendix A: Full set of heuristics and sub-points

A.1 Principle 1. Address the pertinent product design aspects

A.1.1 Heuristic 1.1. A coherent vision, purpose and central idea should be provided

The “central idea” can be regarded as the main purpose of the resource. The principle motivations, methods to be employed, and reasoning behind the creation of the resource should be clearly stated. If the central idea is not conveyed clearly, then it can be easy to misinterpret the aims of the authors.

  1. 1.1.1

    There should be a coherent vision and purpose conveyed/stated as a starting point

  2. 1.1.2

    The “central idea” should be clear and it should be easy to refer back to (return to) at any time

  3. 1.1.3

    The central idea should contain the message that the aim is to improve the design of products for use by people with disabilities and/or people experiencing functional limitations

A.1.2 Heuristic 1.2. End-user and product goals should be addressed

The purpose of the product’s existence for the end-user is to help them achieve specific goals. It is sometimes easy to overlook the main tasks and instead focus on the components of products and this can make it difficult for readers/designers to be creative in problem solving.

  1. 1.2.1

    The goals and needs of end-users should be clear

  2. 1.2.2

    The design solutions should be organized around the human goals of the end-users

  3. 1.2.3

    The focus should be on interaction design which is goal oriented. Component based design which is task oriented is not as likely to meet end-user needs.

  4. 1.2.4

    The requirements for the designer should suggested as to satisfy the goals as far as is practical

  5. 1.2.5

    The readers should know when they have achieved the goals of the end-users. Measurable stopping rules should therefore be provided. If the goals are “fuzzy” or the stated problems are “wicked” then this means they are inherently difficult to measure and therefore the designer will not know when they have got the solution right. Performance-based measures (of the end-users) which relate to the goals are useful.

A.1.3 Heuristic 1.3. Potential end-user errors and failure scenarios should be addressed

Some end-users will inevitably make mistakes and will need help with any interface. The guidance in the resource should address errors that might be made by users and suggest methods to appropriately deal with those errors.

  1. 1.3.1

    Do the guidelines address the consequences of interaction failures and their consequences for users (i.e., no interface is perfect, so error scenarios must be considered)

  2. 1.3.2

    User methods for troubleshooting should be addressed

  3. 1.3.3

    For consumer products, are user manuals and user help addressed? Are the user manuals and help mechanisms accessible to people with disabilities?

A.1.4 Heuristic 1.4. The factors in product development that are beyond the domain of the designer should be considered

The reader/designer will be only one part of the whole team that is responsible for producing any given product. Resources should include advice on how to incorporate other pertinent product development factors into the overall design of the product.

  1. 1.4.1

    Is the larger system and considerations which constrain the designer (/design team) accepted and addressed?

A.2 Principle 2. Support the design process and design psychology

A.2.1 Heuristic 2.1. Action oriented approaches should be supported and encouraged for readers

The design process is seldom straightforward and eludes reduction, and no one design approach or process can yield the complete answer. However, what is clear about designers is that they are action oriented, and this type of approach should be supported in the way the resource is designed.

  1. 2.1.1

    Guidance should be oriented to the tasks of the readers and should be anchored in the task domain

  2. 2.1.2

    There should be an immediate opportunity to act (the reader’s first job should involve less reading and more acting)

  3. 2.1.3

    Readers should feel in control over their own actions and be encouraged to explore and innovate

  4. 2.1.4

    For Initial Learners (readers who are looking at the guidance for the very first time), the document should support “reading to do” (i.e., the readers primarily want to get some job/task done). Therefore, the sequence/flow of actions/concepts should be logical and easy to follow.

  5. 2.1.5

    Many readers will “jump the gun” and skip over introductions. The guidance should make sense if readers employ “opportunistic” reading as they search for relevant information (i.e., a way that is not planful).

A.2.2 Heuristic 2.2. Inevitable trade-off decision-making should be supported

In any design project, there will be design trade-off decisions needed due to limitations. Trouble typically occurs when one (design) factor dominates all of the others (for example, aesthetics, build quality, cost, ease of use, or accessibility). A consideration of balance of various factors is necessary, and this should be explicitly stated/addressed in the guidance.

  1. 2.2.1

    Can readers act successfully when there are a lot of trade-offs to worry about (i.e., can they return to the “central idea”

  2. 2.2.2

    Is the focus on what is possible to do (a common design trap) or what is probable within the limits of the real-world (i.e., more realistic and practical)?

A.2.3 Heuristic 2.3. Designers should be allowed to bring past experience into a prescriptive problem-solving activity

Scientists normally describe how things work whereas designers normally prescribe how things will work. This is an important distinction. The purpose of the resource should not be to only relay specifications (e.g., say what size the buttons need to be), but also to persuade and influence designers to carry out their prescriptive design activities in a manner that incorporates access for people with disabilities.

  1. 2.3.1

    Are the designers allowed to prescribe? Designers want to do the prescriptive actions involved in design, and apply their own way of thinking, rather than be prescribed to.

  2. 2.3.2

    Problem-solving methods should be supported. Most designers are problem solvers rather than people who are seeking to understand problems. Readers should not be expected to understand the problems (lengthy explanations of the problems). If lengthy problem explanations are provided, they should be easy to skip over or included in an appendix of referenced document.

  3. 2.3.3

    The explanation of why something should be done should follow the explanation of how something should be done.

  4. 2.3.4

    Is there an allowance for the clustering of problems and sub-problems? The reader should be able to easily prioritize design actions based on the guidance.

A.2.4 Heuristic 2.4. Commonly employed self-referential viewpoints of readers should be actively countered

A common mistake that all people make is thinking that other people are mostly like themselves. With designers, this self-referential approach can be dangerous and lead to interfaces that are usable for only the user who can behave like a designer. If this view is allowed, the fundamentals of interaction become lost on the typical user as the number of functions and modes increases, and greater likelihood of user error is also expected. This common behavior should be actively discouraged.

  1. 2.4.1

    Designers will naturally be self-referential in the process (i.e., they will think of themselves as end-users in making design decisions concerning interaction). This design trap can be alleviated by proper consideration of user testing and other human factors/evaluation processes.

A.3 Principle 3. Design the document/resource effectively

A.3.1 Heuristic 3.1. The design of the document should be clear and appealing

Designing end-user manuals and documentation requires properly addressing a number of criteria, such as style, grammar, use of white space, illustrations, and so on.

  1. 3.1.1

    The document should be interesting to read and easy to understand. Perhaps it can also be entertaining and in places humorous (but humor should be used only minimally)

  2. 3.1.2

    The typography

  3. 3.1.3

    The structure of the information should be supported by good use of modularization/chunking

  4. 3.1.4

    The document should not be unnecessarily verbose or use unusually long sentences

  5. 3.1.5

    There should be a clear statement of authority (who was responsible for the production and enforcement of the guidance)

  6. 3.1.6

    The number of concepts should be appropriate and not burdensome to the reader

  7. 3.1.7

    The chance that designers can misinterpret or make errors in using the guidance should be minimized?

A.3.2 Heuristic 3.2. Different types of readers, and different usage over time should be supported

Guidelines and standards documents should be readable for first-time learners and also for readers who are coming back to them for reference. The time-lag for returning readers from the initial learning may be extensive, so it should not be assumed that the reader will remember all of the pertinent information that is not contained within any given subsection of the resource. The design of the document, with proper indexing and cross-referencing, should facilitate use as both an introduction and a reference tool.

  1. 3.2.1

    The content should be easily navigable. Exploration of the content should be encouraged.

  2. 3.2.2

    Direction to further assistance should be given (if the reader/designer needs more in-depth information on certain topics covered by the guidelines).

  3. 3.2.3

    Examples, demonstrations, overviews, explanations or case studies should be designed to facilitate learning.

  4. 3.2.4

    The document should be accessible to readers with disabilities (low vision, blindness, physical disabilities etc.)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Law, C.M., Yi, J.S., Choi, Y.S. et al. A systematic examination of universal design resources: part 1, heuristic evaluation. Univ Access Inf Soc 7, 31–54 (2008). https://doi.org/10.1007/s10209-007-0100-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10209-007-0100-1

Keywords

Search

Navigation