A systematic examination of universal design resources: part 2, analysis of the development process

Abstract

In this paper, the development process of four Universal Design Resources (UDRs) was analyzed. The results of a heuristic evaluation (HE) of UDRs (Part 1) were used in this (Part 2) study to create an online survey. Thirty-one individuals involved in the creation of the four UDRs responded, 15 of whom were also interviewed. For three resources, the hypothesis was confirmed that meeting the needs of end-users was assumed to be satisfied without systematically addressing them. Additional findings also revealed a common lack of a clearly defined “central idea” among many of the committee members of two US-based ICT accessibility/UD guidelines.

Appendices

Appendix A: Survey instrument

The following questions were presented online. Respondents entered a password that was included on the invitation-to-participate email. Not shown are the consent and instruction screens that were included in the survey. Respondents were instructed that they survey should take about 30 min.

There were five pages which required scrolling down the web page. The page heading numbers and corresponding instructions have been included below.

For brevity, formatting has been changed here to reduce the page count (the actual survey used a lot of white space). The text is exactly as it appeared on the survey (except where reformatting has been used for brevity, as indicated by italic text).

A.1 Page 1 of 5: questions about your experience (at the time the resource was being created)

The following questions pertain to your experience and situation at the time that the resource was being created. For resources which that took more than 1 year to create, please use the approximate end-date of the creation of the resource for your answers.

• 1. Your age (at the time) ______

• 2. The country you lived in (at the time) _____

• 3. Your native language ______

If your native language is other than English, how would you rate your proficiency in English (at the time)? □ excellent □ good □ fair □ poor

• 4. Your highest degree earned (at the time)

□ Bachelors (or equivalent) □ Masters (or equivalent)

□ Ph.D. (or equivalent) □ No Reply

• 5. Years of professional experience since finishing your education (at the time) ___

• 6. Your primary domain of work (at the time)

□ Organization of/for people with disabilities (e.g., Non-profit, charity, advocacy group)

□ Industry association (e.g., lobbying, policy, industry consortium)

□ Consulting (including private consulting, or part of a small business consulting firm)

□ Student (attending a university or college)

□ Academic (research or teaching faculty at a university or college)

□ Industry (e.g., manufacturing, service delivery)

□ Government (e.g., policy, regulatory, procurement)

□ Other (specify) —

• 7. Please indicate your level of skill/experience (at the time), for Information and Communications Technologies in the following...

Note: for each of the following, the options were:

• □ None □ Beginner/ Novice □ Experienced □ Expert

• Assistive Technologies (Products and Interfaces for Use by People With Disabilities)

• Universal Design / Design-for-All / Inclusive Design

• Industrial Design (including product design, engineering design etc.)

• Design for Usability, Usability testing, User evaluations

• Interaction Design (Human-Computer Interaction, User Interface Design etc.)

• Instructional Design (Designing Coursework, Product User Manuals, How-to Guides etc.)

• Technical Writing

• Reviewing and Editing of Documentation

• Graphic Design, Typography, Layout etc.

A.2 Page 2 of 5: Questions about the meetings for and the process of development of the resource, including correspondence in between meetings

For the following measures, please indicate your opinion of the nature of meeting(s) you attended, or correspondence in between meetings, leading to the creation of the resource:

• 8. boring □ □ □ □ □ □ □ interesting

• 9. hostile □ □ □ □ □ □ □ friendly

• 10. nonproductive □ □ □ □ □ □ □ productive

• 11. disorganized □ □ □ □ □ □ □ well organized

• 12. painful □ □ □ □ □ □ □ enjoyable

• 13. destructive □ □ □ □ □ □ □ constructive

• 14. argumentative □ □ □ □ □ □ □ agreeable

• 15. emotional □ □ □ □ □ □ □ dispassionate

• 16. conjectural □ □ □ □ □ □ □ factual

If you went to multiple meetings, please indicate your opinion on the following:

• 17. The meetings were slow paced □ □ □ □ □ □ □ The meetings were fast paced

• 18. The meetings were too few in number □ □ □ □ □ □ □ The meetings were too many in number

• 19. The meetings were too far apart in time □ □ □ □ □ □ □ The meetings were too close together in time

• 20. What is your opinion of the group consensus of the following issues, based on the meeting(s) you attended, or from correspondence in between meetings?

Note: for each of the following, the options were:

• □ not discussed □ not needed □ important, but not possible

• □ priority, but done poorly □ priority, and done well □ No Reply

• Promotion of the resource

• Presentation of the resource (style, layout, formatting etc.)

• Accessibility of the resource (to blind, low vision users etc.)

• Need for public comment

• Need for professional editing

• Need for professional technical writing

• Need to evaluate a draft with the potential readers of the resource (e.g., a pilot study with designers)

• 21. The ‘central idea’ can be regarded as the main purpose of the document. The ‘central idea’ would encompass the principal motivations, methods to be employed, and reasoning behind the creation of the resource. In your opinion, what was the ‘central idea’ in the creation of this resource? —

• 22. Do you think others involved in the creation of the resource shared your opinion of the ‘central idea’?

□ It was clear that everyone involved shared this idea

□ Most people shared this idea, but some people had different opinions and ideas

□ There were varied opinions about the central idea

□ Most people were confused or in disagreement about the central idea

□ The central idea was never discussed

□ No Reply

A.3 Page 3 of 5: Questions about the content of the final resource

The questions which follow refer to the final deliverable produced by the committee. If this report was sent to another agency and modified before it became a public resource, consider the following questions with respect to the resource that was named and linked to in the email that was sent to you. In this section, we would like to have your opinions on how useful you think the resource would be only for potential readers of the resource who are involved in developing a product or service.

For each statement, please indicate your level of agreement.

Note: for each of the following statements a Lickert scale response was required using the following terms in this order:

□ strongly agree □ agree □ slightly agree □ neither agree nor disagree

□ slightly disagree □ disagree □ strongly disagree

• 23. A coherent vision, purpose and central idea was provided for the readers.

• 24. The goals of end-users of products and services were described and sufficiently addressed.

• 25. The design modifications that the readers would need to address to make components of the products and/or services meet the needs of the target users were well described and sufficiently addressed.

• 26. Some end-users will inevitably make mistakes and will need to correct those mistakes when using a product. This is true of all users, including people with disabilities.

  The guidance in the resource addressed errors that might be made by users, and methods to appropriately deal with those errors were suggested.

• 27. The reader will usually be only one part of the whole team responsible for producing any given product or service. Other factors such as marketing, engineering, quality control etc., may be outside of the domain of the reader but may influence the success of the proposed design changes. How the reader handles these other factors may be crucial to the success of the interaction design proposed by the design team.

  The resource included sound and useful advice on how to incorporate other factors into the overall design of the product or service.

• 28. The design of the resource would support the typical design tasks that product/service developers would have to do in order to meet the needs of the target users.

• 29. Most product development processes involve incremental changes over time rather than a radical overhaul. Designers should therefore be helped in choosing urgent improvements among many less-urgent improvements.

  The resource supports prioritization and trade-off decision-making by readers when it is not possible to implement all proposed changes at once.

• 30. The design of the resource supports creativity and problem-solving by the reader.

• 31. The need for user testing of people with disabilities and/or functional limitations was highlighted to the readers, and addressed sufficiently.

• 32. The design of the resource was clear, laid out well, and easy to navigate.

• 33. It would be very unlikely that readers could misinterpret the meaning of statements given in the resource.

• 34. For readers who are new to the topics addressed by the resource, it would be easy to read and understand.

• 35. For readers who are using the resource as a reference tool (returning to it many times to answer specific questions), it would be easy to find and understand the necessary information.

A.4 Page 4 of 5: Questions about events since the publication of the resource

The following questions refer to your opinions about the situation between the time the resource was published, and now.

• 36. If the resource produced by the committee/meetings differed in content to the final published resource (e.g. it was modified by another group or agency), what is/are your opinion(s), if any, on the modifications (either positive or negative)? —

• 37. Were you involved in promoting the resource? (check all that apply)

□ At conferences as a presented paper

□ At conferences as a workshop or tutorial

□ At conferences in information booths

□ In articles you wrote

□ As part of invited talks to industry / industry groups

□ As part of invited talks to government groups

□ As part of invited talks to disability groups

□ No, not involved in promoting the resource

□ Other (please explain) —

• 38. What is/are your opinion(s), if any, about how the resource was publicized and promoted by other people (either positive or negative)? —

• 39. Have you had inquiries from people wanting more information about the resource? Note: “inquiries” refers to situations in which someone external to your organization initiated contact with you.

Note: for each of the following the possible responses were: □ None □ 2–3 □ 4–10 □ > 10

• The press, TV Media etc.

• Students

• Academics (faculty, professors etc.)

• Consultants

• Industry

• Government

• Organizations of/for people with disabilities

• 40. (If applicable...) When did the inquiries occur?

□ Only immediately after publication

□ Mostly immediately after publication, the rest mostly spread out since then

□ Spread out evenly since publication

□ Sporadically, intermittently

□ Not Applicable

• 41. Have you had opportunities to work with industry on projects to meet the design aims of the resource? (please approximate if necessary. If none, enter zero)

Number of projects initiated —

Number of projects completed —

Number of projects completed which led to new or improved commercially available products or services —

• 42. In your opinion, were these projects a direct result of the publication of the resource?

□ Yes, they were a direct result of the publication of the resource

□ No, but they were closely related to the publication of the resource

□ No, they had nothing to do with the publication of the resource

□ No Reply

• 43. In your own words, what do you think has changed in the design of information and communications technologies since publication of the resource, in terms of universal/inclusive design and/or accommodating people with disabilities in design? —

• 44. Do you think the resource had anything to do with those changes? Please explain why you think this is the case. —

A.5 Page 5 of 5: Any other comments?

• 45. In your opinion, what was the most important positive influence on the development of the resource? —

• 46. In your opinion, what was the most important negative influence on the development of the resource? —

• 47. What would you change, if anything, to improve the design of the resource? —

• 48. Do you have any other comments, thoughts, or observations concerning this resource, and/or advice you would like to pass on to others who would be initiating such an activity? —

Appendix B: Heuristic evaluation results (Part 1 study) for the four UDRs studied in this (Part 2) study

Table 12