Skip to main content
SpringerLink
Log in

Contrasting usability evaluation methods with blind users

  • Long paper
  • Published:
Universal Access in the Information Society Aims and scope Submit manuscript

Abstract

Usability tests are a part of user-centered design. Usability testing with disabled people is necessary, if they are among the potential users. Several researchers have already investigated usability methods with sighted people. However, research with blind users is insufficient, for example, due to different knowledge on the use of assistive technologies and the ability to analyze usability issues from inspection of non-visual output of assistive devices. From here, the authors aspire to extend theory and practice by investigating four usability methods involving the blind, visually impaired and sighted people. These usability methods comprise of local test, synchronous remote test, tactile paper prototyping and computer-based prototyping. In terms of effectiveness of evaluation and the experience of participants and the facilitator, local tests were compared with synchronous remote tests and tactile paper prototyping with computer-based prototyping. Through the comparison of local and synchronous remote tests, it has been found that the number of usability problems uncovered in different categories with both approaches was comparable. In terms of task completion time, there is a significant difference for blind participants, but not for the visually impaired and sighted. Most of the blind and visually impaired participants prefer the local test. As for the comparison of tactile paper prototyping and computer-based prototyping, it has been revealed that tactile paper prototyping provides a better overview of an application while the interaction with computer-based prototypes is closer to reality. Problems regarding the planning and conducting of these methods as they arise in particular with blind people were also discussed. Based on the authors’ experiences, recommendations were provided for dealing with these problems from both the technical and the organization perspectives.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

Notes

  1. Fidelity means how similar are the produced prototypes to the final product.

References

  1. Mayhew, D.: The Usability Engineering Lifecycle—A practitioner’s Handbook For User Interface Design. Morgan Kaufmann, San Francisco (1999). ISBN 1558605614

    Google Scholar 

  2. Miao, M., Köhlmann, W., Schiewe, M., Weber, G.: Tactile paper prototyping with blind subjects. In: Haptic and Audio Interaction Design (HAID), Dresden, pp. 81–90. Springer, Berlin, Heidelberg (2009)

  3. Magnusson, C., Brewster, S. (eds.): Guidelines for Haptic Lo-Fi Prototyping. Workshop, NordiCHI 2008, Lund, Sweden, 19 Now. (2008)

  4. Rudd, J., Stern, K., Isensee, S.: Low vs. high-fidelity prototyping debate. In: Interactions 3, Nr. 1, pp. 76–85 (1996)

  5. ISO 9241-171: Ergonomics of Human-System Interaction—Part 171: Guidance on Software Accessibility (2009)

  6. Web Content Accessibility Guidelines (WCAG) 2.0, W3C Recommendation 11 December 2008. http://www.w3.org/TR/WCAG20/

  7. Virzi, R.A., Sokolov, J.L., Karis, D.: Usability problem identification using both low- and high-fidelity prototypes. In: CHI’96: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 236–243. New York, USA (1996)

  8. Miriam, W., James, A.L.: High-fidelity or low-fidelity, paper or computer? Choosing attributes when testing web prototypes. In: Proceedings of the Human Factors and Ergonomics Society, pp. 661–665 (2002)

  9. Petrie, H., Hamilton, F., King, N. et al.: Remote usability evaluation with disabled people. In: Proceedings of CHI, pp. 1133–1141. ACM Press (2006)

  10. Hartson, H.R., Castillo J.C., Kelso, J., Kamler, J., Neale, W.C.: Remote evaluation: the network as an extension of the usability laboratory. In: Proceedings of CHI Conference on Human Factors in Computing Systems ACM, pp. 228–235. New York (1996)

  11. Selvaraj, P.: Comparative Study of Synchronous Remote and Traditional In-Lab Usability Evaluation Methods. Master of Science Thesis. The Virginia Polytechnic Institute and State University (2004)

  12. Hartson, H.R., Castillo, J.C.: Remote evaluation for post-deployment usability improvement. Proceedings of the Working Conference on Advanced Visual Interfaces, pp. 22–29. L’Aquila, Italy (1998)

    Chapter  Google Scholar 

  13. Brush, A.J.B., Ames, M., Davis, J.: A comparison of synchronous remote and local usability studies for an expert interface. Proceeding of CHI, pp. 1179–1182. Austria, Vienna (2004)

    Google Scholar 

  14. Johnson, J.: Don’ts and Do’s for Software Developers and Web Designers, 1st edn. Morgan Kaufmann (2000)

  15. Nielsen, J.: Usability Engineering. Morgen Kaufmann (1993)

  16. Sefelin, R.: Comparison of Paper- and Computer Based Low-Fidelity Prototypes/Center for Usability Research & Engineering (CURE). Research report (2002)

  17. Nielsen, J., Loranger, H.: Prioritizing Web Usability. New Riders Press, Berkeley (2006)

    Google Scholar 

Download references

Acknowledgments

The authors would like to thank the participants and collaborators for spending their time on this study.

Author information

Authors and Affiliations

  1. Department of Computer Science, Technische Universität Dresden, Nöthnitzer Straße 46, 01187, Dresden, Germany

    Mei Miao, Hoai Anh Pham, Jens Friebe & Gerhard Weber

Authors
  1. Mei Miao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hoai Anh Pham
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jens Friebe
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gerhard Weber
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mei Miao.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Miao, M., Pham, H.A., Friebe, J. et al. Contrasting usability evaluation methods with blind users. Univ Access Inf Soc 15, 63–76 (2016). https://doi.org/10.1007/s10209-014-0378-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10209-014-0378-8

Keywords

Search

Navigation