Hanna Yehuda
Jennifer McGinn
ABSTRACT
In this paper we describe a new way to perform heuristic evaluations, which allows multiple evaluators to easily compare and combine the results of their reviews. This method was developed to provide a single, reliable, result to the client, but it also allowed us to easily negotiate differences in our findings, and to prioritize usability problems identified by the evaluation. An unexpected side effect is that, by using this evaluation method, the practitioner can measure and predict the effect of usability improvements.
- Cockton, G., Lavery, D., Woolrych, A. Inspection--Based Evaluations. In Jacko, J. A., & Sears, A. (Eds) (2003), The Human--Computer Interaction Handbook (pp. 1118--1137). Mahwah, NJ: Lawrence Erlbaum Associates. Google Scholar
Digital Library
- Dumas, J. & Redish G. (1999). A practical guide to usability testing (Revised Ed.) (pp. 322--326). London: Intellect Books. Google ScholarDigital Library
- Hertzum, M., Jacobsen, N. E., & Molich, R. (2002). Usability Inspection by groups of specialists: perceived agreement in spite of disparate observations. Proceeding of CHI 2002, (pp. 662). Minneapolis: ACM Press. Google ScholarDigital Library
- Jacobsen, N. E., Hertzum, M., & John B. E. (1998). The evaluator effect in usability studies: Problem detection and severity judgments, Proceedings of the Human Factors and Ergonomics Society (pp. 1336--1340).Google ScholarCross Ref
- Molich, R. & Nielsen, J. (1990). Improving a human--computer dialogue: what designers know about traditional interface design. Communications of the ACM 33, 3. Google ScholarDigital Library
- Nielson, J. (1994), Heuristic evaluation, in Nielsen, J. & Mack, R. (Eds.) Usability inspection methods, (pp. 25--62), NY: John Wiley and Sons, Inc. Google ScholarDigital Library
- Nielsen, J. (1993). Usability engineering (pp. 103--104). San Diego: Morgan Kauffman. Google ScholarDigital Library
- Nielsen, J., & Molich, R. (1990). Heuristic evaluation of user interfaces, Proceeding of CHI '90 (pp. 241--256). New York: ACM Press. Google ScholarDigital Library
- Nielsen, J., & Landauer, T. K. (1993). A mathematical model of the finding of usability problems, Proceedings ofINTERCHI '93 (pp. 206--213). Google ScholarDigital Library
- Rubin, J. (1994). Handbook of usability testing (pp. 277--280). Portland: John Wiley and Sons.Google Scholar
- Stanton, N., & Baber, C. (1996), Factors affecting the selection of methods and techniques prior to conducting a usability evaluation, in Jordan, P. W., Thomas, B., Weerdmeester, B. A, & McClelland, I. L. (Eds.) Usability evaluation in industry, (pp. 39--48), Bristol, PA: Taylor & Francis.Google Scholar
- Wilson, C. (1999). Reader's questions: Severity scales. Usability Interface, Volume 5, Number 4.Google Scholar
- Wilson, C. & Coyne, K. (2001). Tracking usability issues: To bug or not to bug? Interactions (May--June). Google ScholarDigital Library
- Yahoo UI Design Library, Retrieved October 19th, 2006 from http://developer.yahoo.com/ypatterns/pattern.php?pattern=ratinganobjectGoogle Scholar
Index Terms
- Coming to terms: comparing and combining the results of multiple evaluators performing heuristic evaluation
Recommendations
Usability Heuristics for Transactional Web Sites
ITNG '14: Proceedings of the 2014 11th International Conference on Information Technology: New GenerationsOne of the most recognized methods to evaluate usability in software applications is the heuristic evaluation. In this inspection method, Nielsen's heuristics, are the most widely used evaluation instrument. However, there is evidence in the literature ...
Evaluating Severity Rating Scales for Heuristic Evaluation
CHI EA '16: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing SystemsThe heuristic evaluation is a widely applied discount usability evaluation method. Experts use the method to identify usability issues in interfaces and to rate their severity in order to establish a prioritization of resource allocation. However, in ...
You say "disaster", i say "no problem": unusable problem rating scales
CHI EA '13: CHI '13 Extended Abstracts on Human Factors in Computing SystemsThis paper documents a consistent and considerable evaluator effect in the ratings of usability problem severity carried out by experienced usability professionals. The CUE-9 study, conducted in 2011, showed a reasonable level of rater agreement in ...
Comments