Skip to main content
SpringerLink
Log in

A model-based framework for classifying and diagnosing usability problems

  • Original Article
  • Published:
Cognition, Technology & Work Aims and scope Submit manuscript

Abstract

A great deal of study has been devoted to the problem of how to identify and categorize usability problems; however, there is still a lack of studies dealing with the problem of how to diagnose the causes of usability problems and how to feed them back into design process. The value of classifying usability problems can be enhanced when they are interpreted in connection with design process and activities. Thus, it is necessary to develop a systematic way of diagnosing usability problems in terms of design aspects and applying diagnosis results to improve design process and activities. With this issue in mind, this paper proposes a conceptual framework that supports a systematic classification and diagnosis of usability problems. This paper firstly reviews seven approaches to classifying usability problems. Then, we point out the needs of adopting a model-based approach to classifying and diagnosing usability problems and of developing a comprehensive framework guiding the use of model-based approaches. We then propose a conceptual framework that specifies how a model-based classification and diagnosis of usability problems should be conducted and suggests the combined use of three different types of models, each of which addresses context of use, design knowledge and design activities. Last, we explain how a sound classification scheme of usability problems can be systematically developed, and how the classification of usability problems can be connected to design process and activities on the basis of the framework.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Andre T, Hartson H, Belz S, McCreary F (2001) The user action framework: a reliable foundation for usability engineering support tools. Int J Hum Comput Stud 54(1):107–136

    Article  MATH  Google Scholar 

  • Artman H, Zällh S (2005) Finding a way to usability: procurement of a taxi dispatch system. Cognit Technol Work 7(3):141–155

    Article  Google Scholar 

  • Bekker MM, Baauw E, Barendregt W (2008) A comparison of two analytical evaluation methods for educational computer games for young children. Cognit Technol Work 10(2):129–140

    Article  Google Scholar 

  • Bevan N (1999) Quality in use: meeting user needs in quality. J Syst Softw 49(1):89–96

    Article  Google Scholar 

  • Blandford A, Butterworth R, Curzon PP (2004) Models of interactive systems: a case study on programmable user modelling. Int J Hum Comput Stud 60(2):149–200

    Article  Google Scholar 

  • Boy G (1998) Cognitive function analysis. Ablex Publishing Corporation, Stamford

    Google Scholar 

  • Capra MG, Smith-Jackson TL (2005) Developing guidelines for describing usability problems. Technical Report #ACE/HCI-2005-002, Virginia Tech

  • Card D (1998) Learning from our mistakes with defect causal analysis. IEEE Softw 15(1):56–63

    Article  Google Scholar 

  • Chillarge R, Bhandari I, Chaar J, Halliday M, Moebus D, Ray B, Wong M (1992) Orthogonal defect classification-a concept for in-process measurements. IEEE Trans Softw Eng 18(11):943–956

    Article  Google Scholar 

  • Fernandez A, Insfran E, Abrahäo S (2011) Usability evaluation methods for the web: a systematic mapping study. Inf Softw Technol 53(8):789–817

    Article  Google Scholar 

  • Freimut B (2001) Developing and using defect classification schemes. IESE-Report No. 072.01/E, Fraunhofer IESE

  • Freudenthal A, Mook HJ (2003) The evaluation of an innovative intelligent thermostat interface: universal usability and age differences. Cognit Technol Work 5(1):55–66

    Google Scholar 

  • Gero JS, Kannengiesser U (2004) The situated function-behaviour-structure framework. Des Stud 25(4):373–391

    Article  Google Scholar 

  • Gillan D, Bias R (2001) Usability science I: foundations. Int J Hum Comput Interact 13(4):351–372

    Article  Google Scholar 

  • Hassenzahl M (2000) Prioritizing usability problems: data-driven and judgement-driven severity estimates. Behav Inf Technol 19(1):29–42

    Article  Google Scholar 

  • Heo J, Ham D-H, Park S, Song C, Yoon WC (2009) A framework for evaluating the usability of mobile phones based on multi-level, hierarchical model of usability factors. Interact Comput 21(4):263–275

    Google Scholar 

  • Hollnagel E (1993) The phenotype of erroneous action. Int J Man Machine Stud 39(2):1–32

    Article  Google Scholar 

  • Hollnagel E (1998) Cognitive reliability and error analysis method. Elsevier, Oxford

    Google Scholar 

  • Hornbæk K (2006) Current practice in measuring usability: challenges to usability studies and research. Int J Hum Comput Stud 64(2):79–102

    Article  Google Scholar 

  • Hornbæk K (2009) Dogmas in the assessment of usability evaluation methods. Behav Inf Technol 29(1):97–111

    Article  Google Scholar 

  • Hornbæk K, Frøkjær E (2005) Comparing usability problems and redesign proposals as input to practical systems development. In: Proceedings of ACM conference on human factors in computing systems, pp 391–400

  • Howarth J, Andre T, Hartson R (2007) A structured process for transforming usability data into usability information. J Usability Stud 3(1):7–23

    Google Scholar 

  • Huber J (1999) A comparison of IBM’s orthogonal defect classification to Hewlett Packard’s defect origins, types, and modes. Hewlett Packard Company, Palo Alto

    Google Scholar 

  • ISO/IEC (1998) Ergonomic requirements for office work with visual display terminal-part 11: guidance on usability, ISO/IEC 9241

  • Ivory M, Hearst M (2001) The state of the art in automating usability evaluation of user interfaces. ACM Comput Surv 33(4):470–516

    Article  Google Scholar 

  • Jin BS, Ji YG, Choi K, Cho G (2009) Development of a usability evaluation framework with quality function deployment: from customer sensibility to product design. Hum Factors Ergon Manuf Serv Ind 19(2):177–194

    Article  Google Scholar 

  • Keehan S, Hartson H, Kafura D, Schulman R (1999) The usability problem taxonomy: a framework for classification and analysis. Empir Softw Eng 4(1):71–104

    Article  Google Scholar 

  • Kruchten P (2005) Casting software design in the function-behaviour-structure framework. IEEE Softw 22(2):52–58

    Article  Google Scholar 

  • Kwon G, Ham D-H, Yoon WC (2007) Evaluation of software usability using scenarios organized by abstraction structure. In: Proceedings of ECCE 2007, pp 19–22

  • Lavery D, Cockton G, Atkinson M (1997) Comparison of evaluation methods using structured usability problem reports. Behav Inf Technol 16(4–5):246–266

    Article  Google Scholar 

  • Law E (2004) Proposal for a new COST action 294: towards the maturation of IT usability evaluation, cost office

  • Leszak M, Perry D, Stoll D (2002) Classification and evaluation of defects in a project retrospective. J Syst Softw 61(3):173–187

    Article  Google Scholar 

  • Lin C-H, Hwang S-L, Wang EM-Y, Pen S-L (2009) Design for usability on supply chain management systems implementation. Hum Factors Ergon Manuf Serv Ind 19(5):378–403

    Article  Google Scholar 

  • Lindgaard G (2006) Notions of thoroughness, efficiency, and validity: are they valid in HCI practice? Int J Ind Ergon 36(12):1069–1074

    Article  Google Scholar 

  • Naikar N, Moylan A, Pearce B (2006) Analysing activity in complex systems with cognitive work analysis: concepts, guidelines and case study for control task analysis. Theor Issues Ergon Sci 7(4):371–394

    Article  Google Scholar 

  • Nielsen J (1993) Usability engineering. AP Professional, San Diego

    MATH  Google Scholar 

  • Pahl G, Beitz W, Feldhusen J, Grote K-H (2007) Engineering design: a systematic approach, 3rd edn. London, Springer

    Book  Google Scholar 

  • Peuple JL, Scane R (2004) User interface design. Crucial, Exeter

    Google Scholar 

  • Rasmussen J (1985) The role of hierarchical knowledge representation in decision making and system management. IEEE Trans Syst Man Cybern 15(2):234–243

    Article  Google Scholar 

  • Rogers Y, Sharp H, Preece J (2011) Interaction design: beyond human-computer interaction, 3rd edn. Wiley, Chichester

    Google Scholar 

  • Ryu H, Monk A (2004) Analysing interaction problems with cyclic interaction theory: low-level interaction walkthrough. Psychnol J 2(3):304–330

    Google Scholar 

  • Saffer D (2007) Designing for interaction: creating smart applications and clever devices. New Riders, Berkeley

    Google Scholar 

  • Savioja P, Norros L (2012) Systems usability framework for evaluating tools in safety-critical work. Cognit Technol Work. doi:10.1007/s10111-012-0224-9

  • Schoeffel R (2003) The concept of product usability: a standard to help manufacturers to help consumers. ISO Bull March: 5–7

  • Shneiderman B, Plaisant C (1998) Designing the user interface: strategies for effective human-computer interaction, 5th edn. Pearson Higher Education, Boston

    Google Scholar 

  • Springett M (1998) Linking surface error characteristics to root problems in user-based evaluation studies. In: Proceedings of AVI 1998, pp 102–113

  • Strawderman L, Koubek R (2008) Human factors and usability in service quality measurement. Hum Factors Ergon Manuf Serv Ind 18(4):454–463

    Article  Google Scholar 

  • Te’eni D, Carey J, Zhang P (2007) Human-computer interaction: developing effective organizational information systems. Wiley, Hoboken

    Google Scholar 

  • Tullis T, Albert B (2008) Measuring the user experience. Morgan Kaufmann, Burlington

    Google Scholar 

  • Vermeeren APOS, Attema J, Akar E, Ridder H, von Doorn AJ, Erbu C, Berkman AE, Maquire MC (2008) Usability problem reports for comparative studies: consistency and inspectability. Hum Comput Interact 23(4):329–380

    Article  Google Scholar 

  • Vicente KJ (2002) Ecological interface design: progress and challenge. Hum Factors 44(1):62–78

    Article  MathSciNet  Google Scholar 

  • Vilbergsdottir S, Hvannberg E, Law E (2006) Classification of usability problems (CUP) scheme: augmentation and exploitation. In: Proceedings of NordiCHI 2006, pp 281–290

  • Zhang Z (2011) Overview of usability evaluation methods. http://www.usabilityhome.com. Accessed Oct 10, 2011

Download references

Acknowledgments

This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education, Science and Technology (Grant Code: 2012042146).

Author information

Authors and Affiliations

  1. Department of Industrial Engineering, Chonnam National University, 77 Yongbong-ro, Buk-gu, Gwangju, 500-757, South Korea

    Dong-Han Ham

Authors
  1. Dong-Han Ham
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dong-Han Ham.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ham, DH. A model-based framework for classifying and diagnosing usability problems. Cogn Tech Work 16, 373–388 (2014). https://doi.org/10.1007/s10111-013-0267-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10111-013-0267-6

Keywords

Search

Navigation