Elsevier

Computers in Human Behavior

Volume 54, January 2016, Pages 577-586
Computers in Human Behavior

Full length article
Improving physicians' performance with a stroke CDSS: A cognitive fit design approach

https://doi.org/10.1016/j.chb.2015.07.054Get rights and content

Highlights

  • CDSS design guidelines are developed based on cognitive fit theory, SOAP, and CPGs.

  • The guidelines fit physicians' mental models in the problem-solving process.

  • CDSS designed with the guidelines improves physicians' assessment performance.

  • CDSS designed with the guidelines reduces cognitive effort and load.

  • The design guidelines are useful in helping practitioners develop effective CDSSs.

Abstract

A large body of evidence over many years suggests that clinical decision support systems (CDSSs) can be helpful in improving both clinical outcomes and adherence to evidence-based guidelines. CDSSs have however failed to show their effectiveness due to poor ease of use and integration within clinical workflows. This research therefore emphasizes a cognitive fit design approach to developing an effective CDSS to solve those issues. According to the cognitive fit theory CDSS should align with the problem and task representation in order to match the physician's mental model to reduce cognitive effort. Several guidelines based on the cognitive fit design are proposed. A stroke CDSS prototype following the design guidelines is developed to demonstrate its feasibility. The system usability test results showed that the developed stroke CDSS was acceptably accurate, able to lessen the cognitive effort as desired, and preferable for use due to the significant reduction in cognitive load. The developed artifact has shown the potential to benefit physicians. Finally, implications and conclusions are discussed.

Introduction

Clinical professionals perform numerous decision-making activities in their interactions with patients. They need to make decisions when diagnosing patients' conditions, when arranging patient treatment plans, and when monitoring patients' evolving health states. Physicians, however, are prone to error due to the complexity of the tasks and/or the mental condition of the decision makers (Barchard & Pace, 2011). A range of information technologies (ITs) have been developed to support physicians over recent decades. One of the most significant IT developments for improving clinical decision making is the clinical decision support system (CDSS), which is a computer system that provides decision-making aids during healthcare processes. Different CDSSs provide a variety of decision support modes, including critical value alerts (Kesselheim, Cresswell, Phansalkar, Bates, & Sheikh, 2011), reminders of overdue preventive health tasks (Garg & Tonelli, 2005), advice for drug prescriptions (Pruszydlo, Walk-Fritz, Hoppe-Tichy, Kaltschmidt, & Haefeli, 2012), critiques of existing healthcare orders (Garg et al., 2005), and suggestions for various medical care issues (Gilmer et al., 2012). Overall, CDSSs generate outputs to support decision makers with the aim of achieving gains in medical performance, narrowing the gaps between knowledge and practice and improving patient safety (Kaplan, 2011).

However, the effects of current CDSSs are still questionable. According to Kellermann and Jones (2013), health IT impacts on healthcare efficiency and safety are inconclusive, while healthcare expenditures have increased significantly (Kellermann & Jones, 2013). In their opinion, such frustrating results can be attributed to the slow adoption of IT systems, the improper design of systems that are neither interoperable nor easy to use, and the failure of process reengineering to fully integrate the systems into clinical practices (Muller et al., 2001, Purcell, 2005, Wyatt, 1997). On the other hand, recent reviews of CDSSs have suggested that CDSSs can improve performance, although they have not been uniformly effective in every aspect. Jaspers, Smeulers, Vermeulen, and Peute (2011) found that CDSSs significantly improved practitioner performance in 57% of studies reviewed (Jaspers et al., 2011). The positive evidence resulted mainly from computer reminder systems for preventive care and computer-assisted drug ordering and dosing systems.

CDSSs that aim to increase physician compliance with recommended assessment and treatment plans do not turn out to be effective because certain issues, such as ease of use to reduce workload and incorporation of integrated and flexible clinical workflows, remain unsolved (Ben-Zvi, 2012, Blackmore et al., 2011, Main et al., 2010, Musen et al., 2014). In fact, information system use is an essential construct that indicates human behavior in IT utilization and successful IS adoption in organizations (Sun & Teng, 2012). Chan (2009) indicated that DSS use increased when perceived ease of use and perceived DSS usefulness were elevated. In order to increase CDSS compliance and use by physicians, from the design science research point of view, a designed artifact (system) requires rigorous processes with theoretical foundations applied in both the construction and evaluation stages. Cognitive fit theory (CFT) (Vessey, 1991) is the fundamental that can be employed in CDSS design to address the aforementioned challenges. CFT deals with problem-solving performance and suggests that problem-solving tools (e.g., CDSSs) should be carefully designed to align with the problem and task representation that take place in the solver's (e.g., the physician's) mental model. If this can be achieved, the problem solver's task performance will be enhanced. CDSS alignment with problem representation helps overcome the issue of information arrangement and representation for ease of use. On the other hand, CDSS alignment with task representation helps deal with the issue of incorporating clinical practice guidelines (CPGs) (Fox, Patkar, Chronakis, & Begent, 2009) and integrating with clinical workflows (Kesselheim et al., 2011, Niazkhani et al., 2009).

The purpose of this research is first to propose a set of design guidelines based on CFT and then to follow the guidelines to design an effective CDSS. Stroke CDSS is chosen in this study because stroke is a major cause of death and permanent disability throughout the world (Mozaffarian et al., 2015). We aim to enhance CDSS ease of use and flexibility in supporting physicians making effective diagnoses and providing proper treatment with less cognitive effort and load.

This paper is organized as follows. In the next section we introduce the fundamental concepts on which our CDSS is designed. In Section 3 we develop a set of CDSS design guidelines that can be used to guide CDSS development. Section 4 describes how we apply the guidelines to develop a stroke CDSS as a demonstration, followed by our testing processes and evaluation results for the stroke CDSS in Section 5. We then address the implications and concluding remarks for this study.

Section snippets

Theoretical foundation

Researchers have successfully proposed design science concepts to make design methodology a major component of research (Peffers, Tuunanen, Rothenberger, & Chatterjee, 2007). Design science research is rooted in engineering and the sciences of artificial intelligence (Simon, 1996). According to Hevner, March, Park, and Ram (2004), any design artifact should rely upon the application of rigorous methods in its construction. In particular, moving from an objective definition to design and

CDSS design guidelines

Following the CFT clinical problem-solving model provides general guidelines for CDSS design with respect to its fit with problem and task representation. First of all, the SOAP process represents the major progression in physicians' problem-solving activities in recording patients' medical diagnosis and progress. Performing this procedure could explicitly develop physicians' mental models. Therefore, the CDSS design should follow the SOAP steps as closely as possible.

We then consider the fit

System development

We developed a stroke CDSS to support physicians' decision making for stroke patients to demonstrate the cognitive fit design approach feasibility. Stroke is a well-known disease that causes rapid loss of brain function through a lack of blood supply. Five million people die from stroke worldwide each year and another five million permanently disabled (Mendis et al., 2005). Stroke has remained the third leading cause of death in Taiwan for a long time and is the leading cause of permanent

Testing processes

After the stroke CDSS was developed, we needed to examine how it performed to evaluate the usability of the cognitive fit design approach. We first invited eight neurology specialists from Kaohsiung Medical University Hospital (a medical center in Taiwan) to join the system usability test for the CDSS. When meeting with these specialists, we introduced the objectives of this CDSS and then illustrated the information and CDSS operation process.

We next prepared eight to 12 medical record test

Implications and conclusions

This study developed a set of cognitive fit design guidelines and implemented the guidelines to develop a clinical decision support system (CDSS) to fit the problem and task representation by enhancing system ease of use and clinical workflow flexibility. The results showed that designing a CDSS with the proposed guidelines can support physicians in making more efficient diagnoses and providing proper treatment with less cognitive effort and load.

The contribution of this research is two-fold.

Acknowledgments

This research is supported by the National Science Council of Taiwan under operating grant NSC102-2410-H-110-035-MY3 and an affiliated grant from National Sun Yat-Sen University and Kaohsiung Medical University in Taiwan: NSYSUKMU 101-030 and NSYSUKMU102-P007. It was also partially supported by the Aim for the Top University Plan of National Sun Yat-Sen University and the Ministry of Education, Taiwan. In addition, we would like to particularly thank Robert A. Greenes, Ira A Fulton Chair and

References (42)

  • J.H. Wu et al.

    A methodology for ERP misfit analysis

    Information and Management

    (2007)
  • J.H. Wu et al.

    Improving searching and reading performance: the effect of highlighting and text color coding

    Information and Management

    (2003)
  • J. Wyatt

    Quantitative evaluation of clinical software, exemplified by decision support systems

    International Journal of Medical Informatics

    (1997)
  • H.P. Adams et al.

    Guidelines for the early management of adults with Ischemic stroke a guideline from the American heart association/American stroke association stroke council, clinical cardiology council, cardiovascular radiology and intervention council, and the atherosclerotic peripheral vascular disease and quality of care outcomes in research interdisciplinary working groups: the American Academy of Neurology affirms the value of this guideline as an educational tool for neurologists

    Circulation

    (2007)
  • R. Agarwal et al.

    Time flies when you're having fun: cognitive absorption and beliefs about information technology usage

    Mis Quarterly

    (2000)
  • B.S. Bloom et al.

    Taxonomy of educational objectives: Handbook I: Cognitive domain

    (1956)
  • J. Fox et al.

    From practice guidelines to clinical decision support: closing the loop

    Journal of the Royal Society of Medicine

    (2009)
  • A.X. Garg et al.

    Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review

    Journal of the American Medical Association

    (2005)
  • A.X. Garg et al.

    The tension between E-health innovation and E-valuation

    Archives of Internal Medicine

    (2005)
  • T.P. Gilmer et al.

    Cost-effectiveness of an electronic medical record based clinical decision support system

    Health Services Research

    (2012)
  • A.R. Hevner et al.

    Design science in information systems research

    MIS Quarterly

    (2004)
  • Cited by (10)

    • The association between interparental conflict and problematic internet use among Chinese adolescents: Testing a moderated mediation model

      2021, Computers in Human Behavior
      Citation Excerpt :

      Given that all the questionnaires were self-reported by adolescents, we employed both exploratory and confirmatory approaches to diagnose whether common method bias would pose a threat to the interpretation of our findings. First, we employed Harman's single factor test to check common method bias (Chang et al., 2016; Podsakoff et al., 2003). The result found that the first factor only explained 20.82% of the variance.

    • Evaluating effectiveness of information visualizations using cognitive fit theory: A neuroergonomics approach

      2020, Applied Ergonomics
      Citation Excerpt :

      The CFT has been validated in a variety of domains including, healthcare (e.g., Chang et al., 2016), finance (e.g., Martin et al., 2019), social network visualization (e.g., Welles and Xu, 2018), quality assurance (e.g., Teets et al., 2010), and emergency management (e.g., Shen et al., 2012). Chang et al. (2016) considered clinical decision making as a set of problem-solving activities in which (1) the problem is represented by subjective responses from a patient and objective data from the patient encounter and (2) the task is represented by assessment of a patient's condition based on consolidated subjective and objective information, and treatment planning. They posited that clinical decision support systems (CDSSs) that align with the problem representation and task representation in order to match clinicians' mental schemas should lead to reduced cognitive effort and enhanced performance.

    • Design of lactoferrin modified lipid nano-carriers for efficient brain-targeted delivery of nimodipine

      2018, Materials Science and Engineering C
      Citation Excerpt :

      NLC was used to load NMD, in order to increase its aqueous solubility or stability, when it experienced intravenous administration into bloodstream. Furthermore, DSPE-PEG2000-COOH was utilized to modify the surface of the NLC(PEG-NLC) to reach long circulation feature and increase its bioavailability, as the surface modification by using PEG or its derivatives was reported to enhance the stability and avoid the elimination from reticuloendothelial system (RES) [17–20]. Last but not least, the carboxyl group on the surface of PEG-NLC could be further reacted with the amine groups of Lf via amidation reaction, which could facilitate the delivery efficacy increase of NMD to brain tissue by receptor-mediated endocytosis.

    • Ontology enhanced fuzzy clinical decision support system

      2018, U-Healthcare Monitoring Systems: Volume 1: Design and Applications
    View all citing articles on Scopus
    View full text