Understanding technology adoption in clinical care: Clinician adoption behavior of a point-of-care reminder system

doi:10.1016/j.ijmedinf.2005.03.007

International Journal of Medical Informatics

Volume 74, Issues 7–8, August 2005, Pages 535-543

https://doi.org/10.1016/j.ijmedinf.2005.03.007 Get rights and content

Summary

Background:

Evaluation studies of clinical decision support systems (CDSS) have tended to focus on assessments of system quality and clinical performance in a laboratory setting. Relatively few studies have used field trials to determine if CDSS are likely to be used in routine clinical settings and whether reminders generated are likely to be acted upon by end-users. Moreover, such studies when performed tend not to identify distinct user groups, nor to classify user feedback.

Aim:

To assess medical residents’ acceptance and adoption of a clinical reminder system for chronic disease and preventive care management and to use expressed preferences for system attributes and functionality as a basis for system re-engineering.

Design of study:

Longitudinal, correlational study using a novel developmental trajectory analysis (DTA) statistical method, followed by a qualitative analysis based on user satisfaction surveys and field interviews.

Setting:

An ambulatory primary care clinic of an urban teaching hospital offering comprehensive healthcare services. 41 medical residents used a CDSS over 10 months in their daily practice. Use of this system was strongly recommended but not mandatory.

Methods:

A group-based, semi-parametric statistical modeling method to identify distinct groups, with distinct usage trajectories, followed by qualitative instruments of usability and satisfaction surveys and structured interviews to validate insights derived from usage trajectories.

Results:

Quantitative analysis delineates three types of user adoption behavior: “light”, “moderate” and “heavy” usage. Qualitative analysis reveals that clinicians of distinct types tend to exhibit views of the system consistent with their demonstrated adoption behavior. Drawbacks in the design of the CDSS identified by users of all types (in different ways) motivate a redesign based on current physician workflows.

Conclusion:

We conclude that this mixed methodology has considerable promise to provide new insights into system usability and adoption issues that may benefit clinical decision support systems as well as information systems more generally.

Section snippets

Summary of research results

What was known before the study?

•
Clinical decision support systems can enhance the clinical performance in a variety of areas.
•
Many CDSS fail in real-life installations regardless of the technological quality of the application.
•
Research on the effects of CDSS on user adoption and organization workflows in realistic settings is lacking.

What the study has added to the body of knowledge?

•
Introduced a new statistical methodology drawn from the social sciences for analyzing revealed behavior versus

Clinical reminder system

Clinical reminder system (CRS) uses patients’ medical status data to provide “just-in-time” reminders to clinicians at the point of the care consistent with the latest evidence-based medicine guidelines for chronic disease and preventive care management. CRS is made available to physicians and clinic staff via desktop computers installed in every examination room of the clinic. The application integrates the hospital's administrative, laboratory, and clinical records systems into a single

Distinct user adoption groups

Our application of DTA identified three distinct user adoption types. The developmental trajectories and the group compositions are depicted in Fig. 2. Solid and dashed lines denote the observed and predicted trends, respectively. Observed data values are computed as the mean use rate of users assigned to each of these groups identified by estimation. Predicted values are computed using DTA model coefficient estimates. Numbers above each trendline denote the percentage of residents that

Methods

The mixed methods model we have used in this paper has enabled us to successfully distinguish distinct groups of CRS users, to validate the results of statistical analysis with actual user comments, and to finally identify key user concerns that have enabled system redesign towards more efficient and more effective use.

Results

Comments and related system data related to iterative advisories indicate that information gathering and reminder-generating functions of the system, particularly in situations

Conclusion

In this study, we assess clinician users’ acceptance and adoption of a clinical reminder system. We apply a mixed methods approach that combines quantitative methods to identify distinct developmental trajectories of user adoption with qualitative instruments to examine the causal processes of such adoption behaviors.

We find that a significant level of resistance exists to the use of the reminder system; a large proportion of users demonstrated a consistently low or decreasing level of usage

References (21)

W.M. Tierney et al.
Computerizing guidelines to improve care and patient outcomes: the example of heart failure
J. Am. Med. Inform. Assoc.
(1995)
E.A. Balas et al.
Improving preventive care by prompting physicians
Arch. Intern. Med.
(2000)
D.L. Hunt et al.
Effects of computer-based clinical decision support systems on physician performance and patient outcomes
JAMA
(2000)
I. Sim et al.
Clinical decision support systems for the practice of evidence-based medicine
JAMIA
(2001)
C.L. McDonald
Protocol-based computer reminders, the quality of care, and the non-perfectability of man
N. Engl. J. Med.
(1975)
C.P. Friedman et al.
Enhancement of clinicians’ diagnostic reasoning by computer-based consultation: a multisite study of 2 systems
JAMA
(1999)
R.M. Gardner et al.
Evaluation of user acceptance of a clinical expert system
JAMIA
(1994)
B. Kaplan
Evaluating informatics applications—some alternative approaches: theory, social interactionism and call for methodological pluralism
Int. J. Med. Inf.
(2001)
B. Kaplan
Evaluating informatics applications—clinical decision support systems literature review
Int. J. Med. Inf.
(2001)
N. Rousseau et al.
Practice based, longitudinal, qualitative interview study of computerized evidence-based guidelines in primary care
Br. Med. J.
(2003)

There are more references available in the full text version of this article.

Cited by (78)

An application of activity theory to the “problem of e-books”
2020, Heliyon
Citation Excerpt :
In this research, Sun found that students were reluctant to adopt a technical tool when they experienced a low sense of control or cognitive harmony which is supported by the findings of this investigation. When investigating the adoption of technical products into environments that require a high level of cognitive work, the impact of insufficient support for the processes is frequently found to contribute to user resistance (Bodker, 1996; Rundo et al., 2020; Zheng et al., 2005). To address this issue, Sun (2016) recommended redesign and better support of technical innovations and it is proposed that innovative product design or re-engineering of e-book technology to address the requirements of the academic work domain may result in a far superior fit as a student-centric product.
The “problem of e-books” is defined as the difficulty improving the adoption rates of e-books by students. The adoption rates of e-books for academic use remain low, and research into the reasons for this have resulted in inconclusive findings. Factors such as student perception, and variations in experimental methodology and technology, contribute to difficulties in generalising findings and establishing conclusive causes for this problem. To better understand the causal factors for low adoption rates and the student's experience with ereaders and digital text, an investigation was conducted by the lead researcher as a student enrolled in a postgraduate course. The experiment was designed using e-book and digital text documents on an ereader for academic study and the results analysed with the framework of Activity Theory. This methodology allowed exploration of the problem within the authentic experience of a student to examine the effects of this social environment on ereader and e-book use.
Analysis of the work domain was conducted and a comparative assessment of the observed effect of using the digital documents on an ereader compared with the paper book. Findings show that attempts to apply self-regulation and metacognitive learning techniques within the activity using the ereader were abandoned due to breakdowns in operations, and that this resulted in a perceived lower quality of achievement. The effect on the processes used by the student were extreme and were observed to be highly dependent on the student's use of specific learning strategies. The experimental methodology employed in this investigation enabled identification of the role of the social environment in the use of course documents on an ereader for academic study. The functionality of the ereader was such an extremely poor fit with the observed academic processes that a redesign approach for ereader and e-book technology is proposed as a solution to the low adoption rates of e-books.
Factors influencing implementation success of guideline-based clinical decision support systems: A systematic review and gaps analysis
2017, International Journal of Medical Informatics
Citation Excerpt :
Concerning the dimension of System use, reported factors most often concerned ‘expectations and beliefs’ (n = 32), ‘knowledge and expertise’ (n = 32), ‘motivation to use’ (n = 13) and ‘training’ (n = 12) [6,19–31]. Of particular concern are physician beliefs that the guideline-based CDSS use will hamper the patient-physician relationship [6,21,32–34], or might disturb the patient or clinical work [32,35,36]. They fear that computer use during the patient encounter would adversely affect communication between the patient and clinician [21,34].
To provide an integrated and differentiated understanding of factors influencing guideline-based CDSS implementation and illustrate the gaps in the current literature.
A systematic literature review and in-depth exploration of factors impeding or facilitating successful implementation of guideline-based CDSS supporting physicians in clinical decision-making was performed. Factors were identified thematically by textual analysis of the included publications and were individually mapped to the human, organization and technology-fit (HOT-fit) framework for evaluating implementations of health information systems.
A total of 421 factors were found in 35 included publications from a total of 3676 publications. The mapping of factors concerning CDSS implementation on the HOT-fit framework revealed gaps in each domain of the framework and showed that research has mainly focused on human and technology factors and less on organizational factors.
Future research within the field of guideline-based CDSS should focus on evaluating implementations through the use of socio-technical models to study guideline-based CDSS system implementations from a multidimensional view. Furthermore, research is needed to explore whether use of these models during the planning phases of a CDSS project is useful in anticipating and preventing implementation barriers from occurring and exploiting facilitators to a successful implementation of the system.
A systematic review of mixed methods research on human factors and ergonomics in health care
2015, Applied Ergonomics
This systematic literature review provides information on the use of mixed methods research in human factors and ergonomics (HFE) research in health care. Using the PRISMA methodology, we searched four databases (PubMed, PsycInfo, Web of Science, and Engineering Village) for studies that met the following inclusion criteria: (1) field study in health care, (2) mixing of qualitative and quantitative data, (3) HFE issues, and (4) empirical evidence. Using an iterative and collaborative process supported by a structured data collection form, the six authors identified a total of 58 studies that primarily address HFE issues in health information technology (e.g., usability) and in the work of healthcare workers. About two-thirds of the mixed methods studies used the convergent parallel study design where quantitative and qualitative data were collected simultaneously. A variety of methods were used for collecting data, including interview, survey and observation. The most frequent combination involved interview for qualitative data and survey for quantitative data. The use of mixed methods in healthcare HFE research has increased over time. However, increasing attention should be paid to the formal literature on mixed methods research to enhance the depth and breadth of this research.
A case study evaluation of a Critical Care Information System adoption using the socio-technical and fit approach
2015, International Journal of Medical Informatics
Clinical information systems have long been used in intensive care units but reports on their adoption and benefits are limited. This study evaluated a Critical Care Information System implementation.
A case study summative evaluation was conducted, employing observation, interview, and document analysis in operating theatres and 16-bed adult intensive care units in a 400-bed Malaysian tertiary referral centre from the perspectives of users (nurses and physicians), management, and information technology staff. System implementation, factors influencing adoption, fit between these factors, and the impact of the Critical Care Information System were evaluated after eight months of operation.
Positive influences on system adoption were associated with technical factors, including system ease of use, usefulness, and information relevancy; human factors, particularly user attitude; and organisational factors, namely clinical process-technology alignment and champions. Organisational factors such as planning, project management, training, technology support, turnover rate, clinical workload, and communication were barriers to system implementation and use. Recommendations to improve the current system problems were discussed. Most nursing staff positively perceived the system's reduction of documentation and data access time, giving them more time with patients. System acceptance varied among doctors. System use also had positive impacts on timesaving, data quality, and clinical workflow.
Critical Care Information Systems is crucial and has great potentials in enhancing and delivering critical care. However, the case study findings showed that the system faced complex challenges and was underutilised despite its potential. The role of socio-technical factors and their fit in realizing the potential of Critical Care Information Systems requires continuous, in-depth evaluation and stakeholder understanding and acknowledgement. The comprehensive and specific evaluation measures of the Human–Organisation–Technology Fit framework can flexibly evaluate Critical Care Information Systems.
Organizational decision to adopt hospital information system: An empirical investigation in the case of Malaysian public hospitals
2015, International Journal of Medical Informatics
This study mainly integrates the mature Technology-Organization-Environment (TOE) framework and recently developed Human-Organization-Technology (HOT) fit model to identify factors that affect the hospital decision in adopting Hospital Information System (HIS).
Accordingly, a hybrid Multi-Criteria-Decision-Making (MCDM) model is used to address the dependence relationships of factors with the aid of Analytic Network Processes (ANP) and Decision Making Trial and Evaluation Laboratory (DEMATEL) approaches. The initial model of the study is designed by considering four main dimensions with 13 variables as organizational innovation adoption factors with respect to HIS. By using DEMATEL, the interdependencies strength among the dimensions and variables are tested. The ANP method is then adopted in order to determine the relative importance of the adoption factors, and is used to identify how these factors are weighted and prioritized by the public hospital professionals, who are wholly familiar with the HIS and have years of experience in decision making in hospitals’ Information System (IS) department.
The results of this study indicate that from the experts’ viewpoint “Perceived Technical Competence” is the most important factor in the Human dimension. In the Technology dimension, the experts agree that the “Relative Advantage” is more important in relation to the other factors. In the Organization dimension, “Hospital Size” is considered more important rather than others. And, in the Environment dimension, according to the experts judgment, “Government Policy” is the most important factor. The results of ANP survey from experts also reveal that the experts in the HIS field believed that these factors should not be overlooked by managers of hospitals and the adoption of HIS is more related to more consideration of these factors. In addition, from the results, it is found that the experts are more concerned about Environment and Technology for the adoption HIS.
The findings of this study make a novel contribution in the context of healthcare industry that is to improve the decision process of innovation in adoption stage and to help enhance more the diffusion of IS in the hospital setting, which by doing so, can provide plenty of profits to the patient community and the hospitals.
Complexity Analysis of a Business Process Automation: Case Study on a Healthcare Organization
2015, Procedia Computer Science
Healthcare organizations have been struggling to get Business Process Management (BPM) and associated Information and Communication Technologies (ICT) properly aligned to improve their patients’ service and quality of care. Nevertheless, the highly structured nature of larger organizations such as hospitals hampers this alignment, and commonly ICT is applied to isolated tasks or fragments of processes. In this paper, we present and discuss the results, in terms of complexity, of the introduction of a new scheduling system within the medical appointment and exam business processes of a large hospital. During the case study, we began by modelling the processes using the Business Process Modelling and Notation (BPMN) standard. We then used abstract metrics to compare the complexity between old (before the introduction of the scheduling system) and new processes, and interpreted the obtained results. Finally, we derived important conclusions that will help guide us in further business process optimization endeavors.

View all citing articles on Scopus

View full text

Understanding technology adoption in clinical care: Clinician adoption behavior of a point-of-care reminder system

Summary

Background:

Aim:

Design of study:

Setting:

Methods:

Results:

Conclusion:

Section snippets

Summary of research results

Clinical reminder system

Distinct user adoption groups

Methods

Results

Conclusion

Computerizing guidelines to improve care and patient outcomes: the example of heart failure

J. Am. Med. Inform. Assoc.

Improving preventive care by prompting physicians

Arch. Intern. Med.

Effects of computer-based clinical decision support systems on physician performance and patient outcomes

JAMA

Clinical decision support systems for the practice of evidence-based medicine

JAMIA

Protocol-based computer reminders, the quality of care, and the non-perfectability of man

N. Engl. J. Med.

Enhancement of clinicians’ diagnostic reasoning by computer-based consultation: a multisite study of 2 systems

JAMA

Evaluation of user acceptance of a clinical expert system

JAMIA

Evaluating informatics applications—some alternative approaches: theory, social interactionism and call for methodological pluralism

Int. J. Med. Inf.

Evaluating informatics applications—clinical decision support systems literature review

Int. J. Med. Inf.

Practice based, longitudinal, qualitative interview study of computerized evidence-based guidelines in primary care

Br. Med. J.