The IT productivity paradox in health: A stakeholder's perspective

https://doi.org/10.1016/j.ijmedinf.2010.11.004Get rights and content

Abstract

Purpose

While information technology (IT) investments in healthcare are higher than ever, there are contradictions in the literature regarding their ability to improve productivity, quality of care and/or healthcare system efficiency. Using theories that can help make sense of these contradictions, we propose a new framework to assess the actual impacts of health information technology (HIT) implementation.

Method

Following an exhaustive literature review, we build upon the ‘productivity paradox’ and ‘stakeholder’ theories to improve the evaluation of IT impacts in healthcare. Interview data from three case studies of HIT implementation in different hospitals were used to develop our proposed framework.

Results

The empirical data analysis suggests that it is important: (1) to identify, account for and accurately measure the appropriate impacts (beneficial/adverse, expected/unforeseen effects); (2) to consider the context of implementation; (3) to adopt a multi-level perspective (individual, group and organization); and (4) to take into account the various stakeholders’ perspectives (managers, health professionals and patients).

Conclusions

An assessment framework was developed to provide general guidance on how to assess HIT impacts. The proposed framework will be useful for researchers and practitioners as it takes into account the underlying reasons for the HIT productivity paradox and identifies the salient outcomes of interest linked to HIT implementation.

Introduction

Investments in information technology (IT) in healthcare are greater than they have ever been, and in most countries, not less than 2.6–6% of health budgets are dedicated to IT [1]. In the United States, as part of the February 2009 economic stimulus package, Congress appropriated more than U.S. $20 billion for health information technology (HIT). In Western Europe, the healthcare sector modernization will generate a growth in IT spendings from U.S. $9 billion in 2006 to U.S. $12 billion in 2011 [2]. In the healthcare sector, HITs are being presented as a means to improve productivity, quality of health and/or system efficiency [3], [4]. However, there is growing concern about the lack of actual positive outcomes that can be directly tied to HIT implementation [5].

On the one hand, the reasons cited in favour of the implementation of new HITs are numerous and diverse, both in the scientific literature and in government reports [6], [7], [8]. HITs are presented as one of the principal ways to improve quality of care [6], [7], [9], [10]. It can improve medical practices, assist the decision-making process by facilitating access to good practice guidelines, simplifying the prescription of diagnostic procedures, and producing reminders [7], [11], [12]. Also, the use of IT in health is said to translate into lower rates of medication errors and adverse drug events [13], [14], [15]. Finally, some studies show that it can lead to greater productivity among professionals, and lower costs [6], [7], [16].

On the other hand, some findings on the actual effectiveness and efficiency of HIT remain inconclusive [5], [17], [18]. Some studies have even suggested that, in some cases, HIT implementations appear to be counter-productive [19]. Moreover, according to the literature in information systems and health informatics, not many HIT implementations have been successful [13], [20], [21], [22], [23].

Faced with contradictory evidence, some researchers, health professionals and hospital administrators have expressed doubts about the actual value of investing time, money and energy in the implementation of new HITs [17], [24], [25], [26]. Being able to explain these contradictions and finding a way to determine the actual impacts of IT on health has therefore become a crucial endeavour that stands to benefit both research and practice.

In this paper, we seek to make sense out of the contradictory findings reported in the literature by providing a comprehensive framework that can be further used to improve the evaluation of IT impacts in healthcare. In the following sections, we review the literature on the impacts of HIT in healthcare and introduce the productivity paradox [27] and stakeholder theory [28]. From these, and based on empirical evidence from three case studies of HIT implementations, we propose our HIT Comprehensive Assessment Framework. Finally, we discuss the implications of our paper and its limitations.

Section snippets

Background: the real impact of information technologies in health

A growing number of studies seek to determine the real impacts of HIT. A review of the literature on HIT impacts [19] found that between 1995 and 2004 (a 10-year period), 257 studies were published. This number is growing exponentially. Our literature review has allowed us to determine that in the past five years alone (2005–2009) more than 1300 articles have been published on this very issue. A close examination of these studies allowed us to identify three emergent themes: (1) the impact of

Theoretical foundations

Over the last decade, IT research has identified and helped us understand what is called the productivity paradox [27]. In the fields of management and finance, another stream of research based on stakeholder theory has helped explain contradictions in evaluations of outcomes. Both theories are described below and their application with regards to HIT implementation is reviewed.

Research methods

Our research approach was based on a multiple-case design study [73]. As indicated in Table 2, the three sites varied in terms of the type of hospital (teaching vs. general), the type of HIT (Alpha vs. Delta) implemented and the overall outcome (success vs. failure). In each setting, three distinct groups of stakeholders were identified: physicians, nurses and administrators. Data were collected as part of a larger research program on CIS implementations [74], [75], [76], [77].

At each site,

Empirical evidence: three case studies

In order to develop our framework, we use evidence from three case studies of HIT implementation in hospitals [23], [74]. The first case is set in a new general hospital that was never intended to process any paper documents. With this in mind, a multidisciplinary committee at the hospital chose the Alpha software package to support both clinical and administrative processes. In fact, two years passed before the HIT was actually implemented. The first module to be introduced – Test

Data analysis results

Based on our empirical data analysis, as evidenced by the quotes provided below, a wide diversity of impacts have been experienced in all three cases, from impacts on clinical processes and quality of care to security and privacy, as well as on organizational climate and power distribution:

Case 1 I thought that it was a good idea… If it can simplify our work and give us more time with patients, that's great… You could take someone's blood pressure and enter it right away, you stayed with the

A framework for assessing impacts of it in health

Based on our theoretical foundations and considering our empirical data, we propose the following assessment framework of the impacts of IT in health (see Fig. 1). The productivity paradox and stakeholder theory are combined here with our case analyses to provide a new lens for studying IT phenomena in healthcare, as illustrated by our framework. From productivity paradox theory, we learned that the absence of visible signs of success following an IT implementation could be related to four

Discussion and conclusion

Our article has underscored how difficult it is to draw any conclusions about the actual impacts of IT in health based on the extant literature, given that findings often contradict each other. Evidence from three empirical case studies suggest that a true assessment of HIT impacts require: (1) to identify, account for and accurately measure a wide range of impacts (beneficial/adverse, expected/unforeseen effects); (2) to consider the context of implementation; (3) to adopt a multi-level

Authors’ contributions

All three authors contributed to the design of the study. The first author conducted the semi-structured interviews and analyzed the data. The authors interpreted the results together. IV and LL developed the assessment framework. All three wrote the paper as a team; all approved the final manuscript.

Statement on conflict of interest

The authors declare that they have no conflict of interest.

Summary points

What was already known on the topic:

  • IT investments in healthcare are greater than they have ever been.

  • HITs are being presented as a means to improve productivity, quality of health and/or system efficiency.

  • Current study results are however contradictory.

    • Some studies present HITs as a means to greater productivity and/or efficiency.

    • Other findings remain inconclusive. Some studies even show that HIT can be counter-productive.

Acknowledgements

This study was supported by grants from SSHRC (Social Sciences and Humanities Research Council of Canada) and CIHR (Canadian Institutes of Health Research). Muriel Gueriton, Librarian, provided help with the literature search. The sponsors played no role in the study design; the collection, analysis and interpretation of data; the writing of the manuscript; or the decision to submit the manuscript for publication.

References (90)

  • J.S. Ash et al.

    Some unintended consequences of information technology in health care: the nature of patient care information system-related errors

    J. Am. Med. Inform. Assoc.

    (2004)
  • S.J. Wang et al.

    A cost–benefit analysis of electronic medical records in primary care

    Am. J. Med.

    (2003)
  • R. Mador et al.

    The impact of a Critical Care Information System (CCIS) on time spent charting and in direct patient care by staff in the ICU: a review of the literature

    Int. J. Med. Inform.

    (2009)
  • M.J. Ball et al.

    Enabling technologies promise to revitalize the role of nursing in an era of patient safety

    Int. J. Med. Inform.

    (2003)
  • M. Prgomet et al.

    The impact of mobile handheld technology on hospital physicians’ work practices and patient care: a systematic review

    J. Am. Med. Inform. Assoc.

    (2009)
  • M.E. Frisse

    Comments on Return on Investment (ROI) as it applies to clinical systems

    J. Am. Med. Inform. Assoc.

    (2006)
  • J.M. Rothschild et al.

    Use and perceived benefits of handheld computer-based clinical references

    J. Am. Med. Inform. Assoc.

    (2006)
  • P.P.M. Harteloh

    Quality systems in health care: a sociotechnical approach

    Health Policy

    (2003)
  • B. Dehning et al.

    Information technology and organizational slack

    Int. J. Acc. Inf. Sys.

    (2004)
  • T. Stratopoulos et al.

    Does successful investment in information technology solve the productivity paradox?

    Inform. Manage.

    (2000)
  • A. Grant et al.

    The TEAM methodology for the evaluation of information systems in biomedicine

    Comput. Biol. Med.

    (2002)
  • A. Kazanjian et al.

    Beyond effectiveness: the evaluation of information systems using a comprehensive health technology assessment framework

    Comput. Biol. Med.

    (2002)
  • S.S. Lyons et al.

    VA QUERI informatics paper Information technology for clinical guideline implementation: Perceptions of multidisciplinary stakeholders

    J. Am. Med. Inform. Assoc.

    (2005)
  • J.J. Cimino et al.

    The patient clinical information system (PatCIS): technical solutions for and experience with giving patients access to their electronic medical records

    Int. J. Med. Inform.

    (2002)
  • J.S. Simon et al.

    Adoption of order entry with decision support for chronic care by physician organizations

    J. Am. Med. Inform. Assoc.

    (2007)
  • J. Conn

    Still taking small steps. IT spending to grow, but don’t expect large increases, annual survey says

    Mod. Healthc.

    (2008)
  • IDC Report. Western Europe Healthcare Sector IT Spending Forecast Update, 2007–2011. 2008 February 2008 Contract No.:...
  • R. Meyer et al.

    Assessing the capital efficiency of healthcare information technologies investments: an econometric perspective

    Yearb. Med. Inform.

    (2008)
  • R. Bodell et al.

    Achieving a “Therapeutic Dose” of IT

    Stud. Health Technol. Inform.

    (2004)
  • J. Sidorov

    It ain’t necessarily so: the electronic health record and the unlikely prospect of reducing health care costs

    Health Aff.

    (2006)
  • Committee on Quality of Health Care in A, Institute of M. Crossing the Quality Chasm: A New Health System for the 21st Century

    (2001)
  • R. Koppel et al.

    Role of computerized physician order entry systems in facilitating medication errors

    Am. Med. Assoc.

    (2005)
  • C.L. Delpierre et al.

    A systematic review of computer-based patient record systems and quality of care: more randomized clinical trials or a broader approach?

    Int. J. Qual. Health Care

    (2004)
  • R. Kaushal et al.

    Effects of computerized physician order entry and clinical decision support systems on medication safety a systematic review

    Arch. Intern. Med.

    (2003)
  • M.F. Thouin et al.

    The effect of information technology investment on firm-level performance in the health care industry

    Health Care Manage. Rev.

    (2008)
  • A.S. Kazley et al.

    Do hospitals with electronic medical records (EMRs) provide higher quality care? An examination of three clinical conditions

    Med. Care Res. Rev.

    (2008)
  • W.T. Lester et al.

    Designing healthcare information technology to catalyse change in clinical care

    Inform. Prim. Care.

    (2008)
  • B. Chaudry et al.

    Systematic review: impact of health information technology on quality, efficiency, and costs of medical care

    Ann. Intern. Med.

    (2006)
  • D. Avison et al.

    Time to rethink health care and ICT?

    Commun ACM

    (2007)
  • L. Lapointe et al.

    Clinical information systems: understanding and preventing their premature demise

    Healthc. Q

    (2005)
  • B. Demchak et al.

    Data quality for situational awareness during mass-casualty events

    AMIA Annu. Symp. Proc.

    (2007)
  • R. Nolan et al.

    Information technology and the board of directors

    Harv. Bus. Rev.

    (2005)
  • E. Brynjolfsson

    The productivity paradox of information technology: review and assessment

    Commun. ACM

    (1993)
  • R.E. Freeman

    Strategic Management: A Stakeholder Approach

    (1984)
  • M. Skouroliakou et al.

    Data analysis of the benefits of an electronic registry of information in a neonatal intensive care unit in Greece

    Perspect. Health. Inf. Manag.

    (2008)
  • Cited by (0)

    View full text