The impact of data quality defects on clinical decision-making in the intensive care unit

https://doi.org/10.1016/j.cmpb.2021.106359Get rights and content

Highlights

  • A poor data quality level affects clinical decision making about medication prescribed in the ICU.

  • A poor data quality level increases the likelihood of medication prescription/ invasive procedure in five clinical scenarios in the ICU.

  • It is important to emphasize that quality defects in clinical data affect decision making even without practitioners’ awareness.

Abstract

Objective

Poor clinical data quality might affect clinical decision making and patient treatment. This study identifies quality defects in clinical data collected automatically by bedside monitoring devices in the Intensive Care Unit (ICU) and examines their effect on clinical decisions.

Methods

Real-world data collected from 7688 patients admitted to the general ICU in a tertiary referral hospital over seven years was retrospectively analyzed. Data quality defect detection methods that use time-series analysis techniques identified two types of data quality defects: (a) completeness: the extent of non-missing values, and (b) validity: the extent of non-extreme values within the continuous range of values. Data quality defects were compared to five scenarios of medication and procedure prescriptions that are common in ICU settings: Blood-pressure reduction, blood-pressure elevation, anesthesia medications, intubation procedures, and muscle relaxant medications.

Results

Results from a logistic regression revealed a strong connection between data quality and the clinical interventions examined: lower validity level increased the likelihood of prescription decisions for all five scenarios, and lower completeness level increased the likelihood of prescription decisions for some scenarios.

Discussion

The results highlight the possible effect of data quality defects on physicians' decisions. Lower validity of certain key clinical parameters, and in some scenarios lower completeness, correlated with stronger tendency to prescribe medications or perform invasive procedures.

Conclusions

Data quality defects in clinical data affect decision making even without practitioners’ awareness. Thus, it is important to emphasize these effects to ICU staff, as well as to medical device manufacturers.

Introduction

Clinical data, reflecting patients' clinical characteristics and health condition, support a broad range of decisions and treatments at the point of care. Data quality (DQ) defects in this data have long been a major concern, given their potentially severe impact on clinical decision making and treatment [1,2]. Understanding, assessing, and improving the quality of clinical DQ and exploring the hazardous effects of DQ defects has long been a major research target.

Many studies focused on data acquisition acquired during doctor–patient interactions that is recorded manually into electronic medical record systems [3,4]. Over the years, bedside data acquisition became automatic and continuous, using dedicated devices and computerized monitoring utilities [5,6]. Bedside vital signs monitoring devices acquire and collect vast amount of data, which are typically presented to the clinical staff in real time, and recorded to a database for further tracking and analysis of patients' condition. Studies also indicated negative effects related to low quality monitoring signals such as frequent false alarms (the "crying wolf" phenomenon), erroneous treatment decisions, and mistrust [7], [8], [9]. The nature of data acquisition in such scenarios mandates different DQ assessment methods, as well as different approaches toward understanding the possible hazardous impact of DQ defects. Although other domains try to asses data DQ in order to improve decision-making [10], to the best of our knowledge no such studies were done on the impact of data quality defects on clinical decision-making in intensive care units (ICU).

The goal of this study is to examine DQ defects in clinical data collected automatically by bedside sensors and devices, and to evaluate its effect on actual patient care decisions. It aims to shed light on the crucial effect DQ defects have on decision making.

Section snippets

Hypothesis

The study hypothesized that DQ defects in clinical data will significantly affect clinical decision making.

Methodology

This study focuses on bedside-monitoring data, which is acquired automatically and continually at high sampling frequency through multiple sensors attached to the patient, each handling a different form of clinical measurement – heart rate, blood pressure, blood-oxygen saturation, and many others. The study attempts to associate clinical decisions with the quality of clinical data acquired before a decision was made, focusing on two DQ dimensions [11], each reflecting a relevant quality

Data quality assessment

The true positive rate (TPR) reflects the ratio of the correctly identified invalid values, while the false positive rate (FPR) reflects the ratio of valid values that were tagged as invalid. Obviously, as shown in Table 1, the baseline method's TPR is always 1, as every sample that exceeded the limits was captured.

The impact of DQ defects on clinical decision making

The logistic regression results are presented in Tables 2 and 3.

In the five models estimated, logistic regression results partially or completely support the research hypothesis, and

Discussion

Data quality (DQ) is broadly recognized as a major concern in clinical information systems; however, so far research has not paid much attention to the unique DQ issues involved in automatic data collection of ICU vital signs. This study developed a method for assessment for such data by adapting a combination of analytical tools and time-series analysis techniques. These methods address two DQ dimensions: (a) completeness, reflecting the extent of non-missing values, and (b) validity,

Declaration of Competing Interest

All authors declare that they have no conflicts of interest to disclose.

Acknowledgement

We would like to thank the Tel-Aviv Medical Center staff members involved in our research for their time, immense support and contributions, and Prof. Yisrael Parmet for the statistical consultation.

This work was partially funded by the Israeli Ministry of Science, Technology and Space (#8767311).

References (16)

There are more references available in the full text version of this article.

Cited by (0)

View full text