Prediction of cardiac arrest in critically ill patients based on bedside vital signs monitoring

https://doi.org/10.1016/j.cmpb.2021.106568Get rights and content

Highlights

  • •

    A real-time and interpretable machine learning model for predicting cardiac arrest in critically ill patients is proposed.

  • •

    43 features are extracted based on four vital signs which are commonly used in bedside vital signs monitoring.

  • •

    SHAP value is used to capture the overall and real-time interpretability of the model.

  • •

    Compared with previous studies, this model is more suitable for bedside vital signs monitoring to predict cardiac arrest. The sensitivity and specificity were 0.86 and 0.85, respectively.

Abstract

Purpose

Cardiac arrest (CA) is the most serious death-related event in critically ill patients and the early detection of CA is beneficial to reduce mortality according to clinical research. This study aims to develop and verify a real-time, interpretable machine learning model, namely cardiac arrest prediction index (CAPI), to predict CA of critically ill patients based on bedside vital signs monitoring.

Methods

A total of 1,860 patients were analyzed retrospectively from the Medical Information Mart for Intensive Care III (MIMIC-III) database. Based on vital signs, we extracted a total of 43 features for building machine learning model. Extreme Gradient Boosting (XGBoost) was used to develop a real-time prediction model. Three-fold cross validation determined the consistency of model accuracy. SHAP value was used to capture the overall and real-time interpretability of the model.

Results

On the test set, CAPI predicted 95% of CA events, 80% of which were identified more than 25 min in advance, resulting in an area under the receiver operating characteristic curve (AUROC) of 0.94. The sensitivity, specificity, area under the precision-recall curve (AUPRC) and F1-score were 0.86, 0.85, 0.12 and 0.05, respectively.

Conclusion

CAPI can help predict patients with CA in the vital signs monitoring at bedside. Compared with previous studies, CAPI can give more timely notifications to doctors for CA events. However, current performance was at the cost of alarm fatigue. Future research is still needed to achieve better clinical application.

Introduction

Cardiac arrest (CA) refers to sudden interruption of cardiac activity, usually caused by some specified anomalous events, such as ventricular fibrillation (VF), pulseless ventricular tachycardia (VT), asystole and pulseless electrical activity [1]. About 290,000 in-hospital CAs are reported in the United States each year, and the survival discharge rate of patients with CA is only 25% [2]. A study has shown that 59.4% of patients had at least one abnormal sign within 1–4 h before CA, such as respiratory problems or hemodynamic instability [3]. A study of Bergum et al. further highlighted that early recognition of the causes of CA has been shown to increase the survival rate of patients within an hour of episode by about 29% and by 19% until their discharge [4]. Thus, the early detection of CA is quite beneficial to reduce in-hospital mortality.

From a clinical point of view, the real-time as well as accuracy of predicting CA in the short time window before event have great impact. Additionally, interpretability is a significant factor for doctor to make medical decisions. Over the past decade, multiple attempts have been made to predict CA. Several studies managed to train models with more and more abundant features, e.g., demographic variables and laboratory values [5], [6], [7], [8], [9]. Although the predictions can be more accurate, the models are not real-time and overly rely on the features that are not commonly used in hospital. Other studies developed real-time early warning systems using deep learning networks [10,11]. However, due to the “black box” nature of deep learning, clinician often believe that the results of deep learning are unexplained, so they distrust the medical decision support of deep learning [12].

Bedside vital signs monitoring is the most common nursing assessment [13], which includes five vital signs: blood pressure, peripheral oxygen saturation (SpO2), heart rate (HR), respiratory rate (RR) and body temperature. These can help identify the deterioration of patients. Since they have been widely used in clinical practice, they are suitable to be used as the data source for the CA prediction.

Based on vital signs monitoring, this study aims to develop a real-time and interpretable machine learning model, namely cardiac arrest prediction index (CAPI), to predict the risk of CA in critically ill patients with only four vital signs (blood pressure, SpO2, RR, HR) and verify its clinical practicability.

Section snippets

Dataset

We tested our method using the multi-parameter waveform data in the Medical Information Mart for Intensive Care III (MIMIC III) database, which was developed by the MIT Lab [14]. The MIMIC III publication noted that, “the project was approved by the Institutional Review Boards of Beth Israel Deaconess Medical Center (Boston, MA) and the Massachusetts Institute of Technology (Cambridge, MA). Requirement for individual patient consent was waived because the project did not impact clinical care

Patient characteristics

A total of 1860 patients (169 CA patients and 1691 non-CA patients) were selected. We randomly assigned 80% of the patients for model development and 20% for testing (Table 1). Continuous and categorical characteristics of patients were presented as median with upper/lower quartiles and percentage, respectively. T-test and Chi-square test were conducted to analyze the grouping differences between CA and non-CA patients. The confidence interval and significance level of both sides were lower

Discussion

Based on the MIMIC III database, a real-time, high-accuracy, and interpretable CAPI which uses only four vital signs was developed. As shown in Table 3, compared with previous studies, CAPI can be more appropriate for bedside vital signs monitoring to predict CA. Although the studies of Hong S et al. [8] and Kennedy et al. [9] proved higher accuracy, they relied on some features that couldn't be obtained from vital signs monitoring and were not real-time. An example of a real-time prediction of

Conclusion

We validated the machine-learning algorithm, CAPI, in a study including an ICU population. CAPI used only four common vital signs extracted from vital signs monitoring, which provided high accuracy for the early detection of CA. Every 5 min, CAPI aims to determine the risk of the patient developing CA within the next 1 hour. In addition, we used SHAP value to explain the overall and real-time behavior of CAPI, which helped doctors to gain insights into the relationship between prediction index

Funding/support

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References (29)

  • L.W. Andersen et al.

    In-hospital cardiac arrest: a review

    JAMA

    (2019 Mar 26)
  • M.M. Churpek et al.

    Multicenter comparison of machine learning methods and conventional regression for predicting clinical deterioration on the wards

    Crit. Care Med.

    (2016 Feb)
  • S. Hong et al.

    Prediction of cardiac arrest in the emergency department based on machine learning and sequential characteristics: model development and retrospective clinical validation study

    JMIR Med. Inform

    (2020 Aug 4)
  • C.E. Kennedy et al.

    Using time series analysis to predict cardiac arrest in a PICU

    Pediatr. Crit. Care Med.

    (2015 Nov)

    • Cited by (12)

    View all citing articles on Scopus
    View full text
    © 2021 Elsevier B.V. All rights reserved.