Predictive models for hospital readmission risk: A systematic review of methods
Introduction
Hospital readmissions are defined as admissions to a hospital within a -usually short- time span after discharge from hospital. Readmissions are frequent and costly events that impose tremendous burden on patients and healthcare systems [1], [2].
Hospital readmissions are becoming a strong concern of hospitals and policy makers as a measure of the quality of care given and have been adopted by many organizations as a quality indicator [3]. Centers for Medicare and Medicaid Services (CMS) in the USA [4] and policy makers in the UK [5] have introduced financial penalties to hospitals with high readmission rates by reducing the payment for patients readmitted within 30-day of discharge. Therefore, there is a growing interest within the research community to address this problem from a data analysis perspective.
Some authors have performed bibliographic review studies with the aim of synthesizing the literature on prediction models to estimate the readmission risk. In 2011 Kansagara et al. [6] presented what is probably the most referenced systematic review paper in the field. This thorough review was mainly focused on model performance description and comparison to assess the suitability of the models for clinical or administrative use. The authors of the paper conclude that most readmission risk prediction models perform poorly and efforts to improve their performance are needed. The study also concludes that readmission risk prediction is a complex problem by nature, with many inherent limitations.
In 2015 Swain and Kharrazi [7] conducted a semi-systematic review of readmission predictive factors published prior to March 2013. This review was, to some degree, based on the work by Kansagara et al. and can be considered an extension of it. This work was focused on identifying the most significant predictive factors from previous readmission prediction models.
Other studies place the focus on a certain subpopulation rather than covering all the published risk prediction models. Ross et al. [8] conducted a review of statistical models for the readmission of heart failure (HF) patients. This work included the identification of analytic models, in addition to identifying patient characteristics associated with readmission. A more recent study by Leppin et al. [9] reviewed randomized trials that assessed the effect of interventions intended to prevent 30-day hospital readmissions.
Most of the previous review studies have focused on measuring the discrimination ability of the models and identifying predictive characteristics associated with readmission. In different but related fields, review studies targeting the analysis of data analysis approaches can be found. For instance, a systematic literature review on data mining techniques applied in cardiology was recently presented in [10].
Nevertheless, to our knowledge no study regarding machine learning techniques, including feature selection and class imbalance has been presented in the field of readmission prediction. Since those are areas which may provide improvements over previous methods, we have already carried out some experiments that support our initial thoughts [11], [12].
The objective of this study is to systematically review the prediction models for hospital readmission by describing the data analysis methods and algorithms used for building the models.
This paper is organized as follows: Section 2 summarizes the research methodology of our study. In section 3 we present the review results. In Section 4 we discuss the results and findings of this study. Lastly, Section 5 presents the conclusions and future work.
Section snippets
Materials and methods
In this work, we conduct a systematic review following the three stages proposed by Tranfield et al. [13], namely planning, conducting and reporting. According to this methodology, we first define the research questions. Then we define the search strategy by identifying the source databases and the inclusion and exclusion criteria. Next, we present the data extraction procedure and then present the results.
Results
Table 3 describes in detail the 76 studies included for review, after the selection process. In this section, we present the results of our systematic review study. First, we present an overview of the results and then we discuss the results regarding specific research questions.
Prediction performance of the models
The Area Under Receiver Operating Characteristic (ROC) Curve or c-statistic is the standard de facto metric for measuring the discrimination ability of readmission risk prediction models. Given that the main goal of some studies is to identify predictors associated to readmission, these kind of studies often do not provide the c-statistic as the overall performance metric. In addition, a minority of studies report sensitivity and specificity scores along with PPV instead of AUC [51], [25], [84]
Conclusions
Our literature review included 77 studies that described prediction models for hospital readmission risk. Although statistical modelling techniques have prevailed and are still popular techniques, machine learning approaches have emerged in recent years as a promising technique that can improve the predictive ability of readmission risk prediction models.
Clinical trials tend to follow established procedures when it comes to statistical data analysis. Logistic regression and Cox regression (or
References (98)
- et al.
Prediction of 30-day cardiac-related-emergency-readmissions using simple administrative hospital data
Int. J. Cardiol.
(2013) - et al.
Feasibility of 30-day hospital readmission prediction modeling based on health information exchange data
Int. J. Med. Inf.
(2015) - et al.
Knowledge discovery in cardiology: a systematic literature review
Int. J. Med. Inf.
(2017) - et al.
Using anticipative hybrid extreme rotation forest to predict emergency service readmission risk
J. Comput. Sci.
(2017) - et al.
Predicting the risk of unplanned readmission or death within 30 days of discharge after a heart failure hospitalization
Am. Heart J.
(2012) - et al.
Incidence and predictors of 30-day readmission among patients hospitalized for advanced liver disease
Clin. Gastroenterol. Hepatol.
(2011) - et al.
An absolute risk prediction model to determine unplanned cardiovascular readmissions for adults with chronic heart failure
Heart, Lung and Circ.
(2015) - et al.
Measuring potentially avoidable hospital readmissions
J. Clin. Epidemiol.
(2002) - et al.
Risk stratification model for 30-day heart failure readmission in a multiethnic south east asian community
Am. J. Cardiol.
(2017) - et al.
Factors associated with unplanned hospital readmission among patients 65 years of age and older in a Medicare managed care plan
Am. J. Med.
(1999)
Reliability of predicting early hospital readmission after discharge for an acute coronary syndrome using claims-based data
Am. J. Cardiol.
A mixed-ensemble model for hospital readmission
Artif. Intell. Med.
The effects of data sources, cohort selection, and outcome definition on a predictive model of risk of thirty-day hospital readmissions
J. Biomed. Inform.
Predicting risk of hospitalization or death among patients with heart failure in the veterans health administration
Am. J. Cardiol.
Linking electronic health record-extracted psychosocial data in real-time to risk of readmission for heart failure
Psychosomatics
Predicting readmission risk with institution-specific prediction models
Artif. Intell. Med.
Predictive model of readmission to internal medicine wards
Eur. J. Intern. Med
Predictive modeling of hospital readmissions using metaheuristics and data mining
Expert Syst. Appl.
Do non-clinical factors improve prediction of readmission risk?: Results from the Tele-Hf study
JACC: Heart Fail.
READMIT: a clinical risk index to predict 30-day readmission after discharge from acute psychiatric units
J. Psychiatr. Res.
Trends, causes and timing of 30-day readmissions after hospitalization for heart failure: 11-year population-based analysis with linked data
Int. J. Cardiol.
A comparison of models for predicting early hospital readmissions
J. Biomed. Inform.
Instance sampling in credit scoring: An empirical study of sample size and balancing
Int. J. Forecast
The use of the area under the ROC curve in the evaluation of machine learning algorithms
Pattern Recognit.
An introduction to ROC analysis
Pattern Recognit. Lett.
Factors influencing the predictive power of models for predicting mortality and/or heart failure hospitalization in patients with heart failure
JACC: Heart Fail.
Ensemble risk model of emergency admissions (ERMER)
Int. J. Med. Inf.
Diagnoses and timing of 30-day readmissions after hospitalization for heart failure, acute myocardial infarction, or pneumonia
JAMA
The readmission risk flag: using the electronic health record to automatically identify patients at risk for 30‐day readmission
J. Hosp. Med.
HHS. Medicare program; hospital inpatient prospective payment systems for acute care hospitals and the long-term care hospital prospective payment system and Fiscal Year 2014 rates; quality reporting requirements for specific providers; hospital conditions of participation; payment policies related to patient status
Final rules. Fed. Regist.
Hospitals will be fined for emergency readmissions, says Lansley
BMJ: Br. Med. J. (Online)
Risk prediction models for hospital readmission: a systematic review
JAMA
Statistical models and patient predictors of readmission for heart failure: a systematic review
Arch. Intern. Med.
Preventing 30-day hospital readmissions: a systematic review and meta-analysis of randomized trials
JAMA Intern Med.
Emergency department readmission risk prediction: a case study in Chile
Towards a methodology for developing evidence‐informed management knowledge by means of systematic review
Br. J. Manag.
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement
PLoS Med.
Modeling and variable selection in epidemiologic analysis
Am. J. Public Health
A three-step approach for the derivation and validation of high-performing predictive models using an operational dataset: congestive heart failure readmission case study
BMC Med. Inf. Decis. Making
A tool for prediction of risk of rehospitalisation and mortality in the hospitalised elderly: secondary analysis of clinical trial data
BMJ Open
Redefining readmission risk factors for general medicine patients
J. Hosp. Med.
Rates and predictors of 30-day readmission among commercially insured and medicaid-enrolled patients hospitalized with systolic heart failure
Circ: Heart Fail.
Prediction model for 30-day hospital readmissions among patients discharged receiving outpatient parenteral antibiotic therapy
Clin. Infect. Dis.
A prediction model for COPD readmissions: catching up, catching our breath, and improving a national problem
J. Community Hosp. Intern. Med. Perspect.
An automated model to identify heart failure patients at risk for 30-day readmission or death using electronic medical record data
Med. Care
A validated, risk assessment tool for predicting readmission after open ventral hernia repair
Hernia
An innovative model to predict pediatric emergency department return visits
Pediatr. Emerg. Care
Development of a predictive model to identify inpatients at risk of re-admission within 30 days of discharge (PARR-30)
BMJ Open
Real-time prediction of mortality, readmission, and length of stay using electronic health record data
J. Am. Med. Inform. Assoc.
Cited by (128)
Machine learning algorithms to predict outcomes in children and adolescents with COVID-19: A systematic review
2024, Artificial Intelligence in MedicinePerformance of advanced machine learning algorithms overlogistic regression in predicting hospital readmissions: A meta-analysis
2023, Exploratory Research in Clinical and Social PharmacySystematic review identifies the design and methodological conduct of studies on machine learning-based prediction models
2023, Journal of Clinical Epidemiology