Abstract
Unplanned readmission after primary total knee arthroplasty (TKA) costs an average of US $39,000 per episode and negatively impacts patient outcomes. Although predictive machine learning (ML) models show promise for risk stratification in specific populations, existing studies do not address model generalizability. This study aimed to establish the generalizability of previous institutionally developed ML models to predict 30-day readmission following primary TKA using a national database. Data from 424,354 patients from the ACS-NSQIP database was used to develop and validate four ML models to predict 30-day readmission risk after primary TKA. Individual model performance was assessed and compared based on discrimination, accuracy, calibration, and clinical utility. Length of stay (> 2.5 days), body mass index (BMI) (> 33.21 kg/m2), and operation time (> 93 min) were important determinants of 30-day readmission. All ML models demonstrated equally good accuracy, calibration, and discriminatory ability (Brier score, ANN = RF = HGB = NEPLR = 0.03; ANN, slope = 0.90, intercept = − 0.11; RF, slope = 0.93, intercept = − 0.12; HGB, slope = 0.90, intercept = − 0.12; NEPLR, slope = 0.77, intercept = 0.01; AUCANN = AUCRF = AUCHGB = AUCNEPLR = 0.78). This study validates the generalizability of four previously developed ML algorithms in predicting readmission risk in patients undergoing TKA and offers surgeons an opportunity to reduce readmissions by optimizing discharge planning, BMI, and surgical efficiency.
Graphical Abstract
Similar content being viewed by others
References
Urish KL, Qin Y, Li BY et al (2018) Predictors and cost of readmission in total knee arthroplasty. J Arthroplasty 33:2759. https://doi.org/10.1016/J.ARTH.2018.04.008
Papakostidis C, Giannoudis PV, Watson JT et al (2021) Serious adverse events and 30-day hospital readmission rate following elective total knee arthroplasty: a systematic review and meta-analysis. J Orthop Surg Res 16:1–15. https://doi.org/10.1186/S13018-021-02358-W/TABLES/7
Centers for Medicare and Medicaid Services (CMS), Department of Health and Human Services (HHS) (2019) Total knee arthroplasty (TKA) removal from the medicare inpatient-only (IPO) List and application of the 2-midnight rule. https://www.cms.gov/files/document/se18018pdf. Accessed 22 Mar 2024
Crawford DA, Adams JB, Berend KR, Lombardi AV (2020) Low complication rates in outpatient total knee arthroplasty. Knee Surg Sports Traumatol Arthrosc 28:1458–1464. https://doi.org/10.1007/S00167-019-05538-8
Rodríguez-Merchán EC (2020) Outpatient total knee arthroplasty: is it worth considering? EFORT Open Rev 5:172. https://doi.org/10.1302/2058-5241.5.180101
Mantel J, Ruppenkamp JW, Cantu M, Holy CE (2023) Total knee arthroplasty in the outpatient vs inpatient settings: impact of site of care on early postoperative economic and clinical outcomes. J Orthop Surg Res 18:273. https://doi.org/10.1186/S13018-023-03750-4
Curtis GL, Jawad M, Samuel LT et al (2019) Incidence, causes, and timing of 30-day readmission following total knee arthroplasty. J Arthroplasty 34:2632–2636. https://doi.org/10.1016/J.ARTH.2019.06.009
Su CA, Copp JA, Weinberg DS et al (2020) Are readmissions after total knee arthroplasty preventable? J Knee Surg 33:603–610. https://doi.org/10.1055/S-0039-1683890
Williams CL, Pujalte G, Li Z et al (2022) Which factors predict 30-day readmission after total hip and knee replacement surgery? Cureus 14(3):e23093. https://doi.org/10.7759/cureus.23093
Phruetthiphat OA, Otero JE, Zampogna B et al (2020) Predictors for readmission following primary total hip and total knee arthroplasty. J Orthop Surg 28(3):2309499020959160. https://doi.org/10.1177/2309499020959160
Habehh H, Gohel S (2021) Machine learning in healthcare. Curr Genomics 22:291–300. https://doi.org/10.2174/1389202922666210705124359
Klemt C, Tirumala V, Habibi Y et al (2023) The utilization of artificial neural networks for the prediction of 90- day unplanned readmissions following total knee arthroplasty. Arch Orthop Trauma Surg 143(6):3279–3289. https://doi.org/10.1007/s00402-022-04566-3
Goyal A, Ngufor C, Kerezoudis P et al (2019) Can machine learning algorithms accurately predict discharge to nonhome facility and early unplanned readmissions following spinal fusion? Analysis of a national surgical registry. J Neurosurg Spine 31:568–578. https://doi.org/10.3171/2019.3.SPINE181367
Yeo I, Klemt C, Robinson MG et al (2022) The use of artificial neural networks for the prediction of surgical site infection following TKA. J Knee Surg. https://doi.org/10.1055/S-0041-1741396
Buddhiraju A, Shimizu MR, Subih MA et al (2023) Validation of machine learning model performance in predicting blood transfusion after primary and revision total hip arthroplasty. J Arthroplasty 38:1959–1966. https://doi.org/10.1016/J.ARTH.2023.06.002
Haeberle HS, Helm JM, Navarro SM et al (2019) Artificial intelligence and machine learning in lower extremity arthroplasty: a review. J Arthroplasty 34:2201–2203. https://doi.org/10.1016/J.ARTH.2019.05.055
Bini SA (2018) Artificial intelligence, machine learning, deep learning, and cognitive computing: what do these terms mean and how will they impact health care? J Arthroplasty 33:2358–2361. https://doi.org/10.1016/J.ARTH.2018.02.067
Labott JR, Brinkmann EJ, Hevesi M et al (2021) Utility of the ACS-NSQIP surgical risk calculator in predicting postoperative complications in patients undergoing oncologic proximal femoral replacement. J Surg Oncol 124:852–857. https://doi.org/10.1002/JSO.26583
Cabrera A, Bouterse A, Nelson M et al (2023) Use of random forest machine learning algorithm to predict short term outcomes following posterior cervical decompression with instrumented fusion. J Clin Neurosci 107:167–171. https://doi.org/10.1016/J.JOCN.2022.10.029
Katakam A, Karhade AV, Collins A et al (2022) Development of machine learning algorithms to predict achievement of minimal clinically important difference for the KOOS-PS following total knee arthroplasty. J Orthopaedic Res 40:808–815. https://doi.org/10.1002/JOR.25125
Crawford AM, Karhade AV, Agaronnik ND et al (2023) Development of a machine learning algorithm to identify surgical candidates for hip and knee arthroplasty without in-person evaluation. Arch Orthop Trauma Surg 143:5985–5992. https://doi.org/10.1007/S00402-023-04827-9/FIGURES/3
Kugelman D, Huang S, Teo G et al (2022) A novel machine learning predictive tool assessing outpatient or inpatient designation for medicare patients undergoing total knee arthroplasty. Arthroplast Today 13:120. https://doi.org/10.1016/J.ARTD.2021.12.004
Wei C, Quan T, Wang KY et al (2021) Artificial neural network prediction of same-day discharge following primary total knee arthroplasty based on preoperative and intraoperative variables. Bone Joint J 103-B:1358–1366. https://doi.org/10.1302/0301-620X.103B8.BJJ-2020-1013.R2
Debaun MR, Chavez G, Fithian A et al (2021) Artificial neural networks predict 30-day mortality after hip fracture: insights from machine learning. J Am Acad Orthop Surg 29:977–983. https://doi.org/10.5435/JAAOS-D-20-00429
Mohammadi R, Jain S, Namin AT et al (2020) Predicting unplanned readmissions following a hip or knee arthroplasty: retrospective observational study. JMIR Med Inform 8(11):e19761. https://doi.org/10.2196/19761
Chen TLW, Buddhiraju A, Seo HH et al (2023) Can machine learning models predict prolonged length of hospital stay following primary total knee arthroplasty based on a national patient cohort data? Arch Orthop Trauma Surg 143:7185–7193. https://doi.org/10.1007/S00402-023-05013-7/FIGURES/3
Alsoof D, McDonald CL, Kuris EO, Daniels AH (2022) Machine learning for the orthopaedic surgeon: uses and limitations. J Bone Joint Surg Am 104:1586–1594. https://doi.org/10.2106/JBJS.21.01305
Ramkumar PN, Pang M, Polisetty T et al (2022) Meaningless applications and misguided methodologies in artificial intelligence-related orthopaedic research propagates hype over hope. Arthroscopy 38:2761–2766. https://doi.org/10.1016/J.ARTHRO.2022.04.014
Shiloach M, Frencher SK, Steeger JE et al (2010) Toward robust information: data quality and inter-rater reliability in the American College of Surgeons national surgical quality improvement program. J Am Coll Surg 210:6–16. https://doi.org/10.1016/J.JAMCOLLSURG.2009.09.031
Sloan M, Sheth N, Lee GC (2019) Is obesity associated with increased risk of deep vein thrombosis or pulmonary embolism after hip and knee arthroplasty? A large database study. Clin Orthop Relat Res 477:523–532. https://doi.org/10.1097/CORR.0000000000000615
Wilson JM, Farley KX, Bradbury TL, Guild GN (2020) Is spinal anesthesia safer than general anesthesia for patients undergoing revision THA? Analysis of the ACS-NSQIP database. Clin Orthop Relat Res 478:80. https://doi.org/10.1097/CORR.0000000000000887
Darst BF, Malecki KC, Engelman CD (2018) Using recursive feature elimination in random forest to account for correlated variables in high dimensional data. BMC Genet 19(Suppl 1):65. https://doi.org/10.1186/s12863-018-0633-8
Cohen-Levy WB, Klemt C, Tirumala V et al (2022) Artificial neural networks for the prediction of transfusion rates in primary total hip arthroplasty. Arch Orthop Trauma Surg. https://doi.org/10.1007/s00402-022-04391-8
Choi RY, Coyner AS, Kalpathy-Cramer J et al (2020) Introduction to machine learning, neural networks, and deep learning. Transl Vis Sci Technol 9(2):14. https://doi.org/10.1167/tvst.9.2.14
Klug M, Barash Y, Bechler S et al (2020) A gradient boosting machine learning model for predicting early mortality in the emergency department triage: devising a nine-point triage score. J Gen Intern Med 35:220. https://doi.org/10.1007/S11606-019-05512-7
Greenwood CJ, Youssef GJ, Letcher P et al (2020) A comparison of penalised regression methods for informing the selection of predictive markers. PLoS One 15(11):e0242730. https://doi.org/10.1371/journal.pone.0242730
Wong J, Manderson T, Abrahamowicz M et al (2019) Can hyperparameter tuning improve the performance of a super learner? A case study. Epidemiology 30:521–531. https://doi.org/10.1097/EDE.0000000000001027
Klemt C, Tirumala V, Barghi A et al (2022) Artificial intelligence algorithms accurately predict prolonged length of stay following revision total knee arthroplasty. Knee Surg Sports Traumatol Arthrosc 30:2556–2564. https://doi.org/10.1007/s00167-022-06894-8
Steyerberg EW, Vickers AJ, Cook NR et al (2010) Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 21:128–138
Mandrekar JN (2010) Receiver operating characteristic curve in diagnostic test assessment. J Thorac Oncol 5:1315–1316. https://doi.org/10.1097/JTO.0b013e3181ec173d
Stevens RJ, Poppe KK (2020) Validation of clinical prediction models: what does the “calibration slope” really measure? J Clin Epidemiol 118:93–99. https://doi.org/10.1016/J.JCLINEPI.2019.09.016
Vickers AJ, Elkin EB (2006) Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making 26:565–574. https://doi.org/10.1177/0272989X06295361
Phillips JLH, Rondon AJ, Vannello C et al (2019) How much does a readmission cost the bundle following primary hip and knee arthroplasty? J Arthroplasty 34:819–823. https://doi.org/10.1016/J.ARTH.2019.01.029
Hosp. Readmission Reduction | CMS. https://www.cms.gov/medicare/quality/value-based-programs/hospitalreadmissions. Accessed 7 Dec 2023
Ramaswamy A, Marchese M, Cole AP et al (2019) Comparison of hospital readmission after total hip and total knee arthroplasty vs spinal surgery after implementation of the hospital readmissions reduction program. JAMA Netw Open 2(5):e194634. https://doi.org/10.1001/jamanetworkopen.2019.4634
Centers for Medicare and Medicaid Services (CMS), Department of Health and Human Services (HHS) hospital readmissions reduction program (HRRP) fiscal year 2023 fact sheet
Ramspek CL, Jager KJ, Dekker FW et al (2020) External validation of prognostic models: What, why, how, when and where? Clin Kidney J 14(1):49–58. https://doi.org/10.1093/ckj/sfaa188
Benito J, Stafford J, Judd H et al (2022) Length of Stay Increases 90-day readmission rates in patients undergoing primary total joint arthroplasty. J Am Acad Orthop Surg Glob Res Rev 6. https://doi.org/10.5435/JAAOSGLOBAL-D-21-00271
Thirukumaran CP, McGarry BE, Glance LG et al (2020) Impact of hospital readmissions reduction program penalties on hip and knee replacement readmissions: comparison of hospitals at risk of varying penalty amounts. J Bone Joint Surg Am 102:60–67. https://doi.org/10.2106/JBJS.18.01501
Charpentier PM, Srivastava AK, Zheng H et al (2018) Readmission rates for one versus two-midnight length of stay for primary total knee arthroplasty: analysis of the Michigan arthroplasty registry collaborative quality initiative (MARCQI) database. J Bone Joint Surg Am 100:1757–1764. https://doi.org/10.2106/JBJS.18.00166
Novack TA, Kurowicki J, Issa K et al (2020) Accelerated Discharge following total knee arthroplasty may be safe in a teaching institution. J Knee Surg 33(1):8–11. https://doi.org/10.1055/s-0038-1676066
Sloan M, Sheth NP, Nelson CL (2020) Obesity and hypoalbuminaemia are independent risk factors for readmission and reoperation following primary total knee arthroplasty. Bone Joint J 102-B 31-35. https://doi.org/10.1302/0301-620X.102B6.BJJ-2019-1509.R1
George J, Piuzzi NS, Ng M et al (2018) Association between body mass index and thirty-day complications after total knee arthroplasty. J Arthroplasty 33:865–871. https://doi.org/10.1016/J.ARTH.2017.09.038
Groot OQ, Ogink PT, Lans A et al (2022) Machine learning prediction models in orthopedic surgery: a systematic review in transparent reporting. J Orthop Res 40:475–483. https://doi.org/10.1002/JOR.25036
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Buddhiraju, A., Shimizu, M.R., Seo, H.H. et al. Generalizability of machine learning models predicting 30-day unplanned readmission after primary total knee arthroplasty using a nationally representative database. Med Biol Eng Comput 62, 2333–2341 (2024). https://doi.org/10.1007/s11517-024-03075-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11517-024-03075-2