Skip to main content

Advertisement

Generalizability of machine learning models predicting 30-day unplanned readmission after primary total knee arthroplasty using a nationally representative database

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

Unplanned readmission after primary total knee arthroplasty (TKA) costs an average of US $39,000 per episode and negatively impacts patient outcomes. Although predictive machine learning (ML) models show promise for risk stratification in specific populations, existing studies do not address model generalizability. This study aimed to establish the generalizability of previous institutionally developed ML models to predict 30-day readmission following primary TKA using a national database. Data from 424,354 patients from the ACS-NSQIP database was used to develop and validate four ML models to predict 30-day readmission risk after primary TKA. Individual model performance was assessed and compared based on discrimination, accuracy, calibration, and clinical utility. Length of stay (> 2.5 days), body mass index (BMI) (> 33.21 kg/m2), and operation time (> 93 min) were important determinants of 30-day readmission. All ML models demonstrated equally good accuracy, calibration, and discriminatory ability (Brier score, ANN = RF = HGB = NEPLR = 0.03; ANN, slope = 0.90, intercept =  − 0.11; RF, slope = 0.93, intercept =  − 0.12; HGB, slope = 0.90, intercept =  − 0.12; NEPLR, slope = 0.77, intercept = 0.01; AUCANN = AUCRF = AUCHGB = AUCNEPLR = 0.78). This study validates the generalizability of four previously developed ML algorithms in predicting readmission risk in patients undergoing TKA and offers surgeons an opportunity to reduce readmissions by optimizing discharge planning, BMI, and surgical efficiency.

Graphical Abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Urish KL, Qin Y, Li BY et al (2018) Predictors and cost of readmission in total knee arthroplasty. J Arthroplasty 33:2759. https://doi.org/10.1016/J.ARTH.2018.04.008

    Article  PubMed  PubMed Central  Google Scholar 

  2. Papakostidis C, Giannoudis PV, Watson JT et al (2021) Serious adverse events and 30-day hospital readmission rate following elective total knee arthroplasty: a systematic review and meta-analysis. J Orthop Surg Res 16:1–15. https://doi.org/10.1186/S13018-021-02358-W/TABLES/7

    Article  Google Scholar 

  3. Centers for Medicare and Medicaid Services (CMS), Department of Health and Human Services (HHS) (2019) Total knee arthroplasty (TKA) removal from the medicare inpatient-only (IPO) List and application of the 2-midnight rule. https://www.cms.gov/files/document/se18018pdf. Accessed 22 Mar 2024

  4. Crawford DA, Adams JB, Berend KR, Lombardi AV (2020) Low complication rates in outpatient total knee arthroplasty. Knee Surg Sports Traumatol Arthrosc 28:1458–1464. https://doi.org/10.1007/S00167-019-05538-8

    Article  PubMed  Google Scholar 

  5. Rodríguez-Merchán EC (2020) Outpatient total knee arthroplasty: is it worth considering? EFORT Open Rev 5:172. https://doi.org/10.1302/2058-5241.5.180101

    Article  PubMed  PubMed Central  Google Scholar 

  6. Mantel J, Ruppenkamp JW, Cantu M, Holy CE (2023) Total knee arthroplasty in the outpatient vs inpatient settings: impact of site of care on early postoperative economic and clinical outcomes. J Orthop Surg Res 18:273. https://doi.org/10.1186/S13018-023-03750-4

    Article  PubMed  PubMed Central  Google Scholar 

  7. Curtis GL, Jawad M, Samuel LT et al (2019) Incidence, causes, and timing of 30-day readmission following total knee arthroplasty. J Arthroplasty 34:2632–2636. https://doi.org/10.1016/J.ARTH.2019.06.009

    Article  PubMed  Google Scholar 

  8. Su CA, Copp JA, Weinberg DS et al (2020) Are readmissions after total knee arthroplasty preventable? J Knee Surg 33:603–610. https://doi.org/10.1055/S-0039-1683890

    Article  PubMed  Google Scholar 

  9. Williams CL, Pujalte G, Li Z et al (2022) Which factors predict 30-day readmission after total hip and knee replacement surgery? Cureus 14(3):e23093. https://doi.org/10.7759/cureus.23093

  10. Phruetthiphat OA, Otero JE, Zampogna B et al (2020) Predictors for readmission following primary total hip and total knee arthroplasty. J Orthop Surg 28(3):2309499020959160. https://doi.org/10.1177/2309499020959160

  11. Habehh H, Gohel S (2021) Machine learning in healthcare. Curr Genomics 22:291–300. https://doi.org/10.2174/1389202922666210705124359

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. Klemt C, Tirumala V, Habibi Y et al (2023) The utilization of artificial neural networks for the prediction of 90- day unplanned readmissions following total knee arthroplasty. Arch Orthop Trauma Surg 143(6):3279–3289. https://doi.org/10.1007/s00402-022-04566-3

  13. Goyal A, Ngufor C, Kerezoudis P et al (2019) Can machine learning algorithms accurately predict discharge to nonhome facility and early unplanned readmissions following spinal fusion? Analysis of a national surgical registry. J Neurosurg Spine 31:568–578. https://doi.org/10.3171/2019.3.SPINE181367

    Article  PubMed  Google Scholar 

  14. Yeo I, Klemt C, Robinson MG et al (2022) The use of artificial neural networks for the prediction of surgical site infection following TKA. J Knee Surg. https://doi.org/10.1055/S-0041-1741396

    Article  PubMed  Google Scholar 

  15. Buddhiraju A, Shimizu MR, Subih MA et al (2023) Validation of machine learning model performance in predicting blood transfusion after primary and revision total hip arthroplasty. J Arthroplasty 38:1959–1966. https://doi.org/10.1016/J.ARTH.2023.06.002

    Article  PubMed  Google Scholar 

  16. Haeberle HS, Helm JM, Navarro SM et al (2019) Artificial intelligence and machine learning in lower extremity arthroplasty: a review. J Arthroplasty 34:2201–2203. https://doi.org/10.1016/J.ARTH.2019.05.055

    Article  PubMed  Google Scholar 

  17. Bini SA (2018) Artificial intelligence, machine learning, deep learning, and cognitive computing: what do these terms mean and how will they impact health care? J Arthroplasty 33:2358–2361. https://doi.org/10.1016/J.ARTH.2018.02.067

    Article  PubMed  Google Scholar 

  18. Labott JR, Brinkmann EJ, Hevesi M et al (2021) Utility of the ACS-NSQIP surgical risk calculator in predicting postoperative complications in patients undergoing oncologic proximal femoral replacement. J Surg Oncol 124:852–857. https://doi.org/10.1002/JSO.26583

    Article  PubMed  Google Scholar 

  19. Cabrera A, Bouterse A, Nelson M et al (2023) Use of random forest machine learning algorithm to predict short term outcomes following posterior cervical decompression with instrumented fusion. J Clin Neurosci 107:167–171. https://doi.org/10.1016/J.JOCN.2022.10.029

    Article  PubMed  Google Scholar 

  20. Katakam A, Karhade AV, Collins A et al (2022) Development of machine learning algorithms to predict achievement of minimal clinically important difference for the KOOS-PS following total knee arthroplasty. J Orthopaedic Res 40:808–815. https://doi.org/10.1002/JOR.25125

    Article  Google Scholar 

  21. Crawford AM, Karhade AV, Agaronnik ND et al (2023) Development of a machine learning algorithm to identify surgical candidates for hip and knee arthroplasty without in-person evaluation. Arch Orthop Trauma Surg 143:5985–5992. https://doi.org/10.1007/S00402-023-04827-9/FIGURES/3

    Article  PubMed  Google Scholar 

  22. Kugelman D, Huang S, Teo G et al (2022) A novel machine learning predictive tool assessing outpatient or inpatient designation for medicare patients undergoing total knee arthroplasty. Arthroplast Today 13:120. https://doi.org/10.1016/J.ARTD.2021.12.004

    Article  PubMed  PubMed Central  Google Scholar 

  23. Wei C, Quan T, Wang KY et al (2021) Artificial neural network prediction of same-day discharge following primary total knee arthroplasty based on preoperative and intraoperative variables. Bone Joint J 103-B:1358–1366. https://doi.org/10.1302/0301-620X.103B8.BJJ-2020-1013.R2

    Article  PubMed  Google Scholar 

  24. Debaun MR, Chavez G, Fithian A et al (2021) Artificial neural networks predict 30-day mortality after hip fracture: insights from machine learning. J Am Acad Orthop Surg 29:977–983. https://doi.org/10.5435/JAAOS-D-20-00429

    Article  PubMed  Google Scholar 

  25. Mohammadi R, Jain S, Namin AT et al (2020) Predicting unplanned readmissions following a hip or knee arthroplasty: retrospective observational study. JMIR Med Inform 8(11):e19761. https://doi.org/10.2196/19761

  26. Chen TLW, Buddhiraju A, Seo HH et al (2023) Can machine learning models predict prolonged length of hospital stay following primary total knee arthroplasty based on a national patient cohort data? Arch Orthop Trauma Surg 143:7185–7193. https://doi.org/10.1007/S00402-023-05013-7/FIGURES/3

    Article  PubMed  Google Scholar 

  27. Alsoof D, McDonald CL, Kuris EO, Daniels AH (2022) Machine learning for the orthopaedic surgeon: uses and limitations. J Bone Joint Surg Am 104:1586–1594. https://doi.org/10.2106/JBJS.21.01305

    Article  PubMed  Google Scholar 

  28. Ramkumar PN, Pang M, Polisetty T et al (2022) Meaningless applications and misguided methodologies in artificial intelligence-related orthopaedic research propagates hype over hope. Arthroscopy 38:2761–2766. https://doi.org/10.1016/J.ARTHRO.2022.04.014

    Article  PubMed  Google Scholar 

  29. Shiloach M, Frencher SK, Steeger JE et al (2010) Toward robust information: data quality and inter-rater reliability in the American College of Surgeons national surgical quality improvement program. J Am Coll Surg 210:6–16. https://doi.org/10.1016/J.JAMCOLLSURG.2009.09.031

    Article  PubMed  Google Scholar 

  30. Sloan M, Sheth N, Lee GC (2019) Is obesity associated with increased risk of deep vein thrombosis or pulmonary embolism after hip and knee arthroplasty? A large database study. Clin Orthop Relat Res 477:523–532. https://doi.org/10.1097/CORR.0000000000000615

    Article  PubMed  PubMed Central  Google Scholar 

  31. Wilson JM, Farley KX, Bradbury TL, Guild GN (2020) Is spinal anesthesia safer than general anesthesia for patients undergoing revision THA? Analysis of the ACS-NSQIP database. Clin Orthop Relat Res 478:80. https://doi.org/10.1097/CORR.0000000000000887

    Article  PubMed  Google Scholar 

  32. Darst BF, Malecki KC, Engelman CD (2018) Using recursive feature elimination in random forest to account for correlated variables in high dimensional data. BMC Genet 19(Suppl 1):65. https://doi.org/10.1186/s12863-018-0633-8

  33. Cohen-Levy WB, Klemt C, Tirumala V et al (2022) Artificial neural networks for the prediction of transfusion rates in primary total hip arthroplasty. Arch Orthop Trauma Surg. https://doi.org/10.1007/s00402-022-04391-8

    Article  PubMed  Google Scholar 

  34. Choi RY, Coyner AS, Kalpathy-Cramer J et al (2020) Introduction to machine learning, neural networks, and deep learning. Transl Vis Sci Technol 9(2):14. https://doi.org/10.1167/tvst.9.2.14

  35. Klug M, Barash Y, Bechler S et al (2020) A gradient boosting machine learning model for predicting early mortality in the emergency department triage: devising a nine-point triage score. J Gen Intern Med 35:220. https://doi.org/10.1007/S11606-019-05512-7

    Article  PubMed  Google Scholar 

  36. Greenwood CJ, Youssef GJ, Letcher P et al (2020) A comparison of penalised regression methods for informing the selection of predictive markers. PLoS One 15(11):e0242730. https://doi.org/10.1371/journal.pone.0242730

  37. Wong J, Manderson T, Abrahamowicz M et al (2019) Can hyperparameter tuning improve the performance of a super learner? A case study. Epidemiology 30:521–531. https://doi.org/10.1097/EDE.0000000000001027

    Article  PubMed  PubMed Central  Google Scholar 

  38. Klemt C, Tirumala V, Barghi A et al (2022) Artificial intelligence algorithms accurately predict prolonged length of stay following revision total knee arthroplasty. Knee Surg Sports Traumatol Arthrosc 30:2556–2564. https://doi.org/10.1007/s00167-022-06894-8

    Article  PubMed  Google Scholar 

  39. Steyerberg EW, Vickers AJ, Cook NR et al (2010) Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 21:128–138

    Article  PubMed  PubMed Central  Google Scholar 

  40. Mandrekar JN (2010) Receiver operating characteristic curve in diagnostic test assessment. J Thorac Oncol 5:1315–1316. https://doi.org/10.1097/JTO.0b013e3181ec173d

    Article  PubMed  Google Scholar 

  41. Stevens RJ, Poppe KK (2020) Validation of clinical prediction models: what does the “calibration slope” really measure? J Clin Epidemiol 118:93–99. https://doi.org/10.1016/J.JCLINEPI.2019.09.016

    Article  PubMed  Google Scholar 

  42. Vickers AJ, Elkin EB (2006) Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making 26:565–574. https://doi.org/10.1177/0272989X06295361

    Article  PubMed  PubMed Central  Google Scholar 

  43. Phillips JLH, Rondon AJ, Vannello C et al (2019) How much does a readmission cost the bundle following primary hip and knee arthroplasty? J Arthroplasty 34:819–823. https://doi.org/10.1016/J.ARTH.2019.01.029

    Article  PubMed  Google Scholar 

  44. Hosp. Readmission Reduction | CMS. https://www.cms.gov/medicare/quality/value-based-programs/hospitalreadmissions. Accessed 7 Dec 2023

  45. Ramaswamy A, Marchese M, Cole AP et al (2019) Comparison of hospital readmission after total hip and total knee arthroplasty vs spinal surgery after implementation of the hospital readmissions reduction program. JAMA Netw Open 2(5):e194634. https://doi.org/10.1001/jamanetworkopen.2019.4634

  46. Centers for Medicare and Medicaid Services (CMS), Department of Health and Human Services (HHS) hospital readmissions reduction program (HRRP) fiscal year 2023 fact sheet

  47. Ramspek CL, Jager KJ, Dekker FW et al (2020) External validation of prognostic models: What, why, how, when and where? Clin Kidney J 14(1):49–58. https://doi.org/10.1093/ckj/sfaa188

  48. Benito J, Stafford J, Judd H et al (2022) Length of Stay Increases 90-day readmission rates in patients undergoing primary total joint arthroplasty. J Am Acad Orthop Surg Glob Res Rev 6. https://doi.org/10.5435/JAAOSGLOBAL-D-21-00271

  49. Thirukumaran CP, McGarry BE, Glance LG et al (2020) Impact of hospital readmissions reduction program penalties on hip and knee replacement readmissions: comparison of hospitals at risk of varying penalty amounts. J Bone Joint Surg Am 102:60–67. https://doi.org/10.2106/JBJS.18.01501

    Article  PubMed  Google Scholar 

  50. Charpentier PM, Srivastava AK, Zheng H et al (2018) Readmission rates for one versus two-midnight length of stay for primary total knee arthroplasty: analysis of the Michigan arthroplasty registry collaborative quality initiative (MARCQI) database. J Bone Joint Surg Am 100:1757–1764. https://doi.org/10.2106/JBJS.18.00166

    Article  CAS  PubMed  Google Scholar 

  51. Novack TA, Kurowicki J, Issa K et al (2020) Accelerated Discharge following total knee arthroplasty may be safe in a teaching institution. J Knee Surg 33(1):8–11. https://doi.org/10.1055/s-0038-1676066

  52. Sloan M, Sheth NP, Nelson CL (2020) Obesity and hypoalbuminaemia are independent risk factors for readmission and reoperation following primary total knee arthroplasty. Bone Joint J 102-B 31-35. https://doi.org/10.1302/0301-620X.102B6.BJJ-2019-1509.R1

  53. George J, Piuzzi NS, Ng M et al (2018) Association between body mass index and thirty-day complications after total knee arthroplasty. J Arthroplasty 33:865–871. https://doi.org/10.1016/J.ARTH.2017.09.038

    Article  PubMed  Google Scholar 

  54. Groot OQ, Ogink PT, Lans A et al (2022) Machine learning prediction models in orthopedic surgery: a systematic review in transparent reporting. J Orthop Res 40:475–483. https://doi.org/10.1002/JOR.25036

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Young-Min Kwon.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 16 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Buddhiraju, A., Shimizu, M.R., Seo, H.H. et al. Generalizability of machine learning models predicting 30-day unplanned readmission after primary total knee arthroplasty using a nationally representative database. Med Biol Eng Comput 62, 2333–2341 (2024). https://doi.org/10.1007/s11517-024-03075-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-024-03075-2

Keywords