Skip to main content

Advertisement

SpringerLink
Log in

An AI-Enabled Dynamic Risk Stratification for Emergency Department Patients with ECG and CXR Integration

  • Original Paper
  • Published:
Journal of Medical Systems Aims and scope Submit manuscript

Abstract

Emergency department (ED) triage scale determines the priority of patient care and foretells the prognosis. However, the information retrieved from the initial assessment is limited, hindering the risk identification accuracy of triage. Therefore, we sought to develop a 'dynamic' triage system as secondary screening, using artificial intelligence (AI) techniques to integrate information from initial assessment data and subsequent examinations. This retrospective cohort study included 134,112 ED visits with at least one electrocardiography (ECG) and chest X-ray (CXR) in a medical center from 2012 to 2022. Additionally, an independent community hospital provided 45,614 ED visits as an external validation set. We trained an eXtreme gradient boosting (XGB) model using initial assessment data to predict all-cause mortality in 7 days. Two deep learning models (DLMs) using ECG and CXR were trained to stratify mortality risks. The dynamic triage levels were based on output from the XGB-triage and DLMs from ECG and CXR. During the internal and external validation, the area under the receiver operating characteristic curve (AUC) of the XGB-triage model was >0.866; furthermore, the AUCs of DLMs using ECG and CXR were >0.862 and >0.886, respectively. The dynamic triage scale provided a higher C-index (0.914-0.920 vs. 0.827-0.843) than the original one and demonstrated better predictive ability for 5-year mortality, 30-day ED revisit, and 30-day discharge. The AI-based risk scale provides a more accurate and dynamic stratification of mortality risk in ED patients, particularly in identifying patients who tend to be overlooked due to atypical symptoms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data availability statement

The data analyzed in this study is not publicly available due to privacy and security concerns. The data may be shared with a third party upon execution of data sharing agreement for reasonable requests, such requests should be addressed to J.Y.H.C. (e-mail: jamiechen@mail.ndmctsgh.edu.tw) or S.J.C.

References

  1. N. Farrohknia, M. Castren, A. Ehrenberg, L. Lind, S. Oredsson, H. Jonsson, K. Asplund, K.E. Goransson, Emergency department triage scales and their components: a systematic review of the scientific evidence, Scand J Trauma Resusc Emerg Med 19 (2011) 42.

    Article  PubMed  PubMed Central  Google Scholar 

  2. C.M. Fernandes, P. Tanabe, N. Gilboy, L.A. Johnson, R.S. McNair, A.M. Rosenau, P. Sawchuk, D.A. Thompson, D.A. Travers, N. Bonalumi, R.E. Suter, Five-level triage: a report from the ACEP/ENA Five-level Triage Task Force, J Emerg Nurs 31(1) (2005) 39-50; quiz 118.

  3. P.J. Pronovost, D.C. Angus, T. Dorman, K.A. Robinson, T.T. Dremsizov, T.L. Young, Physician staffing patterns and clinical outcomes in critically ill patients: a systematic review, Jama 288(17) (2002) 2151-2162.

    Article  PubMed  Google Scholar 

  4. S. Jones, C. Moulton, S. Swift, P. Molyneux, S. Black, N. Mason, R. Oakley, C. Mann, Association between delays to patient admission from the emergency department and all-cause 30-day mortality, Emergency medicine journal : EMJ 39(3) (2022) 168-173.

    Article  PubMed  Google Scholar 

  5. N. Gilboy, T. Tanabe, D. Travers, A.M. Rosenau, Emergency Severity Index (ESI): A Triage Tool for Emergency Department Care, 4th ed., Emergency Nurses Association2020.

  6. F.A. Nishi, F. de Oliveira Motta Maia, I. de Souza Santos, D. de Almeida Lopes Monteiro da Cruz, Assessing sensitivity and specificity of the Manchester Triage System in the evaluation of acute coronary syndrome in adult patients in emergency care: a systematic review, JBI Database System Rev Implement Rep 15(6) (2017) 1747-1761.

  7. R. Beveridge, J. Ducharme, L. Janes, S. Beaulieu, S. Walter, Reliability of the Canadian emergency department triage and acuity scale: interrater agreement, Ann Emerg Med 34(2) (1999) 155-9.

    Article  CAS  PubMed  Google Scholar 

  8. C.J. Ng, Z.S. Yen, J.C. Tsai, L.C. Chen, S.J. Lin, Y.Y. Sang, J.C. Chen, T.n.w. group, Validation of the Taiwan triage and acuity scale: a new computerised five-level triage system, Emerg Med J 28(12) (2011) 1026-31.

  9. J.S. Hinson, D.A. Martinez, S. Cabral, K. George, M. Whalen, B. Hansoti, S. Levin, Triage Performance in Emergency Medicine: A Systematic Review, Annals of emergency medicine 74(1) (2019) 140-152.

    Article  PubMed  Google Scholar 

  10. T. Htay, K. Aung, Review: Some ED triage systems better predict ED mortality than in-hospital mortality or hospitalization, Ann Intern Med 170(8) (2019) JC47.

  11. T. Olsson, A. Terent, L. Lind, Rapid Emergency Medicine score: a new prognostic tool for in-hospital mortality in nonsurgical emergency department patients, J Intern Med 255(5) (2004) 579-87.

    Article  CAS  PubMed  Google Scholar 

  12. S.B. Lee, D.H. Kim, T. Kim, C. Kang, S.H. Lee, J.H. Jeong, S.C. Kim, Y.J. Park, D. Lim, Emergency Department Triage Early Warning Score (TREWS) predicts in-hospital mortality in the emergency department, Am J Emerg Med 38(2) (2020) 203-210.

    Article  PubMed  Google Scholar 

  13. T.C. Lu, C.H. Wang, F.Y. Chou, J.T. Sun, E.H. Chou, E.P. Huang, C.L. Tsai, M.H. Ma, C.C. Fang, C.H. Huang, Machine learning to predict in-hospital cardiac arrest from patients presenting to the emergency department, Intern Emerg Med (2022).

  14. F. Xie, M.E.H. Ong, J. Liew, K.B.K. Tan, A.F.W. Ho, G.D. Nadarajan, L.L. Low, Y.H. Kwan, B.A. Goldstein, D.B. Matchar, B. Chakraborty, N. Liu, Development and Assessment of an Interpretable Machine Learning Triage Tool for Estimating Mortality After Emergency Admissions, JAMA Netw Open 4(8) (2021) e2118467.

    Article  PubMed  PubMed Central  Google Scholar 

  15. S. Benjamens, P. Dhunnoo, B. Meskó, The state of artificial intelligence-based FDA-approved medical devices and algorithms: an online database, NPJ Digit Med 3 (2020) 118.

    Article  PubMed  PubMed Central  Google Scholar 

  16. S. Raghunath, A.E. Ulloa Cerna, L. Jing, D.P. vanMaanen, J. Stough, D.N. Hartzel, J.B. Leader, H.L. Kirchner, M.C. Stumpe, A. Hafez, A. Nemani, T. Carbonati, K.W. Johnson, K. Young, C.W. Good, J.M. Pfeifer, A.A. Patel, B.P. Delisle, A. Alsaid, D. Beer, C.M. Haggerty, B.K. Fornwalt, Prediction of mortality from 12-lead electrocardiogram voltage data using a deep neural network, Nature medicine 26(6) (2020) 886-891.

  17. M.T. Lu, A. Ivanov, T. Mayrhofer, A. Hosny, H. Aerts, U. Hoffmann, Deep Learning to Assess Long-term Mortality From Chest Radiographs, JAMA network open 2(7) (2019) e197416.

    Article  PubMed  PubMed Central  Google Scholar 

  18. D. Brieger, K.A. Eagle, S.G. Goodman, P.G. Steg, A. Budaj, K. White, G. Montalescot, Acute coronary syndromes without chest pain, an underdiagnosed and undertreated high-risk group: insights from the Global Registry of Acute Coronary Events, Chest 126(2) (2004) 461-9.

    Article  PubMed  Google Scholar 

  19. P.F. Huang, P.T. Kung, W.Y. Chou, W.C. Tsai, Characteristics and related factors of emergency department visits, readmission, and hospital transfers of inpatients under a DRG-based payment system: a nationwide cohort study, PloS one 15(12) (2020) e0243373.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  20. C.S. Lin, Y.T. Lee, W.H. Fang, Y.S. Lou, F.C. Kuo, C.C. Lee, C. Lin, Deep learning algorithm for management of diabetes mellitus via electrocardiogram-based glycated hemoglobin (ECG-HbA1c): a retrospective cohort study, J Pers Med 11(8) (2021) 725.

    Article  PubMed  PubMed Central  Google Scholar 

  21. W.T. Liu, C.S. Lin, T.P. Tsao, C.C. Lee, C.C. Cheng, J.T. Chen, C.S. Tsai, W.S. Lin, C. Lin, A deep-learning algorithm-enhanced system integrating electrocardiograms and chest X-rays for diagnosing aortic dissection, The Canadian journal of cardiology 38(2) (2022) 160-168.

    Article  PubMed  Google Scholar 

  22. A. Kirubarajan, A. Taher, S. Khan, S. Masood, Artificial intelligence in emergency medicine: A scoping review, Journal of the American College of Emergency Physicians open 1(6) (2020) 1691-1702.

    Article  PubMed  PubMed Central  Google Scholar 

  23. C. Mosquera, F. Binder, F.N. Diaz, A. Seehaus, G. Ducrey, J.A. Ocantos, M. Aineseder, L. Rubin, D.A. Rabinovich, A.E. Quiroga, Integration of a deep learning system for automated chest x-ray interpretation in the emergency department: A proof-of-concept, Intelligence-Based Medicine 5 (2021) 100039.

    Article  Google Scholar 

  24. Y.L. Liu, C.S. Lin, C.C. Cheng, C. Lin, A deep learning algorithm for detecting acute pericarditis by electrocardiogram, J Pers Med 12(7) (2022) 1150.

    Article  PubMed  PubMed Central  Google Scholar 

  25. C.C. Lee, C.S. Lin, C.S. Tsai, T.P. Tsao, C.C. Cheng, J.T. Liou, W.S. Lin, C.C. Lee, J.T. Chen, C. Lin, A deep learning-based system capable of detecting pneumothorax via electrocardiogram, European journal of trauma and emergency surgery : official publication of the European Trauma Society 48(4) (2022) 3317-3326.

    Article  PubMed  Google Scholar 

  26. W.C. Liu, C.S. Lin, C.S. Tsai, T.P. Tsao, C.C. Cheng, J.T. Liou, W.S. Lin, S.M. Cheng, Y.S. Lou, C.C. Lee, C. Lin, A deep-learning algorithm for detecting acute myocardial infarction, EuroIntervention : journal of EuroPCR in collaboration with the Working Group on Interventional Cardiology of the European Society of Cardiology 17(9) (2021) 765-773.

    Google Scholar 

  27. D.W. Chang, C.S. Lin, T.P. Tsao, C.C. Lee, J.T. Chen, C.S. Tsai, W.S. Lin, C. Lin, Detecting digoxin toxicity by artificial intelligence-assisted electrocardiography, International journal of environmental research and public health 18(7) (2021) 3839.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  28. C.S. Lin, C. Lin, W.H. Fang, C.J. Hsu, S.J. Chen, K.H. Huang, W.S. Lin, C.S. Tsai, C.C. Kuo, T. Chau, S.J. Yang, S.H. Lin, A deep-learning algorithm (ECG12Net) for detecting hypokalemia and hyperkalemia by electrocardiography: algorithm development, JMIR medical informatics 8(3) (2020) e15931.

    Article  PubMed  PubMed Central  Google Scholar 

  29. M. Fernandes, S.M. Vieira, F. Leite, C. Palos, S. Finkelstein, J.M.C. Sousa, Clinical Decision Support Systems for Triage in the Emergency Department using Intelligent Systems: a Review, Artif Intell Med 102 (2020) 101762.

    Article  PubMed  Google Scholar 

  30. Y. Raita, T. Goto, M.K. Faridi, D.F.M. Brown, C.A. Camargo, Jr., K. Hasegawa, Emergency department triage prediction of clinical outcomes using machine learning models, Crit Care 23(1) (2019) 64.

    Article  PubMed  PubMed Central  Google Scholar 

  31. L.W. Andersen, M.J. Holmberg, K.M. Berg, M.W. Donnino, A. Granfeldt, In-Hospital Cardiac Arrest: A Review, JAMA 321(12) (2019) 1200-1210.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Y.S. Lou, C.S. Lin, W.H. Fang, C.C. Lee, C.L. Ho, C.H. Wang, C. Lin, Artificial intelligence-enabled electrocardiogram estimates left atrium enlargement as a predictor of future cardiovascular disease, J Pers Med 12(2) (2022) 315.

    Article  PubMed  PubMed Central  Google Scholar 

  33. C. Lin, T. Chau, C.S. Lin, H.S. Shang, W.H. Fang, D.J. Lee, C.C. Lee, S.H. Tsai, C.H. Wang, S.H. Lin, Point-of-care artificial intelligence-enabled ECG for dyskalemia: a retrospective cohort analysis for accuracy and outcome prediction, NPJ Digit Med 5(1) (2022) 8.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Y.T. Lee, C.S. Lin, W.H. Fang, C.C. Lee, C.L. Ho, C.H. Wang, D.J. Tsai, C. Lin, Artificial Intelligence-Enabled Electrocardiography Detects Hypoalbuminemia and Identifies the Mechanism of Hepatorenal and Cardiovascular Events, Frontiers in cardiovascular medicine 9 (2022) 895201.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  35. P.A. Glassman, B. Simon, P. Belperio, A. Lanto, Improving recognition of drug interactions: benefits and barriers to using automated drug alerts, Medical care 40(12) (2002) 1161-71.

    Article  PubMed  Google Scholar 

  36. K.C. Siontis, P.A. Noseworthy, Z.I. Attia, P.A. Friedman, Artificial intelligence-enhanced electrocardiography in cardiovascular disease management, Nature reviews. Cardiology 18(7) (2021) 465-478.

    Article  PubMed  PubMed Central  Google Scholar 

  37. E. Çallı, E. Sogancioglu, B. van Ginneken, K.G. van Leeuwen, K. Murphy, Deep learning for chest X-ray analysis: A survey, Medical image analysis 72 (2021) 102125.

    Article  PubMed  Google Scholar 

  38. T. Isaac, J.S. Weissman, R.B. Davis, M. Massagli, A. Cyrulik, D.Z. Sands, S.N. Weingart, Overrides of medication alerts in ambulatory care, Archives of internal medicine 169(3) (2009) 305-11.

    Article  PubMed  Google Scholar 

  39. D.J. Tsai, S.H. Tsai, H.H. Chiang, C.C. Lee, S.J. Chen, Development and Validation of an Artificial Intelligence Electrocardiogram Recommendation System in the Emergency Department, J Pers Med 12(5) (2022).

Download references

Funding

This study was supported by funding from the Ministry of Science and Technology, Taiwan (MOST110-2314-B-016-010-MY3 to Chin Lin and MOST111-2321-B-016-003 to Chih-Hung Wang), the National Science and Technology Council, Taiwan (NSTC112-2314-B-016-050 to Sy-Jou Chen), the Cheng Hsin General Hospital, Taiwan (CHNDMC-111-07 to Chin Lin), Medical Affairs Bureau, Ministry of Natioal Defense, Taiwan (MND-MAB-110-113, MND-MAB-D-111045, and MND-MAB-C13-112050 to Chin Lin; MND-MAB-C13-112053 to Sy-Jou Chen), Tri-Serive General Hospital, Taiwan (TSGH-E-112215 to Sy-Jou Chen).

Author information

Authors and Affiliations

  1. School of Public Health, National Defense Medical Center, Taipei, Taiwan

    Yu-Hsuan Jamie Chen & Chin Lin

  2. Division of Cardiology, Department of Internal Medicine, Tri-Service General Hospital, National Defense Medical Center Taipei, Taipei, Taiwan

    Chin-Sheng Lin

  3. Medical Technology Education Center, School of Medicine, National Defense Medical Center, Taipei, Taiwan

    Chin-Sheng Lin & Chin Lin

  4. Graduate Institutes of Life Sciences, National Defense Medical Center, Taipei, Taiwan

    Chin Lin & Chih-Hung Wang

  5. Center for Artificial Intelligence and Internet of Things, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan

    Dung-Jang Tsai & Wen-Hui Fang

  6. Department of Statistics and Information Science, Fu Jen Catholic University, New Taipei City, Taiwan

    Dung-Jang Tsai

  7. Department of Family and Community Medicine, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan

    Wen-Hui Fang

  8. Medical Informatics Office, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan

    Chia-Cheng Lee

  9. Division of Colorectal Surgery, Department of Surgery, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan

    Chia-Cheng Lee

  10. Department of Otolaryngology-Head and Neck Surgery, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan

    Chih-Hung Wang

  11. Department of Emergency Medicine, Tri-Service General Hospital, National Defense Medical Center, No.161, Sec. 6, Minquan E. Rd., Neihu Dist., Taipei City, 11490, Taiwan

    Sy-Jou Chen

  12. Graduate Institute of Injury Prevention and Control, College of Public Health and Nutrition, Taipei Medical University, Taipei, Taiwan

    Sy-Jou Chen

Authors
  1. Yu-Hsuan Jamie Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chin-Sheng Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chin Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dung-Jang Tsai
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wen-Hui Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Chia-Cheng Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Chih-Hung Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Sy-Jou Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors participated in designing the study, generating hypotheses, interpreting the data, and critically reviewing the paper. YHJC and SJC wrote the first draft, and CSL, WHF, CCL, and CHW contributed substantially to writing subsequent versions. YHJC designed and conducted statistical analyses with support from CL and DJT. All authors had full access to all the data in the study and accepted responsibility for the decision to submit for publication. YHJC and SJC verified all the data used in this study. The corresponding author (SJC) attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

Corresponding author

Correspondence to Sy-Jou Chen.

Ethics declarations

Ethics approval

The Tri-Service General Hospital, Taipei, Taiwan, conducted the ethical review of this study (IRB No. C202105049). The institutional review board agreed to waive individual consent since the data were collected retrospectively and analyzed on the intranet.

Conflict of interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (PDF 1784 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, YH.J., Lin, CS., Lin, C. et al. An AI-Enabled Dynamic Risk Stratification for Emergency Department Patients with ECG and CXR Integration. J Med Syst 47, 81 (2023). https://doi.org/10.1007/s10916-023-01980-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10916-023-01980-x

Keywords

Search

Navigation