Abstract
Emergency Departments (EDs) are the most overcrowded places in public hospitals. Machine learning can support decisions on effective ED resource management by accurately forecasting the number of ED visits. In addition, Explainable Artificial Intelligence (XAI) techniques can help explain decisions from forecasting models and address challenges like lack of trust in machine learning results. The objective of this paper is to use machine learning and XAI to forecast and explain the ED visits on the next on duty day. Towards this end, a case study is presented that uses the XGBoost algorithm to create a model that forecasts the number of patient visits to the ED of the University Hospital of Ioannina in Greece, based on historical data from patient visits, time-based data, dates of holidays and special events, and weather data. The SHapley Additive exPlanations (SHAP) framework is used to explain the model. The evaluation of the forecasting model resulted in an MAE value of 18.37, revealing a more accurate model than the baseline, with an MAE of 29.38. The number of patient visits is mostly affected by the day of the week of the on duty day, the mean number of visits in the previous four on duty days, and the maximum daily temperature. The results of this work can help policy makers in healthcare make more accurate and transparent decisions that increase the trust of people affected by them (e.g., medical staff).
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
The datasets generated and/or analysed during the current study are not publicly available due to confidentiality, but are available from the corresponding author upon reasonable request.
References
Aboagye-Sarfo, P., Mai, Q., Sanfilippo, F. M., Preen, D. B., Stewart, L. M., & Fatovich, D. M. (2015). A comparison of multivariate and univariate time series approaches to modelling and forecasting emergency department demand in western australia. Journal of Biomedical Informatics, 57, 62–73. https://doi.org/10.1016/j.jbi.2015.06.022.
Afilal, M., Yalaoui, F., Dugardin, F., Amodeo, L., Laplanche, D., & Blua, P. (2016). Forecasting the emergency department patients flow. Journal of Medical Systems, 40(7), 1–18. https://doi.org/10.1007/s10916-016-0527-0.
Aroua, A., & Abdul-Nour, G. (2015). Forecast emergency room visits–a major diagnostic categories based approach. International Journal of Metrology and Quality Engineering, 6(2), 204. https://doi.org/10.1051/ijmqe/2015011.
Asheim, A., Bache-Wiig Bjrnsen, L. P., Nss-Pleym, L. E., Uleberg, O., Dale, J., & Nilsen, S. M. (2019). Real-time forecasting of emergency department arrivals using prehospital data. BMC Emergency Medicine, 19(1), 1–6. https://doi.org/10.1186/s12873-019-0256-z.
Asplin, B. R., Magid, D. J., Rhodes, K. V., Solberg, L. I., Lurie, N., & Camargo, C. A. Jr (2003). A conceptual model of emergency department crowding. Annals of Emergency Medicine, 42(2), 173–180.
Bergs, J., Heerinckx, P., & Verelst, S. (2014). Knowing what to expect, forecasting monthly emergency department visits: a time-series analysis. International Emergency Nursing, 22(2), 112–115. https://doi.org/10.1016/j.ienj.2013.08.001.
Box, G. E. P., Jenkins, G. M., Reinsel, G. C., & Ljung, G. M. (2015). Time series analysis: Forecasting and control. Wiley series in probability and statistics wiley. https://doi.org/10.1111/jtsa.12194.
Butler, M., Gu, H., Kenney, T., & Campbell, S. (2016). P017: Does a busy day predict another busy day? a time-series analysis of multi-centre emergency department volumes. Canadian Journal of Emergency Medicine, 18(S1), 83–84. https://doi.org/10.1017/cem.2016.193.
Calegari, R., Fogliatto, F. S., Lucini, F. R., Neyeloff, J., Kuchenbecker, R. S., & Schaan, B. D. (2016). Forecasting daily volume and acuity of patients in the emergency department. Computational and Mathematical Methods in Medicine 2016. https://doi.org/10.1155/2016/3863268.
Chen, T., & Guestrin, C. (2016). Xgboost: a scalable tree boosting system. In Proceedings of the 22nd Acm Sigkdd International Conference on Knowledge Discovery and Data Mining, pp. 785–794. https://doi.org/10.1145/2939672.2939785.
Cheng, Q., Argon, N. T., Evans, C. S., Liu, Y., Platts-Mills, T. F., & Ziya, S. (2021). Forecasting emergency department hourly occupancy using time series analysis. The American Journal of Emergency Medicine, 48, 177–182. https://doi.org/10.1016/j.ajem.2021.04.075.
Cremonesi, P., di Bella, E., Montefiori, M., & Persico, L. (2015). The robustness and effectiveness of the triage system at times of overcrowding and the extra costs due to inappropriate use of emergency departments. Applied Health Economics and Health Policy, 13(5), 507–514. https://doi.org/10.1007/s40258-015-0166-5.
Davis, R. E., Markle, E. S., Windoloski, S., Houck, M. E., Enfield, K. B., Kang, H., Balling, Jr R. C., Kuehl, D. R., Burton, J. H., Farthing, W., & et al. (2020). A comparison of the effect of weather and climate on emergency department visitation in roanoke and charlottesville, virginia. Environmental Research, 191, 110065. https://doi.org/10.1016/j.envres.2020.110065.
Deeks, A. (2019). The judicial demand for explainable artificial intelligence. Columbia Law Review, 119(7), 1829–1850.
Diehl, A. K., Morris, M. D., & Mannis, S. A. (1981). Use of calendar and weather data to predict walk-in attendance. Southern Medical Journal, 74 (6), 709–712. https://doi.org/10.1097/00007611-198106000-00020.
Durbin, J., & Watson, G. S. (1950). Testing for serial correlation in least squares regression: I. Biometrika, 37(3/4), 409–428. https://doi.org/10.2307/2332391.
Durbin, J., & Watson, G. S. (1951). Testing for serial correlation in least squares regression: Ii. Biometrika, 38(1/2), 159–177. https://doi.org/10.2307/2332325.
Duwalage, K. I., Burkett, E., White, G., Wong, A., & Thompson, M. H. (2020). Forecasting daily counts of patient presentations in australian emergency departments using statistical models with time-varying predictors. Emergency Medicine Australasia, 32(4), 618–625. https://doi.org/10.1111/1742-6723.13481.
Ekström, A., Kurland, L., Farrokhnia, N., Castrén, M., & Nordberg, M. (2015). Forecasting emergency department visits using internet data. Annals of Emergency Medicine, 65(4), 436–442. https://doi.org/10.1016/j.annemergmed.2014.10.008.
Erkamp, N. S., van Dalen, D. H., & de Vries, E. (2021). Predicting emergency department visits in a large teaching hospital. International Journal of Emergency Medicine, 14(1), 1–11. https://doi.org/10.1186/s12245-021-00357-6.
Freund, Y., & Schapire, R. E. (1997). A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1), 119–139. https://doi.org/10.1006/jcss.1997.1504.
Galanis, P., Siskou, O., Charalambous, G., Konstantakopoulou, O., Liarigkovinou, A., Karagkouni, I., Tsolas, G., Chatzi, P., Giotaki, A., Keramidas, G., & et al. (2019). Inappropriate use of public hospitals emergency departments in greece: magnitude and associated factors. Stud Health Technol Inform, 262, 224–227. https://doi.org/10.3233/SHTI190059.
Ghada, W., Estrella, N., Pfoerringer, D., Kanz, K. -G., Bogner-Flatz, V., Ankerst, D. P., & Menzel, A. (2021). Effects of weather, air pollution and oktoberfest on ambulance-transported emergency department admissions in munich, germany. Science of The Total Environment, 755, 143772. https://doi.org/10.1016/j.scitotenv.2020.143772.
Gul, M., & Celik, E. (2018). An exhaustive review and analysis on applications of statistical forecasting in hospital emergency departments. Health Systems, 9, 263–284. https://doi.org/10.1080/20476965.2018.1547348.
Gul, M., & Guneri, A. F. (2016). Planning the future of emergency departments: Forecasting ed patient arrivals by using regression and neural network models. International Journal of Industrial Engineering 23(2).
Harrou, F., Dairi, A., Kadri, F., & Sun, Y. (2020). Forecasting emergency department overcrowding: a deep learning framework. Chaos, Solitons & Fractals, 139, 110247. https://doi.org/10.1016/j.chaos.2020.110247.
Janssen, M., & Kuk, G. (2016). The challenges and limits of big data algorithms in technocratic governance. Government Information Quarterly, 33 (3), 371–377. https://doi.org/10.1016/j.giq.2016.08.011. Open and Smart Governments: Strategies, Tools, and Experiences.
Jones, S. S., Thomas, A., Evans, R. S., Welch, S. J., Haug, P. J., & Snow, G. L. (2008). Forecasting daily patient volumes in the emergency department. Academic Emergency Medicine, 15(2), 159–170. https://doi.org/10.1111/j.1553-2712.2007.00032.x.
Kadri, F., & Abdennbi, K. (2020). Rnn-based deep-learning approach to forecasting hospital system demands: application to an emergency department. International Journal of Data Science, 5(1), 1–25. https://doi.org/10.1016/j.mlwa.2021.100200.
Kalampokis, E., Karamanou, A., & Tarabanis, K. (2021). Applying explainable artificial intelligence techniques on linked open government data. In H.J. Scholl, J.R. Gil-garcia, M. Janssen, E. Kalampokis, I. Lindgren, & M.P Rodríguez Bolívar (Eds.) Electronic Government, pp. 247–258. Springer, Cham.
Kam, H. J., Sung, J. O., & Park, R. W. (2010). Prediction of daily patient numbers for a regional emergency medical center using time series analysis. Healthcare Informatics Research, 16(3), 158–165. https://doi.org/10.4258/hir.2010.16.3.158.
Kang, S. W., & Park, H. S. (2015). Emergency department visit volume variability. Clinical and Experimental Emergency Medicine, 2(3), 150. https://doi.org/10.15441/ceem.14.044.
Khaldi, R., Afia, A. E., & Chiheb, R. (2019). Forecasting of weekly patient visits to emergency department: real case study. Procedia Computer Science, 148, 532–541. https://doi.org/10.1016/j.procs.2019.01.026. The Second International Conference on Intelligent Computing in Data Sciences, ICDS2018.
Lee, D. (2019). Effects of key value co-creation elements in the healthcare system: focusing on technology applications. Service Business, 13(2), 389–417. https://doi.org/10.1007/s11628-018-00388-9.
López Ibáñez, B., Torrent-Fontbona, F., Roman, J., & Inoriza, J. M. (2021). Forecasting of emergency department attendances in a tourist region with an operational time horizon. Information Systems. In press.
Mai, Q., Aboagye-Sarfo, P., Sanfilippo, F. M., Preen, D. B., & Fatovich, D. M. (2015). Predicting the number of emergency department presentations in western australia: A population-based time series analysis. Emergency Medicine Australasia, 27(1), 16–21. https://doi.org/10.1111/1742-6723.12344.
Mardini, M. T., & Raś, Z. W. (2019). Extraction of actionable knowledge to reduce hospital readmissions through patients personalization. Information Sciences, 485, 1–17. https://doi.org/10.1016/j.ins.2019.02.006.
Morris, Z. S., Boyle, A., Beniuk, K., & Robinson, S. (2012). Emergency department crowding: towards an agenda for evidence-based intervention. Emergency Medicine Journal, 29(6), 460–466. https://doi.org/10.1136/emj.2010.107078.
Moukarzel, A., Michelet, P., Durand, A. -C., Sebbane, M., Bourgeois, S., Markarian, T., Bompard, C., & Gentile, S. (2019). Burnout syndrome among emergency department staff: prevalence and associated factors. BioMed Research International 2019. https://doi.org/10.1155/2019/6462472.
OECD. (2019). Heath at a Glance 2019: OECD Indicators. https://doi.org/10.1787/4dd50c09-en.
Padthe, K. K., Kumar, V., Eckert Carly M. MD MPH, M. N. M., Zahid, A., Aurangzeb, A. M., & Teredesai, A. (2020). Emergency department optimization and load prediction in hospitals. In Proceedings of the AAAI Fall Symposium on AI for Social Good 2884.
Petrou, P. (2015). An interrupted time-series analysis to assess impact of introduction of co-payment on emergency room visits in cyprus. Applied Health Economics and Health Policy, 13(5), 515–523. https://doi.org/10.1007/s40258-015-0169-2.
Pines, J. M., Hilton, J. A., Weber, E. J., Alkemade, A. J., Al Shabanah, H., Anderson, P. D., Bernhard, M., Bertini, A., Gries, A., Ferrandiz, S., & et al. (2011). International perspectives on emergency department crowding. Academic Emergency Medicine, 18(12), 1358–1370. https://doi.org/10.1111/j.1553-2712.2011.01235.x.
Ras, Z. (2022). Reduction of hospital readmissions. Advances in Clinical and Experimental Medicine, 31(1), 5–8. https://doi.org/10.17219/acem/144413.
Rocha, C. N., & Rodrigues, F. (2021). Forecasting emergency department admissions. Journal of Intelligent Information Systems, 56(3), 509–528. https://doi.org/10.1007/s10844-021-00638-9.
Rosychuk, R. J., Youngson, E., & Rowe, B. H. (2015). Presentations to alberta emergency departments for asthma: a time series analysis. Academic Emergency Medicine, 22(8), 942–949. https://doi.org/10.1111/acem.12725.
Rosychuk, R. J., Youngson, E., & Rowe, B. H. (2016). Presentations to emergency departments for copd: a time series analysis. Canadian Respiratory Journal 2016. https://doi.org/10.1155/2016/1382434.
Shapley, L. S. (1953). A value for n-person games. Contributions to the Theory of Games, II, 307–318. https://doi.org/10.1515/9781400881970-018.
Slack, D., Hilgard, S., Jia, E., Singh, S., & Lakkaraju, H. (2020). Fooling lime and shap: Adversarial attacks on post hoc explanation methods. In Proceedings of the AAAI ACM Conference on AI, Ethics, and Society, pp. 180–186.
Statement, P. (2019). Crowding. Annals of Emergency Medicine, 74(3), 31. https://doi.org/10.1016/j.annemergmed.2019.05.003.
Sudarshan, V. K., Brabrand, M., Range, T. M., & Wiil, U. K. (2021). Performance evaluation of emergency department patient arrivals forecasting models by including meteorological and calendar information: A comparative study. Computers in Biology and Medicine 104541. https://doi.org/10.1016/j.compbiomed.2021.104541.
Sun, Y., Heng, B. H., Seow, Y. T., & Seow, E. (2009). Forecasting daily attendances at an emergency department to aid resource planning. BMC Emergency Medicine, 9(1), 1–9. https://doi.org/10.1186/1471-227x-9-1.
Sun, T. Q., & Medaglia, R. (2019). Mapping the challenges of artificial intelligence in the public sector: Evidence from public healthcare. Government Information Quarterly, 36(2), 368–383. https://doi.org/10.1016/j.giq.2018.09.008.
Vollmer, M. A., Glampson, B., Mellan, T., Mishra, S., Mercuri, L., Costello, C., Klaber, R., Cooke, G., Flaxman, S., & Bhatt, S. (2021). A unified machine learning approach to time series forecasting applied to demand at emergency departments. BMC Emergency Medicine, 21(1), 1–14. https://doi.org/10.1186/s12873-020-00395-y.
World Bank. (2020). Artificial intelligence in the public sector: Maximizing opportunities managing risks.
Xu, Q., Tsui, K. -L., Jiang, W., & Guo, H. (2016). A hybrid approach for forecasting patient visits in emergency department. Quality and Reliability Engineering International, 32(8), 2751–2759. https://doi.org/10.1002/qre.2095.
Zlotnik, A., Gallardo-Antolin, A., Alfaro, M. C., Pérez, M. C. P., Martínez, J. M. M., & et al. (2015). Emergency department visit forecasting and dynamic nursing staff allocation using machine learning techniques with readily available open-source software. CIN: Computers, Informatics Nursing, 33(8), 368–377. https://doi.org/10.1097/cin.0000000000000173.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interests
The authors declare that they have no conflict of interest.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Petsis, S., Karamanou, A., Kalampokis, E. et al. Forecasting and explaining emergency department visits in a public hospital. J Intell Inf Syst 59, 479–500 (2022). https://doi.org/10.1007/s10844-022-00716-6
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10844-022-00716-6