Skip to main content
SpringerLink
Log in

Predicting Outpatient Appointment Demand Using Machine Learning and Traditional Methods

  • Systems-Level Quality Improvement
  • Published:
Journal of Medical Systems Aims and scope Submit manuscript

Abstract

Traditional methods have long been used for clinical demand forecasting. Machine learning methods represent the next evolution in forecasting, but model choice and optimization remain challenging for achieving optimal results. To determine the best method to predict demand for outpatient appointments comparing machine learning and traditional methods, this retrospective study analyzed “appointment requests” at a major outpatient department in a destination medical center. Two separate locations (A and B) were assessed with 20 traditional, hybrid (traditional + machine learning) and machine learning methods to determine the best forecasting outcome (lowest Forecast Standard Error, FSE). Data characteristics from both datasets were examined. 20 forecasting models were then assessed and compared for the best result. Location A’s data displayed a cyclical and non-trending pattern while Location B’s displayed a cyclical and trending pattern. Both Location A and B yielded the feature engineered XGBoost model (machine learning) with the lowest out-of-sample FSE. It is important to carefully analyze and understand the underlying data set pattern and then test a variety of traditional, machine learning, and hybrid prediction methods to achieve optimal predictive results. Additionally, the use of feature engineering or hybrid methods can augment the usefulness of machine learning methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. DeLurgio, S., Denton, B., Cabanela, R. et al., Forecasting weekly outpatient demands at clinics within a large medical center. Production and Inventory Management Journal 45(2):35–46, 2009.

    Google Scholar 

  2. Brownlee, J., Deep Learning for Time Series Forecasting, 2018. Retrieved from https://machinelearningmastery.com/deep-learning-for-time-series-forecasting/

  3. Jain, A., What I learnt about Time Series Analysis in 3 hour Mini DataHack?, 2016. https://www.analyticsvidhya.com/blog/2016/02/hand-learn-time-series-3-hours-mini-datahack/

  4. How to Compute the Estimated SEs of Forecasted Y and the Forecast Error. www3.wabash.edu/econometrics/econometricsbook/.../HowToFindSEForecast.doc

  5. XGBoost developers. Introduction to Boosted Trees. XGBoost, 2016. https://xgboost.readthedocs.io/en/latest/tutorials/model.html. Accessed 28 December 2018

  6. Abdus-Salaam, H., Davis, L., de Oliveira Mota, D., Modeling dependent demand arrivals within an open-access scheduling system. Paper presented at: Science and Technology for Humanity (TIC-STH), 2009 IEEE Toronto International Conference, 2009.

  7. Abraham, G., Byrnes, G. B., and Bain, C. A., Short-term forecasting of emergency inpatient flow. IEEE transactions on information technology in biomedicine: a publication of the IEEE Engineering in Medicine and Biology Society. 13(3):380–388, 2009.

    Article  Google Scholar 

  8. Afilal, M., Yalaoui, F., Dugardin, F., Amodeo, L., Laplanche, D., and Blua, P., Forecasting the Emergency Department Patients Flow. J Med Syst. 40(7):175, 2016.

    Article  PubMed  Google Scholar 

  9. Ahmadi-Javid, A., Jalali, Z., and Klassen, K. J., Outpatient appointment systems in healthcare: A review of optimization studies. European Journal of Operational Research. 258(1):3–34, 2017.

    Article  Google Scholar 

  10. Akyuz, A. O., Uysal, M., Bulbul, B. A., Uysal, M. O., Ensemble approach for time series analysis in demand forecasting: Ensemble learning. Paper presented at: 2017 IEEE International Conference on Innovations in Intelligent SysTems and Applications (INISTA), 2017.

  11. Al-Haque, S., Ceyhan, M. E., Chan, S. H., and Nightingale, D. J., Responding to traveling patients' seasonal demand for health care services. Mil Med. 180(1):111–117, 2015.

    Article  PubMed  Google Scholar 

  12. Bouckaert, N., Van den Heede, K., and Van de Voorde, C., Improving the forecasting of hospital services: A comparison between projections and actual utilization of hospital services. Health Policy. 122(7):728–736, 2018.

    Article  PubMed  Google Scholar 

  13. Boutsioli, Z., Measuring Unexpected Hospital Demand: The Application of a Univariate Model to Public Hospitals in Greece. Hospital Topics. 87(4):14–21, 2009.

    Article  PubMed  Google Scholar 

  14. Calegari, R., Fogliatto, F. S., Lucini, F. R., Neyeloff, J., Kuchenbecker, R. S., and Schaan, B. D., Forecasting Daily Volume and Acuity of Patients in the Emergency Department. Computational and Mathematical Methods in Medicine. 2016:3863268, 2016.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Cheng, C. Y., Chiang, K. L., and Chen, M. Y., Intermittent Demand Forecasting in a Tertiary Pediatric Intensive Care Unit. J Med Syst. 40(10):217, 2016.

    Article  PubMed  Google Scholar 

  16. Claudio, D., Miller, A., and Huggins, A., Time series forecasting in an outpatient cancer clinic using common-day clustering. IIE Transactions on Healthcare Systems Engineering. 4(1):16–26, 2014.

    Article  Google Scholar 

  17. Côté, M. J., Smith, M. A., Eitel, D. R., and Akçali, E., Forecasting Emergency Department Arrivals: A Tutorial for Emergency Department Directors. Hospital Topics. 91(1):9–19, 2013.

    Article  PubMed  Google Scholar 

  18. Côté, M. J., and Tucker, S. L., Four methodologies to improve healthcare demand forecasting. Healthcare Financial Management. 55(5):54–58, 2001.

    PubMed  Google Scholar 

  19. Heroman, W. M., Davis, C. B., Farmer Jr, K. L., Demand forecasting and capacity management in primary care. Physician executive. 38(1):30–34, 2012. http://ovidsp.ovid.com/ovidweb.cgi?T=JS&PAGE=reference&D=emed13&NEWS=N&AN=369644991

  20. Hong, W. S., Haimovich, A. D., and Taylor, R. A., Predicting hospital admission at emergency department triage using machine learning. PloS one. 13(7):e0201016, 2018.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Kadri, F., Harrou, F., Chaabane, S., and Tahon, C., Time Series Modelling and Forecasting of Emergency Department Overcrowding. J Med Syst. 38(9):1–20, 2014.

    Article  Google Scholar 

  22. Luo, L., Luo, L., Zhang, X., and He, X., Hospital daily outpatient visits forecasting using a combinatorial model based on ARIMA and SES models. BMC Health Serv Res. 17(1):469, 2017.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Mai, Q., Aboagye, S. P., Sanfilippo, F. M., Preen, D. B., and Fatovich, D. M., Predicting the number of emergency department presentations in Western Australia: A population-based time series analysis. Emergency Medicine Australasia. 27(1):16–21, 2015. https://doi.org/10.1111/1742-6723.12344.

    Article  PubMed  Google Scholar 

  24. Ouyang, Y., and Yin, H., Multi-Step Time Series Forecasting with an Ensemble of Varied Length Mixture Models. Int J Neural Syst. 28(4):1750053, 2018.

    Article  PubMed  Google Scholar 

  25. Schweigler, L. M., Desmond, J. S., McCarthy, M. L., Bukowski, K. J., Ionides, E. L., and Younger, J. G., Forecasting models of emergency department crowding. Acad Emerg Med. 16(4):301–308, 2009.

    Article  PubMed  Google Scholar 

  26. Shan, J., Zhang, H., Liu, W., and Liu, Q., Online Active Learning Ensemble Framework for Drifted Data Streams. IEEE Transactions on Neural Networks and Learning Systems.:1–13, 2018. https://doi.org/10.1109/TNNLS.2018.2844332.

    Article  PubMed  Google Scholar 

  27. Sun, Y., Heng, B. H., Seow, Y. T., and Seow, E., Forecasting daily attendances at an emergency department to aid resource planning. BMC Emergency Medicine. 9:1, 2009.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  28. Villani, M., Earnest, A., Nanayakkara, N., Smith, K., de Courten, B., and Zoungas, S., Time series modelling to forecast prehospital EMS demand for diabetic emergencies. BMC Health Services Research. 17:1–9, 2017.

    Article  Google Scholar 

  29. Yu, L., Hang, G., Tang, L., Zhao, Y., and Lai, K., Forecasting Patient Visits to Hospitals using a WD&ANN-based Decomposition and Ensemble Model. Eurasia Journal of Mathematics, Science and Technology Education. 13(12):7615–7627, 2017.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Management Engineering and Internal Consulting, Mayo Clinic, 200 First Street SW, Rochester, MN, 55905, USA

    Brian Klute, Wei Chen & Aaron Stelpflug

  2. Department of Radiology, Mayo Clinic, Rochester, MN, USA

    Andrew Homb

Authors
  1. Brian Klute
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrew Homb
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wei Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Aaron Stelpflug
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wei Chen.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Clinical relevance statement

This study aims to accurately forecast patient appointments by comparing various machine learning, traditional and hybrid methods. The step-by-step approach discussed can help clinics to develop their own patient appointment predictive model, and thus effectively plan the appropriate resources and improve patient satisfaction.

Protection of human and animal subjects

Only retrospectively collected data were used, and no patients were contacted for the study. The study did not involve any protected health information (PHI) and all data points were de-identified.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the Topical Collection on Systems-Level Quality Improvement

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Klute, B., Homb, A., Chen, W. et al. Predicting Outpatient Appointment Demand Using Machine Learning and Traditional Methods. J Med Syst 43, 288 (2019). https://doi.org/10.1007/s10916-019-1418-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10916-019-1418-y

Keywords

Search

Navigation