Abstract
Patients with similar conditions in the intensive care unit (ICU) may have different reactions for a given treatment. An effective personalised medicine can help save patient lives. The availability of recorded ICU data provides a huge potential to train and develop the systems. However, there is no ground truth of best treatments. This makes existing supervised learning based methods are not appropriate. In this paper, we proposed clustering based Bayesian reinforcement learning. Firstly, we transformed the multivariate time series patient record into a real-time Patient Sequence Model (PSM). After that, we computed the likelihood probability of treatments effect for all patients and cluster them based on that. Finally, we computed Bayesian reinforcement learning to derive personalised policies. We tested our proposed method using 11,791 ICU patients records from MIMIC-III database. Results show that we are able to cluster patient based on their treatment effects. In addition, our method also provides better explainability and time-critical recommendation that are very important in a real ICU setting.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Almirall, D., Compton, S.N., Gunlicks-Stoessel, M., Duan, N., Murphy, S.A.: Designing a pilot sequential multiple assignment randomized trial for developing an adaptive treatment strategy. Stat. Med. 31(17), 1887–1902 (2012)
Angus, D.C., van der Poll, T.: Severe sepsis and septic shock. New England J. Med. 369(9), 840–851 (2013)
Bajor, J.M., Lasko, T.A.: Predicting medications from diagnostic codes with recurrent neural networks. In: Proceeding of the 5th International Conference on Learning Representations - (ICLR 2017), pp. 1–19 (2017)
Cheerla, N., Gevaert, O.: MicroRNA based pan-cancer diagnosis and treatment recommendation. BMC Bioinf. 18(1), 32 (2017)
Chen, Z., Marple, K., Salazar, E., Gupta, G., Tamil, L.: A physician advisory system for chronic heart failure management based on knowledge patterns. Theory Pract. Logic Program. 16(5–6), 604–618 (2016)
Hu, J., Perer, A., Wang, F.: Data driven analytics for personalized healthcare. In: Weaver, C.A., Ball, M.J., Kim, G.R., Kiel, J.M. (eds.) Healthcare Information Management Systems. HI, pp. 529–554. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-20765-0_31
Johnson, A.E.W., et al.: A comparative analysis of sepsis identification methods in an electronic database. Crit. Care Med. 46(4), 494–499 (2018)
Johnson, A.E., Pollard, T.J., Shen, L., et al.: MIMIC-III, a freely accessible critical care database. Sci. Data 3, 160035 (2016)
Nemati, S., Ghassemi, M.M., Clifford, G.D.: Optimal medication dosing from suboptimal clinical examples: a deep reinforcement learning approach. In: Proceeding of the 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC 2016), Orlando, FL, USA , pp. 2978–2981, August 2016
Pollard, S., Edwin, S.B., Alaniz, C.: Vasopressor and inotropic management of patients with septic shock. Pharm. Ther. 40(7), 438–50 (2015)
Raghu, A., Komorowski, M., Ahmed, I., Celi, L., Szolovits, P., Ghassemi, M.: Deep Reinforcement Learning for Sepsis Treatment. In: Proceeding of the 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA (2017)
Rhodes, A., Evans, L.E., Alhazzani, W., et al.: Surviving sepsis campaign: international guidelines for management of sepsis and septic shock: 2016. Intensive Care Med. 43(3), 304–377 (2017)
Sakr, Y., Jaschinski, U., Wittebole, X., et al.: Sepsis in intensive care unit patients: worldwide data from the intensive care over nations audit. Open Forum Infect. Dis. 5(12), ofy313 (2018)
Shen, R., Olshen, A.B., Ladanyi, M.: Integrative clustering of multiple genomic data types using a joint latent variable model with application to breast and lung cancer subtype analysis. Bioinformatics 25(22), 2906–2912 (2009)
Singer, M., Deutschman, C.S., Seymour, C., et al.: The third international consensus definitions for sepsis and septic shock (sepsis-3). JAMA - J. Am. Med. Assoc. 315(8), 801–810 (2016)
Tsoukalas, A., Albertson, T., Tagkopoulos, I.: From data to optimal decision making: a data-driven, probabilistic machine learning approach to decision support for patients with sepsis. JMIR Med. Inf. 3(1), e11 (2015)
Acknowledgements
Chandra Prasetyo Utomo is sponsored by Indonesia Endowment Fund for Education (LPDP). This research is partially supported by Australian Research Council (ARC) project ID: DP160104075 and Universitas YARSI, Jakarta. The authors thank Dr. Robert Boots from Royal Brisbane and Women’s Hospital (RBWH) for valuable insight in treatment recommendation research.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Utomo, C.P., Kurniawati, H., Li, X., Pokharel, S. (2019). Personalised Medicine in Critical Care Using Bayesian Reinforcement Learning. In: Li, J., Wang, S., Qin, S., Li, X., Wang, S. (eds) Advanced Data Mining and Applications. ADMA 2019. Lecture Notes in Computer Science(), vol 11888. Springer, Cham. https://doi.org/10.1007/978-3-030-35231-8_47
Download citation
DOI: https://doi.org/10.1007/978-3-030-35231-8_47
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-35230-1
Online ISBN: 978-3-030-35231-8
eBook Packages: Computer ScienceComputer Science (R0)