Abstract
Clinical event sequences in Electronic Health Records (EHRs) record detailed information about the patient condition and patient care as they occur in time. Recent years have witnessed increased interest of machine learning community in developing machine learning models solving different types of problems defined upon information in EHRs. More recently, neural sequential models, such as RNN and LSTM, became popular and widely applied models for representing patient sequence data and for predicting future events or outcomes based on such data. However, a single neural sequential model may not properly represent complex dynamics of all patients and the differences in their behaviors. In this work, we aim to alleviate this limitation by refining a one-fits-all model using a Mixture-of-Experts (MoE) architecture. The architecture consists of multiple (expert) RNN models covering patient sub-populations and refining the predictions of the base model. That is, instead of training expert RNN models from scratch we define them on the residual signal that attempts to model the differences from the population-wide model. The heterogeneity of various patient sequences is modeled through multiple experts that consist of RNN. Particularly, instead of directly training MoE from scratch, we augment MoE based on the prediction signal from pretrained base GRU model. With this way, the mixture of experts can provide flexible adaptation to the (limited) predictive power of the single base RNN model. We experiment with the newly proposed model on real-world EHRs data and the multivariate clinical event prediction task. We implement RNN using Gated Recurrent Units (GRU). We show 4.1% gain on AUPRC statistics compared to a single RNN prediction.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bajor, J.M., Lasko, T.A.: Predicting medications from diagnostic codes with recurrent neural networks. In: ICLR (2017)
Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv:1406.1078 (2014)
Choi, E., et al.: Medical concept representation learning from electronic health records and its application on heart failure prediction. arXiv:1602.03686 (2016)
Choi, E., et al.: RETAIN: an interpretable predictive model for healthcare using reverse time attention mechanism. In: Advances in NeurIPS (2016)
Choi, E., et al.: Using recurrent neural network models for early detection of heart failure onset. J. AMIA 24(2), 361–370 (2017)
Choi, Y., et al.: Learning low-dimensional representations of medical concepts. AMIA Summits Transl. Sci. Proc. 2016, 41 (2016)
Fojo, A.T., et al.: A precision medicine approach for psychiatric disease based on repeated symptom scores. J. Psychiatr. Res. 95, 147–155 (2017)
Grave, E., et al.: Unbounded cache model for online language modeling with open vocabulary. In: Advances in NeurIPS, pp. 6042–6052 (2017)
Hauskrecht, M., Batal, I., Valko, M., Visweswaran, S., Cooper, G.F., Clermont, G.: Outlier detection for patient monitoring and alerting. J. Biomed. Inform. 46(1), 47–55 (2013)
Hauskrecht, M., et al.: Outlier-based detection of unusual patient-management actions: an ICU study. J. Biomed. Inform. 64, 211–221 (2016)
Henry, K.E., Hager, D.N., Pronovost, P.J., Saria, S.: A targeted real-time early warning score (trewscore) for septic shock. Sci. Transl. Med. (2015)
Huang, Z., et al.: Medical inpatient journey modeling and clustering: a Bayesian hidden Markov model based approach. In: AMIA, vol. 2015 (2015)
Huang, Z., et al.: Similarity measure between patient traces for clinical pathway analysis: problem, method, and applications. IEEE J-BHI 18, 4–14 (2013)
Jacobs, R.A., Jordan, M.I., Nowlan, S.J., Hinton, G.E.: Adaptive mixtures of local experts. Neural Comput. 3(1), 79–87 (1991)
Johnson, A.E., et al.: MIMIC-III, a freely accessible critical care database. Sci. Data 3, 160035 (2016)
Kingma, D.P., Ba, J.: A method for stochastic optimization. arXiv:1412.6980 (2014)
Krause, B., et al.: Dynamic evaluation of neural sequence models. In: International Conference on Machine Learning, pp. 2766–2775 (2018)
Lee, J.M., Hauskrecht, M.: Recent context-aware LSTM for clinical event time-series prediction. In: Riaño, D., Wilk, S., ten Teije, A. (eds.) AIME 2019. LNCS (LNAI), vol. 11526, pp. 13–23. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-21642-9_3
Lee, J.M., Hauskrecht, M.: Clinical event time-series modeling with periodic events. In: The 33rd International FLAIRS Conference (2020)
Lee, J.M., Hauskrecht, M.: Multi-scale temporal memory for clinical event time-series prediction. In: Michalowski, M., Moskovitch, R. (eds.) AIME 2020. LNCS (LNAI), vol. 12299, pp. 313–324. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59137-3_28
Lee, J.M., Hauskrecht, M.: Modeling multivariate clinical event time-series with recurrent temporal mechanisms. Artif. Intell. Med. (2021)
Lee, J.M., Hauskrecht, M.: Neural clinical event sequence prediction through personalized online adaptive learning. In: Tucker, A., Henriques Abreu, P., Cardoso, J., Pereira Rodrigues, P., Riaño, D. (eds.) AIME 2021. LNCS (LNAI), vol. 12721, pp. 175–186. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-77211-6_20
Liu, S., Hauskrecht, M.: Nonparametric regressive point processes based on conditional Gaussian processes. In: Advances in NeurIPS (2019)
Liu, Z., Hauskrecht, M.: Learning adaptive forecasting models from irregularly sampled multivariate clinical data. In: The 30th AAAI Conference (2016)
Malakouti, S., Hauskrecht, M.: Hierarchical adaptive multi-task learning framework for patient diagnoses and diagnostic category classification. In: IEEE BIBM (2019)
Malakouti, S., Hauskrecht, M.: Predicting patient’s diagnoses and diagnostic categories from clinical-events in EHR data. In: Riaño, D., Wilk, S., ten Teije, A. (eds.) AIME 2019. LNCS (LNAI), vol. 11526, pp. 125–130. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-21642-9_17
Miotto, R., Li, L., Kidd, B.A., Dudley, J.T.: Deep patient: an unsupervised representation to predict the future of patients from the electronic health records. Sci. Rep. 6, 26094 (2016)
Nguyen, P., Tran, T., Venkatesh, S.: Finding algebraic structure of care in time: a deep learning approach. arXiv abs/1711.07980 (2017)
Nguyen, P., et al.: Deepr: a convolutional net for medical records. IEEE J. Biomed. Health Inform. 21(1), 22–30 (2016)
Pham, T., et al.: Predicting healthcare trajectories from medical records: a deep learning approach. J. Biomed. Inform. 69, 218–229 (2017)
Saito, T., Rehmsmeier, M.: The precision-recall plot is more informative than ROC plot when evaluating binary classifiers on imbalanced datasets. PloS One (2015)
Visweswaran, S., Cooper, G.F.: Instance-specific Bayesian model averaging for classification. In: Advances in NeurIPS (2005)
Yu, K., et al.: Monitoring ICU mortality risk with a long short-term memory recurrent neural network. In: Pacific Symposium on Biocomputing. World Scientific (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendix: Event-Specific Prediction Results (AURPC)
Appendix: Event-Specific Prediction Results (AURPC)
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Lee, J.M., Hauskrecht, M. (2022). Learning to Adapt Dynamic Clinical Event Sequences with Residual Mixture of Experts. In: Michalowski, M., Abidi, S.S.R., Abidi, S. (eds) Artificial Intelligence in Medicine. AIME 2022. Lecture Notes in Computer Science(), vol 13263. Springer, Cham. https://doi.org/10.1007/978-3-031-09342-5_15
Download citation
DOI: https://doi.org/10.1007/978-3-031-09342-5_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-09341-8
Online ISBN: 978-3-031-09342-5
eBook Packages: Computer ScienceComputer Science (R0)