Shanshan Wang
ABSTRACT
Medicine Combination Prediction (MCP) based on Electronic Health Record (EHR) can assist doctors to prescribe medicines for complex patients. Previous studies on MCP either ignore the correlations between medicines (i.e., MCP is formulated as a binary classifcation task), or assume that there is a sequential correlation between medicines (i.e., MCP is formulated as a sequence prediction task). The latter is unreasonable because the correlations between medicines should be considered in an order-free way. Importantly, MCP must take additional medical knowledge (e.g., Drug-Drug Interaction (DDI)) into consideration to ensure the safety of medicine combinations. However, most previous methods for MCP incorporate DDI knowledge with a post-processing scheme, which might undermine the integrity of proposed medicine combinations. In this paper, we propose a graph convolutional reinforcement learning model for MCP, named Combined Order-free Medicine Prediction Network (CompNet), that addresses the issues listed above. CompNet casts the MCP task as an order-free Markov Decision Process (MDP) problem and designs a Deep Q Learning (DQL) mechanism to learn correlative and adverse interactions between medicines. Specifcally, we frst use a Dual Convolutional Neural Network (Dual-CNN) to obtain patient representations based on EHRs. Then, we introduce the medicine knowledge associated with predicted medicines to create a dynamic medicine knowledge graph, and use a Relational Graph Convolutional Network (R-GCN) to encode it. Finally, CompNet selects medicines by fusing the combination of patient information and the medicine knowledge graph. Experiments on a benchmark dataset, i.e., MIMIC-III, demonstrate that CompNet signifcantly outperforms state-of-the-art methods and improves a recently proposed model by 3.74%pt, 6.64%pt in terms of Jaccard and F1 metrics.
- Oron Anschel, Nir Baram, and Nahum Shimkin. 2017. Averaged-DQN: variance reduction and stabilization for deep reinforcement learning. In ICML 2017 . 176--185.Google Scholar
- Jacek M. Bajor and Thomas A. Lasko. 2017. Predicting medications from diagnostic codes with recurrent neural networks. In ICLR 2017 .Google Scholar
- Remi Besson, Erwan Le Pennec, Stephanie Allassonniere, J Stirnemann, Emmanuel Spaggiari, and Antoine Neuraz. 2018. A model-based reinforcement learning approach for a rare disease diagnostic task. CoRR , Vol. abs/1811.10112 (2018).Google Scholar
- Nikhil Cheerla and Olivier Gevaert. 2017. MicroRNA based Pan-Cancer diagnosis and treatment recommendation. BMC bioinformatics , Vol. 18, 1 (2017), 32.Google Scholar
- Feixiong Cheng, István A Kovács, and Albert-László Barabási. 2019. Network-based prediction of drug combinations. Nature communications , Vol. 10, 1 (2019), 1197.Google Scholar
- Edward Choi, Mohammad Taha Bahadori, Jimeng Sun, Joshua Kulas, Andy Schuetz, and Walter Stewart. 2016. Retain: an interpretable predictive model for healthcare using reverse time attention mechanism. In NIPS 2016. 3504--3512.Google Scholar
- Leslie Citrome. 2009. Quantifying risk: the role of absolute and relative measures in interpreting risk of adverse reactions from product labels of antipsychotic medications. Current drug safety , Vol. 4, 463 (2009), 229--237.Google Scholar
- Victor J Dzau and Celynne A Balatbat. 2018. Health and societal implications of medical and technological advances. Science translational medicine , Vol. 10, 463 (2018), 463.Google Scholar
- David J Eveson, Thompson G Robinson, and John F Potter. 2007. Lisinopril for the treatment of hypertension within the first 24 hours of acute ischemic stroke and follow-up. American J. Hypertension , Vol. 20, 3 (2007), 270--277.Google ScholarCross Ref
- Gregg C Fonarow, R Scott Wright, Frederick A Spencer, Paul D Fredrick, Wei Dong, Nathan Every, William J French, National Registry of Myocardial Infarction 4 Investigators, et almbox. 2005. Effect of statin use within the first 24 hours of admission for acute myocardial infarction on early morbidity and mortality. American J. Cardiology , Vol. 96, 5 (2005), 611--616.Google ScholarCross Ref
- Matthew W. Gardner and Stephen R. Dorling. 1998. Artificial neural networks (the multilayer perceptron): a review of applications in the atmospheric sciences. Atmospheric Environment , Vol. 32, 14--15 (1998), 2627--2636.Google ScholarCross Ref
- Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In AISTATS 2010 . 249--256.Google Scholar
- David N Hager, Varshitha Tanykonda, Zeba Noorain, Sarina K Sahetya, Catherine E Simpson, Juan Felipe Lucena, and Dale M Needham. 2018. Hospital mortality prediction for intermediate care patients: assessing the generalizability of the Intermediate Care Unit Severity Score (IMCUSS). J. Critical Care , Vol. 46 (2018), 94--98.Google ScholarCross Ref
- Haibo He and Edwardo A. Garcia. 2009. Learning from imbalanced data. IEEE Transactions on Knowledge and Data Engineering , Vol. 21, 9 (2009), 1263--1284.Google ScholarDigital Library
- Shiyi He, Chang Xu, Tianyu Guo, Chao Xu, and Dacheng Tao. 2018. Reinforced multi-label image classification by exploring curriculum. In AAAI 2018 . 3183--3190.Google Scholar
- Chen Jie, Tengfei Ma, and Xiao Cao. 2018. FastGCN: fast learning with graph convolutional networks via importance sampling. In ICLR 2018 .Google Scholar
- Alistair EW Johnson, Tom J Pollard, Lu Shen, H Lehman Li-wei, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. 2016. MIMIC-III, a freely accessible critical care database. Scientific data , Vol. 3 (2016), 160035.Google Scholar
- Haocheng Kao, Kaifu Tang, and Edward Y Chang. 2018. Context-aware symptom checking for disease diagnosis using hierarchical reinforcement learning. In AAAI 2018. 2305--2313.Google ScholarCross Ref
- Thomas N Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In ICLR 2017 .Google Scholar
- Sven Kosub. 2019. A note on the triangle inequality for the Jaccard distance. Pattern Recognition Letters , Vol. 120 (2019), 36--38.Google ScholarDigital Library
- C Krittanawong. 2018. The rise of artificial intelligence and the uncertain future for physicians. European J. Internal Medicine , Vol. 48 (2018), e13--e14.Google ScholarCross Ref
- Hung Le, Truyen Tran, and Svetha Venkatesh. 2018. Dual memory neural computer for asynchronous two-view sequential learning. In ACM SIGKDD 2018. 1637--1645.Google ScholarDigital Library
- Cheng Li, Bingyu Wang, Virgil Pavlu, and Javed A Aslam. 2016. Conditional bernoulli mixtures for multi-label classification. In ICML 2016 . 2482--2491.Google Scholar
- Tengfei Ma, Cao Xiao, Jiayu Zhou, and Fei Wang. 2018. Drug similarity integration through attentive multi-view graph auto-encoders. In IJCAI 2018 . 3477--3483.Google ScholarCross Ref
- Chengsheng Mao, Liang Yao, and Yuan Luo. 2019. MedGCN: graph convolutional networks for multiple medical tasks. CoRR , Vol. abs/1904.00326 (2019).Google Scholar
- James Mullenbach, Sarah Wiegreffe, Jon Duke, Jimeng Sun, and Jacob Eisenstein. 2018. Explainable prediction of medical codes from clinical text. In NAACL-HLT 2018 . 1101--1111.Google Scholar
- Shamim Nemati, Mohammad M Ghassemi, and Gari D Clifford. 2016. Optimal medication dosing from suboptimal clinical examples: a deep reinforcement learning approach. In EMBC 2016. 2978--2981.Google ScholarCross Ref
- John Ross Quinlan. 1986. Induction of decision trees. Machine Learning , Vol. 1, 1 (1986), 81--106.Google ScholarCross Ref
- Aniruddh Raghu, Matthieu Komorowski, and Sumeetpal Singh. 2018. Model-based reinforcement learning for sepsis treatment. CoRR , Vol. abs/1811.09602 (2018).Google Scholar
- Jesse Read, Bernhard Pfahringer, Geoff Holmes, and Eibe Frank. 2011. Classifier chains for multi-label classification. Machine Learning , Vol. 85, 3 (2011), 333--359.Google ScholarDigital Library
- Pengjie Ren, Zhumin Chen, Jing Li, Zhaochun Ren, Jun Ma, and Maarten de Rijke. 2019. RepeatNet: A Repeat Aware Neural Recommendation Machine for Session-Based Recommendation. In AAAI 2019. 4806--4813.Google ScholarDigital Library
- Pengjie Ren, Zhumin Chen, Zhaochun Ren, Furu Wei, Liqiang Nie, Jun Ma, and Maarten Rijke. 2018. Sentence Relations for Extractive Summarization with Deep Neural Networks. ACM Trans. Inf. Syst. , Vol. 36, 4 (2018), 39:1--39:32.Google ScholarDigital Library
- Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. 2018. Modeling relational data with graph convolutional networks. In ESWC 2018 . 593--607.Google ScholarCross Ref
- Junyuan Shang, Shenda Hong, Yuxi Zhou, Meng Wu, and Hongyan Li. 2018. Knowledge guided multi-instance multi-label learning via neural networks in medicines prediction. In ACML 2018. 831--846.Google Scholar
- Junyuan Shang, Cao Xiao, Tengfei Ma, Hongyan Li, and Jimeng Sun. 2019. GAMENet: graph augmented memory networks for recommending medication combination. In AAAI 2019 .Google ScholarCross Ref
- Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. J. Machine Learning Research , Vol. 15, 1 (2014), 1929--1958.Google ScholarDigital Library
- Kai-Fu Tang, Hao-Cheng Kao, Chun-Nan Chou, and Edward Y Chang. 2016. Inquire and diagnose: neural symptom checking ensemble using deep reinforcement learning. In NIPS 2016 .Google Scholar
- Nicholas P Tatonetti, Patrick Ye, Roxana Daneshjou, and Russ B Altman. 2012. Data-driven prediction of drug effects and interactions. Science Translational Medicine , Vol. 4, 125 (2012), 125ra31--125ra31.Google ScholarCross Ref
- Michel Tokic. 2010. Adaptive $varepsilon$-greedy exploration in reinforcement learning based on value differences. In GCAAI 2010 .Google Scholar
- Grigorios Tsoumakas and Ioannis Katakis. 2007. Multi-label classification: an overview. International J. Data Warehousing and Mining , Vol. 3, 3 (2007), 1--13.Google ScholarCross Ref
- Lu Wang, Wei Zhang, Xiaofeng He, and Hongyuan Zha. 2018. Personalized prescription for comorbidity. In DASFAA 2018. 3--19.Google ScholarDigital Library
- Meng Wang, Mengyue Liu, Jun Liu, Sen Wang, Guodong Long, and Buyue Qian. 2017. Safe medicine recommendation via medical knowledge graph embedding. CoRR , Vol. abs/1710.05980 (2017).Google Scholar
- T G Wolfsberg, P. Primakoff, D G Myles, and J M White. 1995. ADAM, a novel family of membrane proteins containing a disintegrin and metalloprotease domain: multipotential functions in cell-cell and cell-matrix interactions. J. Cell Biology , Vol. 131, 2 (1995), 275--8.Google ScholarCross Ref
- Keyang Xu, Mike Lam, Jingzhi Pang, Xin Gao, Charlotte Band, Pengtao Xie, and Eric Xing. 2018. Multimodal machine learning for automated ICD coding. CoRR , Vol. abs/1810.13348 (2018).Google Scholar
- Pengcheng Yang, Xu Sun, Wei Li, Shuming Ma, Wei Wu, and Houfeng Wang. 2018b. SGM: sequence generation model for multi-label classification. In COLING 2018 . 3915--3926.Google Scholar
- Zhongliang Yang, Yongfeng Huang, Yiran Jiang, Yuxi Sun, Yu-Jin Zhang, and Pengcheng Luo. 2018a. Clinical assistant diagnosis for electronic medical record based on convolutional neural network. Scientific reports , Vol. 8, 1 (2018), 6329.Google Scholar
- Muhan Zhang, Zhicheng Cui, Marion Neumann, and Yixin Chen. 2018. An end-to-end deep learning architecture for graph classification. In AAAI 2018 . 4438--4445.Google Scholar
- Yutao Zhang, Robert Chen, Tang Jie, Walter F. Stewart, and Jimeng Sun. 2017. LEAP: learning to prescribe effective and safe treatment combinations for multimorbidity. In ACM SIGKDD 2017. 1315--1324.Google ScholarDigital Library
- Hanzhong Zheng and Dejia Shi. 2018. Using a LSTM-RNN based deep learning framework for ICU mortality prediction. In WISA 2018 . 60--67.Google ScholarDigital Library
- Marinka Zitnik, Monica Agrawal, and Jure Leskovec. 2018. Modeling polypharmacy side effects with graph convolutional networks. Bioinformatics , Vol. 34, 13 (2018), i457--i466.Google ScholarCross Ref
Index Terms
- Order-free Medicine Combination Prediction with Graph Convolutional Reinforcement Learning
Recommendations
An intelligent intervention strategy for patients to prevent chronic complications based on reinforcement learning
AbstractChronic complications refer to defects that initially have a limited impact on chronic patients’ normal lives, but that may accumulate into a major danger if they are not treated in a timely manner. A natural problem is how to take ...
A reinforcement learning approach for individualizing erythropoietin dosages in hemodialysis patients
This paper presents a reinforcement learning (RL) approach for anemia management in patients undergoing chronic renal failure. Erythropoietin (EPO) is the treatment of choice for this kind of anemia but it is an expensive drug and with some dangerous ...
Disentangled Dynamic Heterogeneous Graph Learning for Opioid Overdose Prediction
KDD '22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data MiningOpioids (e.g., oxycodone and morphine) are highly addictive prescription (aka Rx) drugs which can be easily overprescribed and lead to opioid overdose. Recently, the opioid epidemic is increasingly serious across the US as its related deaths have risen ...
Comments