skip to main content
10.1145/3357384.3357965acmconferencesArticle/Chapter ViewAbstractPublication PagescikmConference Proceedingsconference-collections
research-article

Order-free Medicine Combination Prediction with Graph Convolutional Reinforcement Learning

Published: 03 November 2019 Publication History

Abstract

Medicine Combination Prediction (MCP) based on Electronic Health Record (EHR) can assist doctors to prescribe medicines for complex patients. Previous studies on MCP either ignore the correlations between medicines (i.e., MCP is formulated as a binary classifcation task), or assume that there is a sequential correlation between medicines (i.e., MCP is formulated as a sequence prediction task). The latter is unreasonable because the correlations between medicines should be considered in an order-free way. Importantly, MCP must take additional medical knowledge (e.g., Drug-Drug Interaction (DDI)) into consideration to ensure the safety of medicine combinations. However, most previous methods for MCP incorporate DDI knowledge with a post-processing scheme, which might undermine the integrity of proposed medicine combinations. In this paper, we propose a graph convolutional reinforcement learning model for MCP, named Combined Order-free Medicine Prediction Network (CompNet), that addresses the issues listed above. CompNet casts the MCP task as an order-free Markov Decision Process (MDP) problem and designs a Deep Q Learning (DQL) mechanism to learn correlative and adverse interactions between medicines. Specifcally, we frst use a Dual Convolutional Neural Network (Dual-CNN) to obtain patient representations based on EHRs. Then, we introduce the medicine knowledge associated with predicted medicines to create a dynamic medicine knowledge graph, and use a Relational Graph Convolutional Network (R-GCN) to encode it. Finally, CompNet selects medicines by fusing the combination of patient information and the medicine knowledge graph. Experiments on a benchmark dataset, i.e., MIMIC-III, demonstrate that CompNet signifcantly outperforms state-of-the-art methods and improves a recently proposed model by 3.74%pt, 6.64%pt in terms of Jaccard and F1 metrics.

References

[1]
Oron Anschel, Nir Baram, and Nahum Shimkin. 2017. Averaged-DQN: variance reduction and stabilization for deep reinforcement learning. In ICML 2017 . 176--185.
[2]
Jacek M. Bajor and Thomas A. Lasko. 2017. Predicting medications from diagnostic codes with recurrent neural networks. In ICLR 2017 .
[3]
Remi Besson, Erwan Le Pennec, Stephanie Allassonniere, J Stirnemann, Emmanuel Spaggiari, and Antoine Neuraz. 2018. A model-based reinforcement learning approach for a rare disease diagnostic task. CoRR, Vol. abs/1811.10112 (2018).
[4]
Nikhil Cheerla and Olivier Gevaert. 2017. MicroRNA based Pan-Cancer diagnosis and treatment recommendation. BMC bioinformatics, Vol. 18, 1 (2017), 32.
[5]
Feixiong Cheng, István A Kovács, and Albert-László Barabási. 2019. Network-based prediction of drug combinations. Nature communications, Vol. 10, 1 (2019), 1197.
[6]
Edward Choi, Mohammad Taha Bahadori, Jimeng Sun, Joshua Kulas, Andy Schuetz, and Walter Stewart. 2016. Retain: an interpretable predictive model for healthcare using reverse time attention mechanism. In NIPS 2016. 3504--3512.
[7]
Leslie Citrome. 2009. Quantifying risk: the role of absolute and relative measures in interpreting risk of adverse reactions from product labels of antipsychotic medications. Current drug safety, Vol. 4, 463 (2009), 229--237.
[8]
Victor J Dzau and Celynne A Balatbat. 2018. Health and societal implications of medical and technological advances. Science translational medicine, Vol. 10, 463 (2018), 463.
[9]
David J Eveson, Thompson G Robinson, and John F Potter. 2007. Lisinopril for the treatment of hypertension within the first 24 hours of acute ischemic stroke and follow-up. American J. Hypertension, Vol. 20, 3 (2007), 270--277.
[10]
Gregg C Fonarow, R Scott Wright, Frederick A Spencer, Paul D Fredrick, Wei Dong, Nathan Every, William J French, National Registry of Myocardial Infarction 4 Investigators, et almbox. 2005. Effect of statin use within the first 24 hours of admission for acute myocardial infarction on early morbidity and mortality. American J. Cardiology, Vol. 96, 5 (2005), 611--616.
[11]
Matthew W. Gardner and Stephen R. Dorling. 1998. Artificial neural networks (the multilayer perceptron): a review of applications in the atmospheric sciences. Atmospheric Environment, Vol. 32, 14--15 (1998), 2627--2636.
[12]
Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In AISTATS 2010 . 249--256.
[13]
David N Hager, Varshitha Tanykonda, Zeba Noorain, Sarina K Sahetya, Catherine E Simpson, Juan Felipe Lucena, and Dale M Needham. 2018. Hospital mortality prediction for intermediate care patients: assessing the generalizability of the Intermediate Care Unit Severity Score (IMCUSS). J. Critical Care, Vol. 46 (2018), 94--98.
[14]
Haibo He and Edwardo A. Garcia. 2009. Learning from imbalanced data. IEEE Transactions on Knowledge and Data Engineering, Vol. 21, 9 (2009), 1263--1284.
[15]
Shiyi He, Chang Xu, Tianyu Guo, Chao Xu, and Dacheng Tao. 2018. Reinforced multi-label image classification by exploring curriculum. In AAAI 2018 . 3183--3190.
[16]
Chen Jie, Tengfei Ma, and Xiao Cao. 2018. FastGCN: fast learning with graph convolutional networks via importance sampling. In ICLR 2018 .
[17]
Alistair EW Johnson, Tom J Pollard, Lu Shen, H Lehman Li-wei, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. 2016. MIMIC-III, a freely accessible critical care database. Scientific data, Vol. 3 (2016), 160035.
[18]
Haocheng Kao, Kaifu Tang, and Edward Y Chang. 2018. Context-aware symptom checking for disease diagnosis using hierarchical reinforcement learning. In AAAI 2018. 2305--2313.
[19]
Thomas N Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In ICLR 2017 .
[20]
Sven Kosub. 2019. A note on the triangle inequality for the Jaccard distance. Pattern Recognition Letters, Vol. 120 (2019), 36--38.
[21]
C Krittanawong. 2018. The rise of artificial intelligence and the uncertain future for physicians. European J. Internal Medicine, Vol. 48 (2018), e13--e14.
[22]
Hung Le, Truyen Tran, and Svetha Venkatesh. 2018. Dual memory neural computer for asynchronous two-view sequential learning. In ACM SIGKDD 2018. 1637--1645.
[23]
Cheng Li, Bingyu Wang, Virgil Pavlu, and Javed A Aslam. 2016. Conditional bernoulli mixtures for multi-label classification. In ICML 2016 . 2482--2491.
[24]
Tengfei Ma, Cao Xiao, Jiayu Zhou, and Fei Wang. 2018. Drug similarity integration through attentive multi-view graph auto-encoders. In IJCAI 2018 . 3477--3483.
[25]
Chengsheng Mao, Liang Yao, and Yuan Luo. 2019. MedGCN: graph convolutional networks for multiple medical tasks. CoRR, Vol. abs/1904.00326 (2019).
[26]
James Mullenbach, Sarah Wiegreffe, Jon Duke, Jimeng Sun, and Jacob Eisenstein. 2018. Explainable prediction of medical codes from clinical text. In NAACL-HLT 2018 . 1101--1111.
[27]
Shamim Nemati, Mohammad M Ghassemi, and Gari D Clifford. 2016. Optimal medication dosing from suboptimal clinical examples: a deep reinforcement learning approach. In EMBC 2016. 2978--2981.
[28]
John Ross Quinlan. 1986. Induction of decision trees. Machine Learning, Vol. 1, 1 (1986), 81--106.
[29]
Aniruddh Raghu, Matthieu Komorowski, and Sumeetpal Singh. 2018. Model-based reinforcement learning for sepsis treatment. CoRR, Vol. abs/1811.09602 (2018).
[30]
Jesse Read, Bernhard Pfahringer, Geoff Holmes, and Eibe Frank. 2011. Classifier chains for multi-label classification. Machine Learning, Vol. 85, 3 (2011), 333--359.
[31]
Pengjie Ren, Zhumin Chen, Jing Li, Zhaochun Ren, Jun Ma, and Maarten de Rijke. 2019. RepeatNet: A Repeat Aware Neural Recommendation Machine for Session-Based Recommendation. In AAAI 2019. 4806--4813.
[32]
Pengjie Ren, Zhumin Chen, Zhaochun Ren, Furu Wei, Liqiang Nie, Jun Ma, and Maarten Rijke. 2018. Sentence Relations for Extractive Summarization with Deep Neural Networks. ACM Trans. Inf. Syst., Vol. 36, 4 (2018), 39:1--39:32.
[33]
Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. 2018. Modeling relational data with graph convolutional networks. In ESWC 2018 . 593--607.
[34]
Junyuan Shang, Shenda Hong, Yuxi Zhou, Meng Wu, and Hongyan Li. 2018. Knowledge guided multi-instance multi-label learning via neural networks in medicines prediction. In ACML 2018. 831--846.
[35]
Junyuan Shang, Cao Xiao, Tengfei Ma, Hongyan Li, and Jimeng Sun. 2019. GAMENet: graph augmented memory networks for recommending medication combination. In AAAI 2019 .
[36]
Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. J. Machine Learning Research, Vol. 15, 1 (2014), 1929--1958.
[37]
Kai-Fu Tang, Hao-Cheng Kao, Chun-Nan Chou, and Edward Y Chang. 2016. Inquire and diagnose: neural symptom checking ensemble using deep reinforcement learning. In NIPS 2016 .
[38]
Nicholas P Tatonetti, Patrick Ye, Roxana Daneshjou, and Russ B Altman. 2012. Data-driven prediction of drug effects and interactions. Science Translational Medicine, Vol. 4, 125 (2012), 125ra31--125ra31.
[39]
Michel Tokic. 2010. Adaptive $varepsilon$-greedy exploration in reinforcement learning based on value differences. In GCAAI 2010 .
[40]
Grigorios Tsoumakas and Ioannis Katakis. 2007. Multi-label classification: an overview. International J. Data Warehousing and Mining, Vol. 3, 3 (2007), 1--13.
[41]
Lu Wang, Wei Zhang, Xiaofeng He, and Hongyuan Zha. 2018. Personalized prescription for comorbidity. In DASFAA 2018. 3--19.
[42]
Meng Wang, Mengyue Liu, Jun Liu, Sen Wang, Guodong Long, and Buyue Qian. 2017. Safe medicine recommendation via medical knowledge graph embedding. CoRR, Vol. abs/1710.05980 (2017).
[43]
T G Wolfsberg, P. Primakoff, D G Myles, and J M White. 1995. ADAM, a novel family of membrane proteins containing a disintegrin and metalloprotease domain: multipotential functions in cell-cell and cell-matrix interactions. J. Cell Biology, Vol. 131, 2 (1995), 275--8.
[44]
Keyang Xu, Mike Lam, Jingzhi Pang, Xin Gao, Charlotte Band, Pengtao Xie, and Eric Xing. 2018. Multimodal machine learning for automated ICD coding. CoRR, Vol. abs/1810.13348 (2018).
[45]
Pengcheng Yang, Xu Sun, Wei Li, Shuming Ma, Wei Wu, and Houfeng Wang. 2018b. SGM: sequence generation model for multi-label classification. In COLING 2018 . 3915--3926.
[46]
Zhongliang Yang, Yongfeng Huang, Yiran Jiang, Yuxi Sun, Yu-Jin Zhang, and Pengcheng Luo. 2018a. Clinical assistant diagnosis for electronic medical record based on convolutional neural network. Scientific reports, Vol. 8, 1 (2018), 6329.
[47]
Muhan Zhang, Zhicheng Cui, Marion Neumann, and Yixin Chen. 2018. An end-to-end deep learning architecture for graph classification. In AAAI 2018 . 4438--4445.
[48]
Yutao Zhang, Robert Chen, Tang Jie, Walter F. Stewart, and Jimeng Sun. 2017. LEAP: learning to prescribe effective and safe treatment combinations for multimorbidity. In ACM SIGKDD 2017. 1315--1324.
[49]
Hanzhong Zheng and Dejia Shi. 2018. Using a LSTM-RNN based deep learning framework for ICU mortality prediction. In WISA 2018 . 60--67.
[50]
Marinka Zitnik, Monica Agrawal, and Jure Leskovec. 2018. Modeling polypharmacy side effects with graph convolutional networks. Bioinformatics, Vol. 34, 13 (2018), i457--i466.

Cited By

View all

Index Terms

  1. Order-free Medicine Combination Prediction with Graph Convolutional Reinforcement Learning

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management
    November 2019
    3373 pages
    ISBN:9781450369763
    DOI:10.1145/3357384
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 03 November 2019

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. medicine combination prediction
    2. medicine knowledge graph
    3. reinforcement learning
    4. relational graph convolutional network

    Qualifiers

    • Research-article

    Conference

    CIKM '19
    Sponsor:

    Acceptance Rates

    CIKM '19 Paper Acceptance Rate 202 of 1,031 submissions, 20%;
    Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

    Upcoming Conference

    CIKM '25

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)98
    • Downloads (Last 6 weeks)13
    Reflects downloads up to 28 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)AMGNet: An Attention-Guided Multi-Graph Collaborative Decision Network for Safe Medication RecommendationElectronics10.3390/electronics1404076014:4(760)Online publication date: 15-Feb-2025
    • (2025)ExpDrugNeurocomputing10.1016/j.neucom.2024.129021619:COnline publication date: 28-Feb-2025
    • (2024)A Survey of Personalized Medicine RecommendationInternational Journal of Crowd Science10.26599/IJCS.2023.91000138:2(77-82)Online publication date: May-2024
    • (2024)FastRx: Exploring Fastformer and Memory-Augmented Graph Neural Networks for Personalized Medication RecommendationsACM Transactions on Intelligent Systems and Technology10.1145/369611115:6(1-21)Online publication date: 17-Sep-2024
    • (2024)CausalMed: Causality-Based Personalized Medication Recommendation Centered on Patient Health StateProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679542(1276-1285)Online publication date: 21-Oct-2024
    • (2024)Natural Language-Assisted Multi-modal Medication RecommendationProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679529(2200-2209)Online publication date: 21-Oct-2024
    • (2024)Leave No Patient Behind: Enhancing Medication Recommendation for Rare Disease PatientsProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657785(533-542)Online publication date: 10-Jul-2024
    • (2024)Medical-Knowledge-Based Graph Neural Network for Medication Combination PredictionIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.326649035:10(13246-13257)Online publication date: Oct-2024
    • (2024)Enhancing Precision Drug Recommendations via In-Depth Exploration of Motif RelationshipsIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.343777536:12(8164-8178)Online publication date: Dec-2024
    • (2024)DuKA: A Dual-Keyless-Attention Model for Multi-Modality EHR Data Fusion and Organ Failure PredictionIEEE Transactions on Biomedical Engineering10.1109/TBME.2023.333130571:4(1247-1256)Online publication date: Apr-2024
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media