Skip to main content

Advertisement

Springer Nature Link
Log in

MIFNet: multimodal interactive fusion network for medication recommendation

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Medication recommendation aims to provide clinicians with safe medicine combinations for the treatment of patients. Existing medication recommendation models are built based on the temporal structured code data of electronic health records (EHRs) in the medical database; nevertheless, unstructured data in EHRs, such as textual data containing rich information, are underexploited. To fill the gap, a novel multimodal interactive fusion network (MIFNet) is proposed for medication recommendation, which integrates both structured code information and unstructured text information in EHRs. Our model first extracts a series of informative feature representations to encode comprehensive patient health history and control potential drug–drug interactions (DDI), including medical code, clinical notes, and the DDI knowledge graph. Next, a novel cross-modal interaction extraction block is proposed to capture the intricate interaction information between the two modalities. Finally, a multimodal fusion block is adopted to fuse the constructed features and generate a medication combination list. Experiments are conducted on the public MIMIC-III dataset, and the results demonstrate that the proposed model outperforms the state-of-the-art medication recommendation methods on main evaluation metrics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Algorithm 1
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

The dataset that supports the study is available from https://physionet.org/content/mimiciii/1.4/.

References

  • Al-Taie Z, Hannink M, Mitchem J, Papageorgiou C, Shyu C-R (2021a) Drug repositioning and subgroup discovery for precision medicine implementation in triple negative breast cancer. Cancers 13(24):6278. https://doi.org/10.3390/cancers13246278

    Article  Google Scholar 

  • Al-Taie Z, Liu D, Mitchem JB, Papageorgiou C, Kaifi JT, Warren WC, Shyu C-R (2021b) Explainable artificial intelligence in high-throughput drug repositioning for subgroup stratifications with interventionable potential. J Biomed Inf 118:103792. https://doi.org/10.1016/j.jbi.2021.103792

    Article  Google Scholar 

  • Almirall D, Compton SN, Gunlicks-Stoessel M, Duan N, Murphy SA (2012) Designing a pilot sequential multiple assignment randomized trial for developing an adaptive treatment strategy. Stat Med 31(17):1887–1902. https://doi.org/10.1002/sim.4512

    Article  MathSciNet  Google Scholar 

  • Alsentzer E, Murphy JR, Boag W, Weng W-H, Jin D, Naumann T, McDermott M (2019) Publicly available clinical BERT embeddings. In: Proceedings of the 2nd Clinical Natural Language Processing Workshop, https://doi.org/10.18653/v1/W19-1909

  • An Y, Zhang H, Sheng Y, Wang J, Chen X (2021) MAIN: multimodal attention-based fusion networks for diagnosis prediction. In: 2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), https://doi.org/10.1109/BIBM52615.2021.9669634

  • An Y, Zhang L, You M, Tian X, Jin B, Wei X (2021b) Mesin: multilevel selective and interactive network for medication recommendation. Knowl-Based Syst 233:107534. https://doi.org/10.1016/j.knosys.2021.107534

    Article  Google Scholar 

  • Bajor JM, Lasko TA (2016) Predicting medications from diagnostic codes with recurrent neural networks. In: International Conference on Learning Representations

  • Baumel T, Nassour-Kassis J, Cohen R, Elhadad M, Elhadad N (2018) Multi-label classification of patient notes: case study on ICD code assignment. In: Workshops at the Thirty-Second AAAI Conference on Artificial Intelligence

  • Bhoi S, Lee ML, Hsu W, Fang HSA, Tan NC (2021) Personalizing medication recommendation with a graph-based approach. ACM Transact Inf Syst (TOIS) 40(3):1–23. https://doi.org/10.1145/3488668

    Article  Google Scholar 

  • Choi E, Bahadori MT, Sun J, Kulas J, Schuetz A, Stewart W (2016) Retain: an interpretable predictive model for healthcare using reverse time attention mechanism. Adv Neural Inf Process Syst 29

  • Chorowski JK, Bahdanau D, Serdyuk D, Cho K, Bengio Y (2015) Attention-based models for speech recognition. Adv Neural Inf Process Syst 28

  • Gong F, Wang M, Wang H, Wang S, Liu M (2021) SMR: medical knowledge graph embedding for safe medicine recommendation. Big Data Res 23:100174. https://doi.org/10.1016/j.bdr.2020.100174

    Article  Google Scholar 

  • Huang K, Altosaar J, Ranganath R (2019) Clinicalbert: modeling clinical notes and predicting hospital readmission. https://arxiv.org/abs/1904.05342.

  • Jiang Y, Sha L, Rahmaniheris M, Wan B, Hosseini M, Tan P, Berlin RB Jr (2016) Sepsis patient detection and monitor based on auto-BN. J Med Syst 40(4):111. https://doi.org/10.1007/s10916-016-0444-2

    Article  Google Scholar 

  • Johnson AE, Pollard TJ, Shen L, Lehman LW, Feng M, Ghassemi M, Moody B, Szolovits P, Celi LA, Mark RG (2016) MIMIC-III, a freely accessible critical care database. Sci Data 3:160035. https://doi.org/10.1038/sdata.2016.35

    Article  Google Scholar 

  • Kim Y (2014) Convolutional neural networks for sentence classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) Doha, Qatar. https://doi.org/10.3115/v1/D14-1181

  • Lakkaraju H, Rudin C (2017) Learning cost-effective and interpretable treatment regimes. In: Artificial Intelligence and Statistics

  • Le H, Tran T, Venkatesh S (2018) Dual memory neural computer for asynchronous two-view sequential learning. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, https://doi.org/10.1145/3219819.3219981

  • Li R, Ma F, Gao J (2021) Integrating multimodal electronic health records for diagnosis prediction. In: AMIA Annual Symposium Proceedings

  • Li X, Zhang Y, Li X, Wei H, Lu M (2023) DGCL: distance-wise and graph contrastive learning for medication recommendation. J Biomed Inform 139:104301. https://doi.org/10.1016/j.jbi.2023.104301

    Article  Google Scholar 

  • Lian J, Zhou X, Zhang F, Chen Z, Xie X, Sun G (2018) xdeepfm: combining explicit and implicit feature interactions for recommender systems. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, https://doi.org/10.1145/3219819.3220023

  • Lin S, Wang M, Shi C, Xu Z, Chen L, Gao Q, Chen J (2022) MR-KPA: medication recommendation by combining knowledge-enhanced pre-training with a deep adversarial network. BMC Bioinformatics 23(1):1–19. https://doi.org/10.1186/s12859-022-05102-1

    Article  Google Scholar 

  • Liu J, Zhang Z, Razavian N (2018) Deep ehr: chronic disease prediction using medical notes. In: Machine Learning for Healthcare Conference

  • Lopez-Ubeda P, Diaz-Galiano MC, Martin-Noguerol T, Luna A, Urena-Lopez LA, Martin-Valdivia MT (2021) Automatic medical protocol classification using machine learning approaches. Comput Methods Programs Biomed 200:105939. https://doi.org/10.1016/j.cmpb.2021.105939

    Article  Google Scholar 

  • Luaces O, Díez J, Barranquero J, del Coz JJ, Bahamonde A (2012) Binary relevance efficacy for multilabel classification. Progr Artif Intell 1(4):303–313. https://doi.org/10.1007/s13748-012-0030-x

    Article  Google Scholar 

  • Lyu W, Dong X, Wong R, Zheng S, Abell-Hart K, Wang F, Chen C (2022) A multimodal transformer: fusing clinical notes with structured EHR data for interpretable in-hospital mortality prediction. In: AMIA ... Annual Symposium proceedings. AMIA Symposium, 2022, pp 719–728. Retrieved 2022, from https://europepmc.org/articles/PMC10148371

  • Ma L, Zhang Y (2015) Using Word2Vec to process big text data. In: 2015 IEEE International Conference on Big Data (Big Data), https://doi.org/10.1109/BigData.2015.7364114

  • Meng Y, Lu C, Jin M, Xu J, Zeng X, Yang J (2022) A weighted bilinear neural collaborative filtering approach for drug repositioning. Brief Bioinf 23(2):581. https://doi.org/10.1093/bib/bbab581

    Article  Google Scholar 

  • Meng Y, Speier W, Ong MK, Arnold CW (2021) Bidirectional representation learning from transformers using multimodal electronic health record data to predict depression. IEEE J Biomed Health Inform 25(8):3121–3129. https://doi.org/10.1109/JBHI.2021.3063721

    Article  Google Scholar 

  • Nemati S, Holder A, Razmi F, Stanley MD, Clifford GD, Buchman TG (2018) An interpretable machine learning model for accurate prediction of sepsis in the ICU. Crit Care Med 46(4):547–553. https://doi.org/10.1097/CCM.0000000000002936

    Article  Google Scholar 

  • Nguyen H, Patrick J (2016) Text mining in clinical domain: dealing with noise. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, https://doi.org/10.1145/2939672.2939720

  • Pushpakom S, Iorio F, Eyers PA, Escott KJ, Hopper S, Wells A, Doig A, Guilliams T, Latimer J, McNamee C, Norris A, Sanseau P, Cavalla D, Pirmohamed M (2019) Drug repurposing: progress, challenges and recommendations. Nat Rev Drug Discov 18(1):41–58. https://doi.org/10.1038/nrd.2018.168

    Article  Google Scholar 

  • Qiao Z, Wu X, Ge S, Fan W (2019) Mnn: multimodal attentional neural networks for diagnosis prediction. In: International Joint Conference on Artificial Intelligence, https://doi.org/10.24963/ijcai.2019/823

  • Rahman W, Hasan MK, Lee S, Zadeh A, Mao C, Morency L-P, Hoque E (2020) Integrating multimodal information in large pretrained transformers. In: Proceedings of the conference. Association for Computational Linguistics. Meeting, https://doi.org/10.18653/v1/2020.acl-main.214

  • Ramachandram D, Taylor GW (2017) Deep multimodal learning: a survey on recent advances and trends. IEEE Signal Process Mag 34(6):96–108. https://doi.org/10.1109/MSP.2017.2738401

    Article  Google Scholar 

  • Read J, Pfahringer B, Holmes G, Frank E (2011) Classifier chains for multi-label classification. Mach Learn 85(3):333–359. https://doi.org/10.1007/s10994-011-5256-5

    Article  MathSciNet  Google Scholar 

  • Shang J, Xiao C, Ma T, Li H, Sun J (2019) Gamenet: graph augmented memory networks for recommending medication combination. In: Proceedings of the AAAI Conference on Artificial Intelligence, https://doi.org/10.1609/aaai.v33i01.33011126

  • Song J, Wang Y, Tang S, Zhang Y, Chen Z, Zhang Z, Zhang T, Wu F (2020) Local-global memory neural network for medication prediction. IEEE Transact Neural Netw Learn Syst 32(4):1723–1736. https://doi.org/10.1109/TNNLS.2020.2989364

    Article  Google Scholar 

  • Srivastava N, Salakhutdinov RR (2012) Multimodal learning with deep boltzmann machines. Adv Neural Inf Process Syst, 25

  • Su Y, Shi Y, Lee W, Cheng L, Guo H (2022) TAHDNet: time-aware hierarchical dependency network for medication recommendation. J Biomed Inform 129:104069. https://doi.org/10.1016/j.jbi.2022.104069

    Article  Google Scholar 

  • Sundararajan M, Taly A, Yan Q (2017) Axiomatic attribution for deep networks. In: International conference on machine learning

  • Tatonetti NP, Ye PP, Daneshjou R, Altman RB (2012) Data-driven prediction of drug effects and interactions. Sci Transl Med. https://doi.org/10.1126/scitranslmed.3003377

    Article  Google Scholar 

  • Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. Adv Neural Inf Process Syst 30

  • Wang H, Wu Y, Gao C, Deng Y, Zhang F, Huang J, Liu J (2021a) Medication combination prediction using temporal attention mechanism and simple graph convolution. IEEE J Biomed Health Inform 25(10):3995–4004. https://doi.org/10.1109/JBHI.2021.3082548

    Article  Google Scholar 

  • Wang L, Zhang W, He X, Zha H (2018) Personalized prescription for comorbidity. In: Database Systems for Advanced Applications (pp. 3–19). https://doi.org/10.1007/978-3-319-91458-9_1

  • Wang S, Ren P, Chen Z, Ren Z, Ma J, de Rijke M (2019) Order-free medicine combination prediction with graph convolutional reinforcement learning. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management, https://doi.org/10.1145/3357384.3357965

  • Wang Y, Chen W, Pi D, Yue L, Wang S, Xu M (2021) Self-supervised adversarial distribution regularization for medication recommendation. https://doi.org/10.24963/ijcai.2021/431

  • Wu R, Qiu Z, Jiang J, Qi G, Wu X (2022) Conditional generation net for medication recommendation. In: Proceedings of the ACM Web Conference 2022, https://doi.org/10.1145/3485447.3511936

  • Xu Z, So DR, Dai AM (2021) Mufasa: Multimodal fusion architecture search for electronic health records. In: Proceedings of the AAAI Conference on Artificial Intelligence, https://doi.org/10.1609/aaai.v35i12.17260

  • Yang B, Wu L (2021) How to leverage multimodal EHR data for better medical predictions? In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, https://doi.org/10.18653/v1/2021.emnlp-main.329

  • Yang C, Xiao C, Ma F, Glass L, Sun J (2021) Safedrug: dual molecular graph encoders for safe drug recommendations. In: Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, https://doi.org/10.24963/ijcai.2021/514

  • Zadeh A, Chen M, Poria S, Cambria E, Morency L-P (2017) Tensor fusion network for multimodal sentiment analysis.In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, https://doi.org/10.18653/v1/D17-1115

  • Zhang S, Tong H, Xu J, Maciejewski R (2019) Graph convolutional networks: a comprehensive review. Comput Soc Netw 6(1):1–23. https://doi.org/10.1186/s40649-019-0069-y

    Article  Google Scholar 

  • Zhang Y, Chen R, Tang J, Stewart WF, Sun J (2017) LEAP: learning to prescribe effective and safe treatment combinations for multimorbidity. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, https://doi.org/10.1145/3097983.3098109

  • Zhao B-W, Su X-R, Hu P-W, Ma Y-P, Zhou X, Hu L (2022) A geometric deep learning framework for drug repositioning over heterogeneous information networks. Briefings Bioinf 23(6):bbac384. https://doi.org/10.1093/bib/bbac384

    Article  Google Scholar 

Download references

Acknowledgements

The authors thank the editors and the anonymous reviewers for their insightful comments and constructive suggestions throughout the review process.

Funding

The work has been supported by National Natural Science Foundation of China (No. 72171176, NO. 71771179, NO. 72021002) and China-Germany Cooperation project supported by National Natural Science Foundation of China (M-0310).

Author information

Authors and Affiliations

  1. School of Economics and Management, Tongji University, Shanghai, China

    Jiazhen Huo, Zhikai Hong, Mingzhou Chen & Yongrui Duan

Authors
  1. Jiazhen Huo
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Zhikai Hong
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Mingzhou Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Yongrui Duan
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

ZH and YD conceived and designed this study. ZH and JH performed the experiments. ZH, MC, and YD assisted with research and writing. All authors assisted in drafting and revising the manuscript at all stages, and all have approved it in this final form.

Corresponding author

Correspondence to Yongrui Duan.

Ethics declarations

Conflict of interest

The authors have no conflicts of interest to declare.

Ethical approval

Not applicable.

Consent to participation

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huo, J., Hong, Z., Chen, M. et al. MIFNet: multimodal interactive fusion network for medication recommendation. J Supercomput 80, 12313–12345 (2024). https://doi.org/10.1007/s11227-024-05908-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-024-05908-1

Keywords