research-article

HiTANet: Hierarchical Time-Aware Attention Networks for Risk Prediction on Electronic Health Records

Authors:

Fenglong MaAuthors Info & Claims

KDD '20: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

Pages 647 - 656

https://doi.org/10.1145/3394486.3403107

Published: 20 August 2020 Publication History

Abstract

Deep learning methods especially recurrent neural network based models have demonstrated early success in disease risk prediction on longitudinal patient data. Existing works follow a strong assumption to implicitly assume the stationary disease progression during each time period, and thus, take a homogeneous way to decay the information from previous time steps for all patients. However,in reality, disease progression is non-stationary. Besides, the key time steps for a target disease vary among patients. To leverage time information for risk prediction in a more reasonable way, we propose a new hierarchical time-aware attention network, named HiTANet, which imitates the decision making process of doctors inrisk prediction. Particularly, HiTANet models time information in local and global stages. The local evaluation stage has a time aware Transformer that embeds time information into visit-level embed-ding and generates local attention weight for each visit. The global synthesis stage further adopts a time-aware key-query attention mechanism to assign global weights to different time steps. Finally, the two types of attention weights are dynamically combined to generate the patient representations for further risk prediction. We evaluate HiTANet on three real-world datasets. Compared with the best results among twelve competing baselines, HiTANet achieves over 7% in terms of F1 score on all datasets, which demonstrates the effectiveness of the proposed model and the necessity of modeling time information in risk prediction task.

References

[1]

J Malcolm O Arnold, Salim Yusuf, James Young, James Mathew, David Johnstone, Alvaro Avezum, Eva Lonn, Janice Pogue, and Jackie Bosch. 2003. Prevention of heart failure in patients in the Heart Outcomes Prevention Evaluation (HOPE) study. Circulation, Vol. 107, 9 (2003), 1284--1290.

[2]

Tian Bai, Shanshan Zhang, Brian L Egleston, and Slobodan Vucetic. 2018. Interpretable representation learning for healthcare via capturing disease progression through time. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 43--51.

Digital Library

[3]

Inci M Baytas, Cao Xiao, Xi Zhang, Fei Wang, Anil K Jain, and Jiayu Zhou. 2017. Patient subtyping via time-aware LSTM networks. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining. ACM, 65--74.

Digital Library

[4]

Luca Bertinetto, Joao F Henriques, Philip HS Torr, and Andrea Vedaldi. 2019. Meta-learning with differentiable closed-form solvers. In The International Conference on Learning Representations (ICLR).

[5]

T Douglas Bradley and John S Floras. 2003 a. Sleep apnea and heart failure: Part I: obstructive sleep apnea. Circulation, Vol. 107, 12 (2003), 1671--1678.

[6]

T Douglas Bradley and John S Floras. 2003 b. Sleep apnea and heart failure: Part II: central sleep apnea. Circulation, Vol. 107, 13 (2003), 1822--1826.

[7]

Prithwish Chakraborty and Faisal Farooq. 2019. A Robust Framework for Accelerated Outcome-driven Risk Factor Identification from EHR. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1800--1808.

Digital Library

[8]

Zhengping Che, David Kale, Wenzhe Li, Mohammad Taha Bahadori, and Yan Liu. 2015. Deep computational phenotyping. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 507--516.

Digital Library

[9]

Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David Sontag, and Yan Liu. 2018. Recurrent neural networks for multivariate time series with missing values. Scientific reports, Vol. 8, 1 (2018), 1--12.

[10]

Yu Cheng, Fei Wang, Ping Zhang, and Jianying Hu. 2016. Risk prediction with electronic health records: A deep learning approach. In Proceedings of the 2016 SIAM International Conference on Data Mining. SIAM, 432--440.

[11]

Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014).

[12]

Edward Choi, Mohammad Taha Bahadori, Elizabeth Searles, Catherine Coffey, Michael Thompson, James Bost, Javier Tejedor-Sojo, and Jimeng Sun. 2016a. Multi-layer representation learning for medical concepts. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1495--1504.

Digital Library

[13]

Edward Choi, Mohammad Taha Bahadori, Le Song, Walter F Stewart, and Jimeng Sun. 2017. GRAM: graph-based attention model for healthcare representation learning. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 787--795.

Digital Library

[14]

Edward Choi, Mohammad Taha Bahadori, Jimeng Sun, Joshua Kulas, Andy Schuetz, and Walter Stewart. 2016b. Retain: An interpretable predictive model for healthcare using reverse time attention mechanism. In Advances in Neural Information Processing Systems. 3504--3512.

Digital Library

[15]

Edward Choi, Cao Xiao, Walter Stewart, and Jimeng Sun. 2018. Mime: Multilevel medical embedding of electronic health records for predictive healthcare. In Advances in Neural Information Processing Systems. 4547--4557.

[16]

Shannon M Dunlay, Susan A Weston, Steven J Jacobsen, and Véronique L Roger. 2009. Risk factors for heart failure: a population-based case-control study. The American journal of medicine, Vol. 122, 11 (2009), 1023--1028.

[17]

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, Vol. 9, 8 (1997), 1735--1780.

[18]

Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).

[19]

Bum Chul Kwon, Min-Je Choi, Joanne Taery Kim, Edward Choi, Young Bin Kim, Soonwook Kwon, Jimeng Sun, and Jaegul Choo. 2018. Retainvis: Visual analytics with interpretable and interactive recurrent neural networks on electronic medical records. IEEE transactions on visualization and computer graphics, Vol. 25, 1 (2018), 299--309.

[20]

Yikuan Li, Shishir Rao, Jose Roberto Ayala Solares, Abdelaali Hassaine, Dexter Canoy, Yajie Zhu, Kazem Rahimi, and Gholamreza Salimi-Khorshidi. 2019. BEHRT: Transformer for Electronic Health Records. arXiv preprint arXiv:1907.09538 (2019).

[21]

Andy Liaw, Matthew Wiener, et al. 2002. Classification and regression by randomForest. R news, Vol. 2, 3 (2002), 18--22.

[22]

Minh-Thang Luong, Hieu Pham, and Christopher D Manning. 2015. Effective approaches to attention-based neural machine translation. arXiv preprint arXiv:1508.04025 (2015).

[23]

Fenglong Ma, Radha Chitta, Jing Zhou, Quanzeng You, Tong Sun, and Jing Gao. 2017. Dipole: Diagnosis prediction in healthcare via attention-based bidirectional recurrent neural networks. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining. ACM, 1903--1911.

Digital Library

[24]

Fenglong Ma, Jing Gao, Qiuling Suo, Quanzeng You, Jing Zhou, and Aidong Zhang. 2018a. Risk prediction on electronic health records with prior medical knowledge. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 1910--1919.

Digital Library

[25]

Fenglong Ma, Yaqing Wang, Houping Xiao, Ye Yuan, Radha Chitta, Jing Zhou, and Jing Gao. 2018b. A General Framework for Diagnosis Prediction via Incorporating Medical Code Descriptions. In 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). IEEE, 1070--1075.

[26]

Fenglong Ma, Quanzeng You, Houping Xiao, Radha Chitta, Jing Zhou, and Jing Gao. 2018 d. Kame: Knowledge-based attention model for diagnosis prediction in healthcare. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management. ACM, 743--752.

Digital Library

[27]

Liantao Ma, Chaohe Zhang, Yasha Wang, Wenjie Ruan, Jiantao Wang, Wen Tang, Xinyu Ma, Xin Gao, and Junyi Gao. 2019. Concare: Personalized clinical feature embedding via capturing the healthcare context. arXiv preprint arXiv:1911.12216 (2019).

[28]

Liantao Ma, Chaohe Zhang, Yasha Wang, Wenjie Ruan, Jiantao Wang, Wen Tang, Xinyu Ma, Xin Gao, and Junyi Gao. 2020. Concare: Personalized clinical feature embedding via capturing the healthcare context. In Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence.

[29]

Tengfei Ma, Cao Xiao, and Fei Wang. 2018c. Health-atm: A deep architecture for multifaceted patient health record representation and risk prediction. In Proceedings of the 2018 SIAM International Conference on Data Mining. SIAM, 261--269.

[30]

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. PyTorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems. 8024--8035.

[31]

Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. 2011. Scikit-learn: Machine learning in Python. Journal of machine learning research, Vol. 12, Oct (2011), 2825--2830.

Digital Library

[32]

Trang Pham, Truyen Tran, Dinh Phung, and Svetha Venkatesh. 2016. Deepcare: A deep dynamic memory model for predictive medicine. In Pacific-Asia Conference on Knowledge Discovery and Data Mining. Springer, 30--41.

Digital Library

[33]

George AF Seber and Alan J Lee. 2012. Linear regression analysis. Vol. 329. John Wiley & Sons.

[34]

Junyuan Shang, Tengfei Ma, Cao Xiao, and Jimeng Sun. 2019. Pre-training of graph augmented transformers for medication recommendation. arXiv preprint arXiv:1906.00346 (2019).

[35]

Huan Song, Deepta Rajan, Jayaraman J Thiagarajan, and Andreas Spanias. 2018. Attend and diagnose: Clinical time series analysis using attention models. In Thirty-second AAAI conference on artificial intelligence.

[36]

Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, Vol. 15, 1 (2014), 1929--1958.

[37]

Qiuling Suo, Fenglong Ma, Giovanni Canino, Jing Gao, Aidong Zhang, Pierangelo Veltri, and Gnasso Agostino. 2017a. A multi-task framework for monitoring health conditions via attention-based recurrent neural networks. In AMIA annual symposium proceedings, Vol. 2017. American Medical Informatics Association, 1665.

[38]

Qiuling Suo, Fenglong Ma, Ye Yuan, Mengdi Huai, Weida Zhong, Jing Gao, and Aidong Zhang. 2018. Deep patient similarity learning for personalized healthcare. IEEE transactions on nanobioscience, Vol. 17, 3 (2018), 219--227.

[39]

Qiuling Suo, Fenglong Ma, Ye Yuan, Mengdi Huai, Weida Zhong, Aidong Zhang, and Jing Gao. 2017b. Personalized disease prediction using a cnn-based similarity learning method. In 2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). IEEE, 811--816.

[40]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998--6008.

[41]

Lipo Wang. 2005. Support vector machines: theory and applications. Vol. 177. Springer Science & Business Media.

[42]

Xiancheng Xie, Yun Xiong, Philip S Yu, and Yangyong Zhu. 2019. EHR Coding with Multi-scale Feature Attention and Structured Knowledge Graph Propagation. In CIKM. 649--658.

[43]

Changchang Yin, Rongjian Zhao, Buyue Qian, Xin Lv, and Ping Zhang. 2019. Domain Knowledge Guided Deep Learning with Electronic Health Records. In 2019 IEEE International Conference on Data Mining (ICDM).

[44]

Xianli Zhang, Buyue Qian, Yang Li, Changchang Yin, Xudong Wang, and Qinghua Zheng. 2019 a. KnowRisk: An Interpretable Knowledge-Guided Model for Disease Risk Prediction. In ICDM. IEEE, 1492--1497.

[45]

Xi Sheryl Zhang, Fengyi Tang, Hiroko H. Dodge, Jiayu Zhou, and Fei Wang. 2019 b. MetaPred: Meta-Learning for Clinical Risk Prediction with Limited Patient Electronic Health Records (KDD '19). ACM, New York, NY, USA, 2487--2495. http://doi.acm.org/10.1145/3292500.3330779

Cited By

Wen JXue HRush EPanickan VCai TZhou DHo YCosta LBegoli EHong CGaziano JCho KLiao KLu JCai T(2025)DOME: Directional medical embedding vectors from Electronic Health RecordsJournal of Biomedical Informatics10.1016/j.jbi.2024.104768162(104768)Online publication date: Feb-2025
https://doi.org/10.1016/j.jbi.2024.104768
Cissoko MCastelain VLachiche N(2025)Predicting and Interpreting Healthcare Trajectories from Irregularly Collected Sequential Patient Data Using AMITAInformation Sciences10.1016/j.ins.2025.121977(121977)Online publication date: Feb-2025
https://doi.org/10.1016/j.ins.2025.121977
Cissoko MCastelain VLachiche N(2025)Multi-Way adaptive Time Aware LSTM for irregularly collected sequential ICU dataExpert Systems with Applications10.1016/j.eswa.2024.125548261(125548)Online publication date: Feb-2025
https://doi.org/10.1016/j.eswa.2024.125548
Show More Cited By

Index Terms

HiTANet: Hierarchical Time-Aware Attention Networks for Risk Prediction on Electronic Health Records
1. Applied computing
  1. Life and medical sciences
2. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning
        Supervised learning by classification
    2. Machine learning approaches
      1. Neural networks

Recommendations

Dipole: Diagnosis Prediction in Healthcare via Attention-based Bidirectional Recurrent Neural Networks
KDD '17: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Predicting the future health information of patients from the historical Electronic Health Records (EHR) is a core research task in the development of personalized healthcare. Patient EHR data consist of sequences of visits over time, where each visit ...
MedPath: Augmenting Health Risk Prediction via Medical Knowledge Paths
WWW '21: Proceedings of the Web Conference 2021

The broad adoption of electronic health records (EHR) data and the availability of biomedical knowledge graphs (KGs) on the web have provided clinicians and researchers unprecedented resources and opportunities for conducting health risk predictions to ...
StageNet: Stage-Aware Neural Networks for Health Risk Prediction
WWW '20: Proceedings of The Web Conference 2020

Deep learning has demonstrated success in health risk prediction especially for patients with chronic and progressing conditions. Most existing works focus on learning disease patterns from longitudinal patient data, but pay little attention to the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

KDD '20: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

August 2020

3664 pages

ISBN:9781450379984

DOI:10.1145/3394486

General Chairs:
Rajesh Gupta
UC San Diego, USA
,
Yan Liu
USC, USA
,
Program Chairs:
Mohak Shah
LG Electronics, USA
,
Suju Rajan
Linkedin, USA
,
Publications Chairs:
Jiliang Tang
Michigan State, USA
,
B. Aditya Prakash
Georgia Tech, USA

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 August 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

KDD '20

Sponsor:

KDD '20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

July 6 - 10, 2020

CA, Virtual Event, USA

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Upcoming Conference

KDD '25

Sponsor:
sigkdd
sigkdd

The 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 3 - 7, 2025

Toronto , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

130
Total Citations
View Citations
3,498
Total Downloads

Downloads (Last 12 months)474
Downloads (Last 6 weeks)44

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wen JXue HRush EPanickan VCai TZhou DHo YCosta LBegoli EHong CGaziano JCho KLiao KLu JCai T(2025)DOME: Directional medical embedding vectors from Electronic Health RecordsJournal of Biomedical Informatics10.1016/j.jbi.2024.104768162(104768)Online publication date: Feb-2025
https://doi.org/10.1016/j.jbi.2024.104768
Cissoko MCastelain VLachiche N(2025)Predicting and Interpreting Healthcare Trajectories from Irregularly Collected Sequential Patient Data Using AMITAInformation Sciences10.1016/j.ins.2025.121977(121977)Online publication date: Feb-2025
https://doi.org/10.1016/j.ins.2025.121977
Cissoko MCastelain VLachiche N(2025)Multi-Way adaptive Time Aware LSTM for irregularly collected sequential ICU dataExpert Systems with Applications10.1016/j.eswa.2024.125548261(125548)Online publication date: Feb-2025
https://doi.org/10.1016/j.eswa.2024.125548
Park WMulyadi AKang ESuk H(2025)Prototype-Guided Contrastive Knowledge Graph Representation Learning for Diagnosis PredictionPattern Recognition and Artificial Intelligence10.1007/978-981-97-8702-9_18(262-275)Online publication date: 8-Feb-2025
https://doi.org/10.1007/978-981-97-8702-9_18
Ding ZLi ZLi XLi H(2024)DRR: Global Context-Aware Neural Network Using Disease Relationship Reasoning and Attention-Based Feature FusionMathematics10.3390/math1203048812:3(488)Online publication date: 2-Feb-2024
https://doi.org/10.3390/math12030488
Wang JLuo JYe MWang XZhong YChang AHuang GYin ZXiao CSun JMa FLarson K(2024)Recent advances in predictive modeling with electronic health recordsProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/914(8272-8280)Online publication date: 3-Aug-2024
https://dl.acm.org/doi/10.24963/ijcai.2024/914
Sohn HPark KPark BChi MLarson K(2024)Multi-TAProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/667(6035-6043)Online publication date: 3-Aug-2024
https://dl.acm.org/doi/10.24963/ijcai.2024/667
Chen JYin CWang YZhang PLarson K(2024)Predictive modeling with temporal graphical representation on electronic health recordsProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/637(5763-5771)Online publication date: 3-Aug-2024
https://dl.acm.org/doi/10.24963/ijcai.2024/637
Li FChen YGu YWang Y(2024)Extracting Integrated Features of Electronic Medical Records Big Data for Mortality and Phenotype PredictionChinese Journal of Electronics10.23919/cje.2023.00.18133:3(776-792)Online publication date: May-2024
https://doi.org/10.23919/cje.2023.00.181
Hama TAlsaleh MAllery FChoi JTomlinson CWu HLai APontikos NThygesen J(2024)Enhancing Patient Outcome Prediction through Deep Learning with Sequential Diagnosis Codes from Structured EHR data: A systematic review (Preprint)Journal of Medical Internet Research10.2196/57358Online publication date: 19-Feb-2024
https://doi.org/10.2196/57358
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten