Abstract
Recent years have witnessed an increased interest in the biomedical research community in developing machine learning models and methods that can automatically assign diagnostic codes (ICD) to patient stays based on the information in their Electronic Health Records (EHR). However, despite the recent advances, accurate automatic classification of diagnostic codes continues to face challenges, especially for low-prior diagnostic codes. To alleviate the problem, we propose to leverage information in the diagnostic hierarchy and better utilize the dependencies among diseases in this hierarchy. We develop a new hierarchical deep multi-task learning method that learns classification models for multiple diagnostic codes at the different levels of abstraction in the disease hierarchy while allowing the transfer of information from high-level nodes, more general diagnoses codes to the low-level ones, more specific diagnostic codes. After that, we refine the initial hierarchical model by utilizing the relations and information that can discriminate better between competing diseases. Our empirical results show that our new method and its refinement outperform baseline machine learning architectures that do not leverage the hierarchical structure of target diagnoses tasks or disease-disease relationships.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ben-David, S., Schuller, R.: Exploiting task relatedness for multiple task learning. In: Schölkopf, B., Warmuth, M.K. (eds.) COLT-Kernel 2003. LNCS (LNAI), vol. 2777, pp. 567–580. Springer, Heidelberg (2003). https://doi.org/10.1007/978-3-540-45167-9_41
Choi, E., et al.: Mime: multilevel medical embedding of electronic health records for predictive healthcare. arXiv preprint arXiv:1810.09593 (2018)
Crawshaw, M.: Multi-task learning with deep neural networks: a survey. arXiv preprint arXiv:2009.09796 (2020)
Fan, J., et al.: HD-MTL: hierarchical deep multi-task learning for large-scale visual recognition. IEEE Trans. Image Process. 26(4), 1923–1938 (2017)
Han, L., Zhang, Y.: Learning tree structure in multi-task learning. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 397–406. ACM (2015)
Jacob, L., Vert, J.P., Bach, F.R.: Clustered multi-task learning: a convex formulation. In: Advances in Neural Information Processing Systems, pp. 745–752 (2009)
Johnson, A.E., et al.: MIMIC-III, a freely accessible critical care database. Sci. Data 3, 160035 (2016)
Kang, Z., Grauman, K., Sha, F.: Learning with whom to share in multi-task feature learning. In: ICML, vol. 2, p. 4 (2011)
Krishna, K., Toshniwal, S., Livescu, K.: Hierarchical multitask learning for CTC-based speech recognition. arXiv preprint arXiv:1807.06234 (2018)
Lee, G., Yang, E., Hwang, S.: Asymmetric multi-task learning based on task relatedness and loss. In: International Conference on Machine Learning (2016)
Lee, H.B., Yang, E., Hwang, S.J.: Deep asymmetric multi-task feature learning. In: International Conference on Machine Learning, pp. 2956–2964. PMLR (2018)
Lee, J.M., Hauskrecht, M.: Modeling multivariate clinical event time-series with recurrent temporal mechanisms. Artif. Intell. Med. 112 (2021)
Li, Y., et al.: BEHRT: transformer for electronic health records. Sci. Rep. 10(1), 1–12 (2020)
Lipton, Z.C., Kale, D.C., Elkan, C., Wetzel, R.: Learning to diagnose with LSTM recurrent neural networks. arXiv preprint (2015)
Long, M., Cao, Y., Wang, J., Jordan, M.I.: Learning transferable features with deep adaptation networks. arXiv preprint arXiv:1502.02791 (2015)
Lu, Y., Kumar, A., Zhai, S., Cheng, Y., Javidi, T., Feris, R.: Fully-adaptive feature sharing in multi-task networks with applications in person attribute classification. In: IEEE Conference on Computer Vision and Pattern Recognition (2017)
Malakouti, S., Hauskrecht, M.: Hierarchical adaptive multi-task learning framework for patient diagnoses and diagnostic category classification. In: 2019 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). IEEE (2019)
Malakouti, S., Hauskrecht, M.: Predicting patient’s diagnoses and diagnostic categories from clinical-events in EHR data. In: Riaño, D., Wilk, S., ten Teije, A. (eds.) AIME 2019. LNCS (LNAI), vol. 11526, pp. 125–130. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-21642-9_17
Miotto, R., Li, L., Kidd, B.A., Dudley, J.T.: Deep patient: an unsupervised representation to predict the future of patients from the electronic health records. Sci. Rep. 6, 26094 (2016)
Rajkomar, A., et al.: Scalable and accurate deep learning with electronic health records. NPJ Digit. Med. 1(1), 1–10 (2018)
Rasmy, L., Xiang, Y., Xie, Z., Tao, C., Zhi, D.: Med-BERT: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction. NPJ Digit. Med. 4(1), 1–13 (2021)
Rosenstein, M.T., Marx, Z., Kaelbling, L.P., Dietterich, T.G.: To transfer or not to transfer. In: NIPS 2005 Workshop on Transfer Learning, vol. 898, pp. 1–4 (2005)
Sanh, V., Wolf, T., Ruder, S.: A hierarchical multi-task approach for learning embeddings from semantic tasks. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 6949–6956 (2019)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
Zhang, J., Kowsari, K., Harrison, J.H., Lobo, J.M., Barnes, L.E.: Patient2Vec: a personalized interpretable deep representation of the longitudinal electronic health record. IEEE Access 6, 65333–65346 (2018)
Zhang, W., Deng, L., Zhang, L., Wu, D.: Overcoming negative transfer: a survey. arXiv preprint arXiv:2009.00909 (2020)
Zhang, Y., Yang, Q.: A survey on multi-task learning. arXiv preprint arXiv:1707.08114 (2017)
Zweig, A., Weinshall, D.: Hierarchical regularization cascade for joint learning. In: International Conference on Machine Learning, pp. 37–45 (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Malakouti, S., Hauskrecht, M. (2022). Hierarchical Deep Multi-task Learning for Classification of Patient Diagnoses. In: Michalowski, M., Abidi, S.S.R., Abidi, S. (eds) Artificial Intelligence in Medicine. AIME 2022. Lecture Notes in Computer Science(), vol 13263. Springer, Cham. https://doi.org/10.1007/978-3-031-09342-5_12
Download citation
DOI: https://doi.org/10.1007/978-3-031-09342-5_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-09341-8
Online ISBN: 978-3-031-09342-5
eBook Packages: Computer ScienceComputer Science (R0)