Abstract
Cancer patients often experience numerous hospital admissions as a result of their cancer and treatment, which can negatively impact treatment progress and quality of life. Accurately predicting outcomes for cancer patients is therefore crucial in providing personalised care and improving patient outcomes. Existing models leveraging deep learning with Electronic Health Record (EHR) data to predict outcomes for cancer patients are limited, despite the demonstrated success of these approaches with cancer imaging data and non-cancer EHR applications. Additionally, current methods focus on single-task predictions, and increasing evidence suggests jointly training a model on two related tasks can improve predictive performance. To address these limitations, we propose a Transformer-based Multi-Task (TransMT) model that captures relationships between diagnosis codes and sequential hospital visits to simultaneously predict related outcomes for hospitalised cancer patients. Experiments conducted on two public datasets show the proposed model outperforms both single-task and recurrent neural network approaches in predicting future diagnosis and hospital readmission, and demonstrates the benefits of using deep learning with EHR data for cancer-related research.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Brunner-La Rocca, H.P., Peden, C., Soong, J., Holman, P., Bogdanovskaya, M., Barclay, L.: Reasons for readmission after hospital discharge in patients with chronic diseases-information from an international dataset. PLOS ONE 15, e0233457 (2020)
Cai, X., Gao, J., Ngiam, K.Y., Ooi, B.C., Zhang, Y., Yuan, X.: Medical concept embedding with time-aware attention. In: IJCAI, pp. 3984–3990 (2018)
Choi, E., Bahadori, M.T., Sun, J., Kulas, J., Schuetz, A., Stewart, W.: Retain: an interpretable predictive model for healthcare using reverse time attention mechanism. In: NeurIPS, pp. 3504–3512 (2016)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Fadol, A., et al.: A quality improvement approach to reducing hospital readmissions in patients with cancer and heart failure. Cardio-Oncol. 5, 5 (2019)
Gao, R., et al.: Deep multi-task prediction of lung cancer and cancer-free progression from censored heterogenous clinical imaging. CoRR (2019)
Gensheimer, M., et al.: Automated survival prediction in metastatic cancer patients using high-dimensional electronic medical record data. Ann. Oncol. 29 (2018)
Gupta, S., et al.: Machine-learning prediction of cancer survival: a retrospective study using electronic administrative records and a cancer registry. BMJ Open 4(3) (2014)
Harutyunyan, H., Khachatrian, H., Kale, D.C., Ver Steeg, G., Galstyan, A.: Multitask learning and benchmarking with clinical time series data. Sci. Data 6(1) (2019)
Hughes, L., Witham, M.: Causes and correlates of 30 day and 180 day readmission following discharge from a medicine for the elderly rehabilitation unit. BMC Geriatrics 18 (2018)
Ji, H., Abushomar, H., Chen, X.K., Qian, C., Gerson, D.: All-cause readmission to acute care for cancer patients. Healthc. Quart. Toronto Ont. 15, 14–6 (2012)
Johnson, A., Bulgarelli, L., Pollard, T., Horng, S., Celi, L.A., Mark, R.: Mimic-iv (version 0.4). PhysioNet (2020)
Johnson, A.E., et al.: Mimic-iii, a freely accessible critical care database. Sci. Data 3, 160035 (2016)
Lahlou, C., Crayton, A., Trier, C., Willett, E.: Explainable health risk predictor with transformer-based medicare claim encoder. CoRR abs/2105.09428 arxiv.org/abs/2105.09428 (2021)
Le, T.L.T., Thome, N., Bernard, S., Bismuth, V., Patoureaux, F.: Multitask classification and segmentation for cancer diagnosis in mammography. arXiv preprint arXiv:1909.05397 (2019)
Li, Y., et al.: BEHRT: transformer for electronic health records. CoRR (2019)
Lin, Z., Feng, M., Santos, C.N.d., Yu, M., Xiang, B., Zhou, B., Bengio, Y.: A structured self-attentive sentence embedding. arXiv:1703.03130 (2017)
Liu, L., et al.: Multi-task learning via adaptation to similar tasks for mortality prediction of diverse rare diseases. In: AMIA Annual Symposium Proceedings. American Medical Informatics Association, vol. 2020, p. 763 (2020)
Long, G., Shen, T., Tan, Y., Gerrard, L., Clarke, A., Jiang, J.: Federated learning for privacy-preserving open innovation future on digital health. arXiv preprint arXiv:2108.10761 (2021)
Ma, F., Chitta, R., Zhou, J., You, Q., Sun, T., Gao, J.: Dipole: diagnosis prediction in healthcare via attention-based bidirectional recurrent neural networks. In: SIGKDD, pp. 1903–1911. ACM (2017)
McDermott, M., et al.: A comprehensive evaluation of multi-task learning and multi-task pre-training on ehr time-series data. arXiv preprint arXiv:2007.10185 (2020)
Miaskowski, C., et al.: Latent class analysis reveals distinct subgroups of patients based on symptom occurrence and demographic and clinical characteristics. J. Pain Symptom Manage. 50(1), 28–37 (2015)
Min, X., Yu, B., Wang, F.: Predictive modeling of the hospital readmission risk from patients’ claims data using machine learning: a case study on copd. Sci. Reports 9, 2362 (2019)
Parikh, R.B., et al.: Machine learning approaches to predict 6-Month mortality among patients with cancer. JAMA Netw. Open 2(10), e1915997–e1915997 (2019)
Peng, X., Long, G., Shen, T., Wang, S., Jiang, J., Blumenstein, M.: Temporal self-attention network for medical concept embedding. In: ICDM, pp. 498–507. IEEE (2019)
Peng, X., Long, G., Shen, T., Wang, S., Jiang, J., Zhang, C.: Bitenet: bidirectional temporal encoder network to predict medical outcomes. In: ICDM, pp. 412–421. IEEE (2020)
Rao, S., et al.: An explainable transformer-based deep learning model for the prediction of incident heart failure. CoRR (2021)
Rasmy, L., Xiang, Y., Xie, Z., Tao, C., Zhi, D.: Med-bert: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction. NPJ Digital Med. 4(1), 1–13 (2021)
Ruck, J., Canner, J., Smith, T., Johnston, F.: Use of inpatient palliative care by type of malignancy. J. Palliative Med. 21 (2018)
Ruder, S.: An overview of multi-task learning in deep neural networks. arXiv preprint arXiv:1706.05098 (2017)
Vaswani, A., et al.: Attention is all you need. In: NeurIPS, pp. 5998–6008 (2017)
Zeiler, M.D.: Adadelta: an adaptive learning rate method. arXiv preprint arXiv:1212.5701 (2012)
Acknowledgements
This research is supported by an Australian Government Research Training Program Scholarship. We also thank the Australian Government Department of Health for supporting this work.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Gerrard, L., Peng, X., Clarke, A., Schlegel, C., Jiang, J. (2022). Predicting Outcomes for Cancer Patients with Transformer-Based Multi-task Learning. In: Long, G., Yu, X., Wang, S. (eds) AI 2021: Advances in Artificial Intelligence. AI 2022. Lecture Notes in Computer Science(), vol 13151. Springer, Cham. https://doi.org/10.1007/978-3-030-97546-3_31
Download citation
DOI: https://doi.org/10.1007/978-3-030-97546-3_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-97545-6
Online ISBN: 978-3-030-97546-3
eBook Packages: Computer ScienceComputer Science (R0)