Abstract
Lymph node metastasis (LNM) is the most critical prognosis factor in esophageal squamous cell carcinoma (ESCC). Effective and adaptive integration of preoperative CT images and multi-sourced non-imaging clinical factors is a challenging issue. In this work, we propose a graph-based reasoning model to learn new representations from multi-categorical clinical parameters for LNM prediction. Given CT, general, diagnostic, pathological, and hematological clinical information, we firstly propose a graph construction strategy with category-wise contextual attention to embed multi-categorical features as graph node attributes. Secondly, we introduce a co-graph attention layer composed of a conventional graph attention network (con-GAT) and a correlation-based GAT (corr-GAT) to learn new representations. Corr-GAT complements con-GAT by difference-based correlations across image regions in global spectral space. Experimental results of ablation studies and comparison with others over 924 lymph nodes demonstrated improved performance and contributions of our major innovations. Our model has the potential to foster early prognosis and personalized surgery or radiotherapy planning in ESCC patients.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Lee, H.N., Kim, J.I., Shin, S.Y., Kim, D.H., Kim, C., Hong, I.K.: Combined CT texture analysis and nodal axial ratio for detection of nodal metastasis in esophageal cancer. Br. J. Radiol. 93, 20190827 (2020)
Cancer Stat Facts: Esophageal Cancer. National Cancer Institute. https://seer.cancer.gov/statfacts/html/esoph.html
Lee, J.Y., et al.: Improved detection of metastatic lymph nodes in oesophageal squamous cell carcinoma by combined interpretation of fluorine-18-fluorodeoxyglucose positron-emission tomography/computed tomography. Cancer Imaging 19, 40 (2019)
Liu, J., Wang, Z., Shao, H., Qu, D., Liu, J., Yao, L.: Improving CT detection sensitivity for nodal metastases in oesophageal cancer with combination of smaller size and lymph node axial ratio. Eur. Radiol. 28(1), 188–195 (2017). https://doi.org/10.1007/s00330-017-4935-4
Foley, K., Christian, A., Fielding, P., Lewis, W., Roberts, S.: Accuracy of contemporary oesophageal cancer lymph node staging with radiological-pathological correlation. J Clin. Radiol. 72, 693-e691 (2017)
Wu, L., et al.: Multiple level CT radiomics features preoperatively predict lymph node metastasis in esophageal cancer: a multicentre retrospective study. Front Oncol 9, 1548 (2019)
Vallieres, M., Freeman, C.R., Skamene, S.R., El Naqa, I.: A radiomics model from joint FDG-PET and MRI texture features for the prediction of lung metastases in soft-tissue sarcomas of the extremities. Phys Med Biol 60, 5471–5496 (2015)
Chauhan, G., et al.: Joint modeling of chest radiographs and radiology reports for pulmonary edema assessment. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12262, pp. 529–539. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59713-9_51
Sharma, A., Vans, E., Shigemizu, D., Boroevich, K.A., Tsunoda, T.: DeepInsight: a methodology to transform a non-image data to an image for convolution neural network architecture. Sci. Rep. 9, 11399 (2019)
Zhang, K., Zhu, Y., Zhang, W., Zhu, Y.: Cross-modal image sentiment analysis via deep correlation of textual semantic. Knowl.-Based Syst. 216 (2021)
Ye, L., Liu, Z., Wang, Y.: Dual convolutional LSTM network for referring image segmentation. IEEE Trans. Multimedia 22, 3224–3235 (2020)
Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., Philip, S.Y.: A comprehensive survey on graph neural networks. IEEE Trans. Neural Netw. Learn. Syst. 32, 4–24 (2020)
Xuan, P., Zhang, Y., Cui, H., Zhang, T., Guo, M., Nakaguchi, T.: Integrating multi-scale neighbouring topologies and cross-modal similarities for drug–protein interaction prediction. Briefings in Bioinf. 119 (2021)
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. Int. Conf. Learn. Rep. (2018)
Mo, S., et al.: Multimodal priors guided segmentation of liver lesions in MRI using mutual information based graph co-attention networks. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12264, pp. 429–438. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59719-1_42
Isensee, F., Jaeger, P.F., Kohl, S.A.A., Petersen, J., Maier-Hein, K.H.: nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat. Methods 18, 203–211 (2021)
Jin, Q., Meng, Z., Sun, C., Cui, H., Su, R.: RA-UNet: a hybrid deep attention-aware network to extract liver and tumor in CT scans. Front. Bioeng. Biotechnol. 8, 1–15 (2020)
Sheng, N., Cui, H., Zhang, T., Xuan, P.: Attentional multi-level representation encoding based on convolutional and variance autoencoders for lncRNA-disease association prediction. Briefings Bioinf. 22 (2020)
Wang, T., Wang, G., Tan, K.E., Tan, D.: Spectral Pyramid Graph Attention Network for Hyperspectral Image Classification. arXiv preprint arXiv:2001.07108 (2020)
Schlemper, J., et al.: Attention gated networks: learning to leverage salient regions in medical images. Med Image Anal 53, 197–207 (2019)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Cui, H. et al. (2021). Co-graph Attention Reasoning Based Imaging and Clinical Features Integration for Lymph Node Metastasis Prediction. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12905. Springer, Cham. https://doi.org/10.1007/978-3-030-87240-3_63
Download citation
DOI: https://doi.org/10.1007/978-3-030-87240-3_63
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87239-7
Online ISBN: 978-3-030-87240-3
eBook Packages: Computer ScienceComputer Science (R0)
