Abstract
Colorectal Cancer (CRC) is one of the most common cancer diagnosed in humans. Outcomes vary significantly among patients with different tumor status. Accurate staging of colorectal cancer for personalized treatment is thus highly desired. Whole slide pathological images (WSIs) serves as the gold standard for Tumour Node Metastasis (TNM) staging. However, TNM staging for colorectal cancer relies on labor-intensive manual discriminative patch labeling, which is not suitable and scalable for large-scale WSIs TNM staging. Though various methods have been proposed to select key image patches to perform staging, they are unable to consider the structure of tissue types in biopsy samples which is a key evidence for determining tumor status. In this paper, we propose a Graph Attention Multi-instance Learning (Graph Attention MIL) with texture features, which encodes a spatial structure between patches and jointly predicts the TNM staging. We evaluated our proposed method on a large cohort of colorectal cancer dataset. The proposed framework improves the performance over the existing state-of-the-art methods indicating the future research towards graph based learning for TNM staging.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
https://www.cancer.net/cancer-types/colorectal-cancer/statistics (2019)
Chen, H., et al.: Anatomy-aware Siamese network: exploiting semantic asymmetry for accurate pelvic fracture detection in x-ray images (2020)
Gupta, P., et al.: Prediction of colon cancer stages and survival period with machine learning approach. Cancers 11(12), 2007 (2019)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Ilse, M., Tomczak, J.M., Welling, M.: Attention-based deep multiple instance learning. arXiv preprint arXiv:1802.04712 (2018)
Jonnagaddala, J., et al.: Integration and analysis of heterogeneous colorectal cancer data for translational research, p. 387 (2016)
Kather, J.N., et al.: Predicting survival from colorectal cancer histology slides using deep learning: a retrospective multicenter study. PLoS Med. 16(1), e1002730 (2019)
Lee, J., Lee, I., Kang, J.: Self-attention graph pooling. arXiv preprint arXiv:1904.08082 (2019)
Li, R., Yao, J., Zhu, X., Li, Y., Huang, J.: Graph CNN for survival analysis on whole slide pathological images. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11071, pp. 174–182. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00934-2_20
Li, W., Nguyen, V.D., Liao, H., Wilder, M., Cheng, K., Luo, J.: Patch transformer for multi-tagging whole slide histopathology images. In: Shen, D., Liu, T., Peters, T.M., Staib, L.H., Essert, C., Zhou, S., Yap, P.T., Khan, A. (eds.) Medical Image Computing and Computer Assisted Intervention - MICCAI 2019. LNCS, vol. 11764, pp. 532–540. Springer, Cham (2019)
Morkūnas, M., Treigys, P., Bernatavičienė, J., Laurinavičius, A., Korvel, G.: Machine learning based classification of colorectal cancer tumour tissue in whole-slide images. Informatica 29(1), 75–90 (2018)
Muhammad, H., et al.: Unsupervised subtyping of cholangiocarcinoma using a deep clustering convolutional autoencoder. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11764, pp. 604–612. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32239-7_67
Shapcott, C.M., Rajpoot, N., Hewitt, K.: Deep learning with sampling for colon cancer histology images. Front. Bioeng. Biotechnol. 7, 52 (2019)
Tellez, D., van der Laak, J., Ciompi, F.: Gigapixel whole-slide image classification using unsupervised image compression and contrastive training (2018)
Wang, Y., et al.: Weakly Supervised Universal Fracture Detection in Pelvic X-Rays. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11769, pp. 459–467. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32226-7_51
Ward, R.L., Hawkins, N.J.: Molecular and cellular oncology (MCO) study tumour collection. UNSW Aust. (2015). https://doi.org/10.4225/53/5559205bea135
Wu, J.S.: Rectal cancer staging. Clin. Colon Rectal Surg. 20(03), 148–157 (2007)
Wu, Z., Xiong, Y., Yu, S.X., Lin, D.: Unsupervised feature learning via non-parametric instance discrimination. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3733–3742 (2018)
Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., Yu, P.S.: A comprehensive survey on graph neural networks. arXiv preprint arXiv:1901.00596 (2019)
Xu, J., et al.: Multi-tissue partitioning for whole slide images of colorectal cancer histopathology images with deeptissue net. In: Reyes-Aldasoro, C.C., Janowczyk, A., Veta, M., Bankhead, P., Sirinukunwattana, K. (eds.) ECDP 2019. LNCS, vol. 11435, pp. 100–108. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-23937-4_12
Yan, C., Yao, J., Li, R., Xu, Z., Huang, J.: Weakly supervised deep learning for thoracic disease classification and localization on chest x-rays. In: Proceedings of the 2018 ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics, pp. 103–110 (2018)
Yao, J., Cai, J., Yang, D., Xu, D., Huang, J.: Integrating 3D geometry of organ for improving medical image segmentation. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11768, pp. 318–326. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32254-0_36
Yao, J., Wang, S., Zhu, X., Huang, J.: Imaging biomarker discovery for lung cancer survival prediction. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 649–657. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46723-8_75
Yao, J., Zhu, X., Huang, J.: Deep multi-instance learning for survival prediction from whole slide images. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11764, pp. 496–504. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32239-7_55
Yao, Jiawen., Zhu, Xinliang., Zhu, Feiyun, Huang, Junzhou: Deep correlational learning for survival prediction from multi-modality data. In: Descoteaux, M., et al. (eds.) MICCAI 2017. LNCS, vol. 10434, pp. 406–414. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66185-8_46
Ye, M., Zhang, X., Yuen, P.C., Chang, S.F.: Unsupervised embedding learning via invariant and spreading instance feature. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6210–6219 (2019)
Zhang, H., Xue, J., Dana, K.: Deep TEN: texture encoding network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 708–717 (2017)
Zhou, Y., Graham, S., Koohbanani, N.A., Shaban, M., Heng, P.A., Rajpoot,N.: CGC-net: cell graph convolutional network for grading of colorectal cancer histology images. In: Proceedings of the IEEE International Conference on Computer Vision Workshops (2019)
Zhu, X., Yao, J., Zhu, F., Huang, J.: WSISA: making survival prediction from whole slide Histopathological images. In: CVPR., pp. 7234–7242 (2017)
Acknowledgments
This work was partially supported by US National Science Foundation IIS-1718853, the CAREER grant IIS-1553687 and Cancer Prevention and Research Institute of Texas (CPRIT) award (RP190107).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Raju, A., Yao, J., Haq, M.M., Jonnagaddala, J., Huang, J. (2020). Graph Attention Multi-instance Learning for Accurate Colorectal Cancer Staging. In: Martel, A.L., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science(), vol 12265. Springer, Cham. https://doi.org/10.1007/978-3-030-59722-1_51
Download citation
DOI: https://doi.org/10.1007/978-3-030-59722-1_51
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59721-4
Online ISBN: 978-3-030-59722-1
eBook Packages: Computer ScienceComputer Science (R0)
