Abstract
Cancer diagnosis, prognosis, and therapeutic response prediction are heavily influenced by the relationship between the histopathological structures and the function of the tissue. Recent approaches acknowledging the structure-function relationship, have linked the structural and spatial patterns of cell organization in tissue via cell-graphs to tumor grades. Though cell organization is imperative, it is insufficient to entirely represent the histopathological structure. We propose a novel hierarchical cell-to-tissue-graph (HACT) representation to improve the structural depiction of the tissue. It consists of a low-level cell-graph, capturing cell morphology and interactions, a high-level tissue-graph, capturing morphology and spatial distribution of tissue parts, and cells-to-tissue hierarchies, encoding the relative spatial distribution of the cells with respect to the tissue distribution. Further, a hierarchical graph neural network (HACT-Net) is proposed to efficiently map the HACT representations to histopathological breast cancer subtypes. We assess the methodology on a large set of annotated tissue regions of interest from H&E stained breast carcinoma whole-slides. Upon evaluation, the proposed method outperformed recent convolutional neural network and graph neural network approaches for breast cancer multi-class subtyping. The proposed entity-based topological analysis is more in line with the pathological diagnostic procedure of the tissue. It provides more command over the tissue modeling, therefore encourages the further inclusion of pathological priors into task-specific tissue representation.
P. Pati and G. Jaume—The authors contributed equally to this work.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Currently pending approval for releasing the dataset to the research community.
References
Achanta, R., et al.: Slic superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34, 2274–2282 (2012)
Aresta, G., et al.: BACH: grand challenge on breast cancer histology images. Med. Image Anal. 56, 122–139 (2019)
Bankhead, P., et al.: QuPath: open source software for digital pathology image analysis. Sci. Rep. 7, 1–7 (2017)
Bejnordi, B., et al.: Using deep convolutional neural networks to identify and classify tumor-associated stroma in diagnostic breast biopsies. Mod. Pathol. 31, 1502–1512 (2018)
Chen, R.J., et al.: Pathomic fusion: an integrated framework for fusing histopathology and genomic features for cancer diagnosis and prognosis. arXiv preprint arXiv:1912.08937 (2019)
Elmore, J.G., et al.: Diagnostic concordance among pathologists interpreting breast biopsy specimens. JAMA 313, 1122–1132 (2015)
Gadiya, S., et al.: Histographs: graphs in histopathology. arXiv preprint arXiv:1908.05020 (2019)
Gilmer, J., et al.: Neural message passing for quantum chemistry. ICML 70, 1263–1272 (2017)
Graham, S., et al.: Hover-Net: simultaneous segmentation and classification of nuclei in multi-tissue histology images. Med. Image Anal. 58, 101563 (2019)
Gunduz, C., et al.: The cell graphs of cancer. Bioinformatics 20(Suppl 1), i145–i151 (2004)
Hamilton, W., et al.: Inductive representation learning on large graphs. In: NeurIPS, pp. 1024–1034 (2017)
Jaume, G., et al.: Towards explainable graph representations in digital pathology. In: ICML, Workshop on Computational Biology (2020)
Kingma, D., Ba, J.: Adam: a method for stochastic optimization. In: ICLR (2015)
Kipf, T., Welling, M.: Semi-supervised classification with graph convolutional networks. In: ICLR (2017)
Kumar, N., et al.: A dataset and a technique for generalized nuclear segmentation for computational pathology. IEEE Trans. Med. Imaging 36, 1550–1560 (2017)
Litjens, G., et al.: A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60–88 (2017)
Macenko, M., et al.: A method for normalizing histology slides for quantitative analysis. In: IEEE ISBI, pp. 1107–1110 (2009)
Mercan, C., et al.: From patch-level to ROI-level deep feature representations for breast histopathology classification. In: SPIE 10956 Medical Imaging: Digital Pathology, vol. 109560H (2019)
Myers, D.J., Walls, A.L.: Atypical breast hyperplasia. In: StatPearls (2019)
Paszke, A., et al.: PyTorch: an imperative style, high-performance deep learning library. In: NeurIPS, pp. 8024–8035 (2019)
Potjer, F.: Region adjacency graphs and connected morphological operators. In: Maragos, P., Schafer, R.W., Butt, M.A. (eds.) Mathematical Morphology and its Applications to Image and Signal Processing. Computational Imaging and Vision, vol. 5, pp. 111–118. Springer, Heidelberg (1996). https://doi.org/10.1007/978-1-4613-0469-2_13
Sharma, H., et al.: Cell nuclei attributed relational graphs for efficient representation and classification of gastric cancer in digital histopathology. In: SPIE 9791 Medical Imaging: Digital Pathology, vol. 97910X (2016)
Siegel, R.L., et al.: Cancer statistics, 2016. CA Cancer J. Clin. 66, 7–30 (2016)
Sirinukunwattana, K., Alham, N.K., Verrill, C., Rittscher, J.: Improving whole slide segmentation through visual context - a systematic study. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11071, pp. 192–200. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00934-2_22
Velickovic, P., et al.: Graph attention networks. In: International Conference on Learning Representations, ICLR (2018)
Wang, J.O.: Weakly supervised prostate TMA classification via graph convolutional networks. arXiv preprint arXiv:1910.13328 (2019)
Wang, M., et al.: Deep graph library: towards efficient and scalable deep learning on graphs. CoRR, vol. abs/1909.01315 (2019)
Xu, K., et al.: How powerful are graph neural networks? In: ICLR (2019)
Zhou, Y., et al.: CGC-net: cell graph convolutional network for grading of colorectal cancer histology images. In: Proceedings of the IEEE ICCV Workshops (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Pati, P. et al. (2020). HACT-Net: A Hierarchical Cell-to-Tissue Graph Neural Network for Histopathological Image Classification. In: Sudre, C.H., et al. Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Graphs in Biomedical Image Analysis. UNSURE GRAIL 2020 2020. Lecture Notes in Computer Science(), vol 12443. Springer, Cham. https://doi.org/10.1007/978-3-030-60365-6_20
Download citation
DOI: https://doi.org/10.1007/978-3-030-60365-6_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-60364-9
Online ISBN: 978-3-030-60365-6
eBook Packages: Computer ScienceComputer Science (R0)