Abstract
To detect abnormality from histopathology images in a patch-based convolutional neural network (CNN), spatial context is an important cue. However, whole-slide image (WSI) is characterized by high morphological heterogeneity in the shape and scale of tissues, hence a simple visual span to a larger context may not well capture the information associated with the central patch or disease of interest. In this paper, we propose a Deformable Conditional Random Field (DCRF) model to learn the offsets and weights of neighboring patches in a spatial-adaptive manner. Additionally, rather than using regularly tessellated or overlapped patches, we localize patches with more powerful feature representations by the adaptively adjusted offsets in a WSI. Both the employment of DCRF for better feature extraction from spatial sampling patches, as well as utilization of the auto-generated patches as training input, can achieve performance improvement in the target task. This model is feasible to the widespread annotation strategies in histopathology images, either with a contoured region of interest (ROI) or patch-wise multi-tissue labels. The proposed model is validated on the patient cohorts from The Cancer Genome Atlas (TCGA) dataset and the Camelyon16 dataset for performance evaluation. The experimental results demonstrate the advantage of the proposed model in the classification task, by the comparison against the baseline models.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Zarella, M.D., Bowman, D., et al.: A practical guide to whole slide imaging: A white paper from the digital pathology association. Arch. Pathol. Lab. Med. 143(2), 222–234 (2019)
Kather, J.N., Krisam, J., et al.: Predicting survival from colorectal cancer histology slides using deep learning: A retrospective multicenter study. PLoS Med. 16(1), e1002730 (2019)
Bejnordi, B.E., Veta, M., et al.: Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA 318(22), 2199–2210 (2017)
Li, Y., Ping, W.: Cancer metastasis detection with neural conditional random field (2018). arXiv preprint arXiv:1806.07064
Coudray, N., Ocampo, P.S., et al.: Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat. Med. 24(10), 1559–1567 (2018)
Hou, L., Samaras, D., et al.: Patch-based convolutional neural network for whole slide tissue image classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2424–2433 (2016)
Tokunaga, H., Teramoto, Y., et al.: Adaptive weighting multi-field-of-view CNN for semantic segmentation in pathology. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 12597–12606 (2019)
Kong, B., Wang, X., Li, Z., Song, Q., Zhang, S.: Cancer metastasis detection via spatially structured deep network. In: Niethammer, M., Styner, M., Aylward, S., Zhu, H., Oguz, I., Yap, P.-T., Shen, D. (eds.) IPMI 2017. LNCS, vol. 10265, pp. 236–248. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-59050-9_19
Yan, J., Li, C., Li, Y., et al.: Adaptive discrete hypergraph matching. IEEE Trans. Cybern. 48(2), 765–779 (2017)
Yang, L., Zhang, Y., Chen, et al.: Suggestive annotation: A deep active learning framework for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 399–407 (2017)
Kather, J.N., Pearson, A.T., et al.: Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer. Nat. Med. 25(7), 1054–1056 (2019). https://doi.org/10.1038/s41591-019-0462-y
Zanjani, F.G., Zinger, S.: Cancer detection in histopathology whole-slide images using conditional random fields on deep embedded spaces. In: Medical Imaging: Digital Pathology, vol. 10581 (2018). https://doi.org/10.1117/12.2293107
Li, Y., Ping, W.: Cancer metastasis detection with neural conditional random field (2018). arXiv:1806.07064
Krähenbühl, P., Koltun, V.: Efficient inference in fully connected CRFs with Gaussian edge potentials. In: Advances in Neural Information Processing Systems, pp. 109–117 (2011)
Zheng, S., Jayasumana, S., et al.: Conditional random fields as recurrent neural networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1529–1537 (2015)
The Cancer Genome Atlas (TCGA) dataset. https://portal.gdc.cancer.gov
He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Iandola F., Moskewicz M., Karayev S., et al.: Densenet: Implementing efficient convnet descriptor pyramids (2014). arXiv preprint arXiv:1404.1869
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Shen, Y., Ke, J. (2020). A Deformable CRF Model for Histopathology Whole-Slide Image Classification. In: Martel, A.L., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science(), vol 12265. Springer, Cham. https://doi.org/10.1007/978-3-030-59722-1_48
Download citation
DOI: https://doi.org/10.1007/978-3-030-59722-1_48
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59721-4
Online ISBN: 978-3-030-59722-1
eBook Packages: Computer ScienceComputer Science (R0)
