Abstract
The deep learning methods supervised by annotating different regions of histopathology images (patch-level labels) have achieved promising outcomes in assisting pathologic diagnosis. However, most clinical data only contains label information for the whole slide image (WSI-level labels), so the methods supervised by WSI-level labels are more necessary than the ones supervised by patch-level labels. Additionally, various methods supervised by WSI-level labels ignore the contextual relations among patches extracted from a WSI, making incorrect predictions for some patches in a WSI and further misclassifying the WSI. In this paper, we propose to utilize an interpretable dual encoder network with a context-capturing RNN module to capture the contextual relations among all patches extracted from a WSI. Besides, we propose to utilize a feature attention module to weigh the importance of each patch automatically. More importantly, visualization of weight for each patch in a WSI demonstrates that our approach matches the concerns of pathologists. Furthermore, extensive experiments demonstrate the superiority of the interpretable dual encoder network.
J. Feng and L. Yang—Joint corresponding authors.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Li, Y., Chen, P., Li, Z., Su, H., Yang, L., Zhong, D.: Rule-based automatic diagnosis of thyroid nodules from intraoperative frozen sections using deep learning. Artif. Intell. Med. 108, 101918 (2020)
Srinidhi, C.L., Ciga, O., Martel, A.L.: Deep neural network models for computational histopathology: a survey. Medical Image Analysis, p. 101813 (2020)
Kong, B., Wang, X., Li, Z., Song, Q., Zhang, S.: Cancer metastasis detection via spatially structured deep network. In: International Conference on Information Processing in Medical Imaging, pp. 236–248. Springer (2017). https://doi.org/10.1007/978-3-319-59050-9_19
Zanjani, F.G., Zinger, S., et al.: Cancer detection in histopathology whole-slide images using conditional random fields on deep embedded spaces. In: Medical imaging 2018: Digital pathology, vol. 10581. International Society for Optics and Photonics, p. 105810I (2018)
Li, Y., Ping, W.: Cancer metastasis detection with neural conditional random field. arXiv:1806.07064 (2018)
Zhang, Z., et al.: Pathologist-level interpretable whole-slide cancer diagnosis with deep learning. Nat. Mach. Intell. 1(5), 236–245 (2019)
Hou, L., Samaras, D., Kurc, T.M., Gao, Y., Davis, J.E., Saltz, J.H.: Patch-based convolutional neural network for whole slide tissue image classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2424–2433 (2016)
Zhu, X., Yao, J., Zhu, F., Huang, J.: Wsisa: making survival prediction from whole slide histopathological images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7234–7242 (2017)
Wang, X., et al.: Weakly supervised deep learning for whole slide lung cancer image analysis. IEEE Trans. Cybern. 50(9), 3950–3962 (2019)
Chen, H., et al.: Rectified cross-entropy and upper transition loss for weakly supervised whole slide image classifier. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 351–359. Springer (2019). https://doi.org/10.1007/978-3-030-32239-7_39
Campanella, G., et al.: Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat. Med. 25(8), 1301–1309 (2019)
Zhou, P., et al.: Attention-based bidirectional long short-term memory networks for relation classification. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (volume 2: Short papers), pp. 207–212 (2016)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Zhang, S., Zheng, D., Hu, X., Yang, M.: Bidirectional long short-term memory networks for relation classification. In: Proceedings of the 29th Pacific Asia Conference on Language, Information and Computation, pp. 73–78 (2015)
Acknowledgment
This work was funded by the Natural Science Foundation of Shaanxi Province of China(2021JQ-461).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Wu, Z. et al. (2021). Interpretable Histopathology Image Diagnosis via Whole Tissue Slide Level Supervision. In: Lian, C., Cao, X., Rekik, I., Xu, X., Yan, P. (eds) Machine Learning in Medical Imaging. MLMI 2021. Lecture Notes in Computer Science(), vol 12966. Springer, Cham. https://doi.org/10.1007/978-3-030-87589-3_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-87589-3_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87588-6
Online ISBN: 978-3-030-87589-3
eBook Packages: Computer ScienceComputer Science (R0)
