Abstract
Telepathology has enabled the remote cancer diagnosis based on digital pathological whole slide images (WSIs). During the diagnosis, the behavior information of the pathologist can be recorded by the platform and then archived with the digital cases. The diagnosis path of the pathologist on a WSI is valuable information since the image content within the path is highly correlated with the diagnosis report of the pathologist. In this paper, we proposed a novel diagnosis path network (DPathNet). DPathNet utilizes the diagnosis paths of pathologists on the WSIs as the supervision to learn the pathology knowledge from the image content. Based on the DPathNet, we develop a novel approach for computer-aided cancer diagnosis named session-based histopathology image recommendation (SHIR). SHIR summaries the information of a WSI while the pathologist browsing the WSI and actively recommends the relevant cases within similar image content from the database. The proposed approaches are evaluated on a gastric dataset containing 983 cases within 5 categories of gastric lesions. The experimental results have demonstrated the effectiveness of the DPathNet to the SHIR task and the supervision of the diagnosis path is sufficient to train the DPathNet. The MRR and MAP of the proposed SHIR framework are respectively 0.741 and 0.777 on the gastric dataset.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
The blank patches are filtered in the construction of the graph.
- 2.
References
Bejnordi, B.E., Veta, M., Van Diest, P.J., Van Ginneken, B., Karssemeijer, N., Litjens, G., Van Der Laak, et al.: Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA 318(22), 2199–2210 (2017)
Campanella, G., et al.: Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat. Med. 25(8), 1301–1309 (2019)
Cho, K., van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., Bengio, Y.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1724–1734 (2014)
Hollon, T.C., Pandian, B., Adapa, A.R., Urias, E., Save, A.V., Khalsa, S.S.S., Eichberg, D.G., D’Amico, R.S., Farooq, Z.U., Lewis, S., et al.: Near real-time intraoperative brain tumor diagnosis using stimulated raman histology and deep neural networks. Nat. Med. 26(1), 52–58 (2020)
Hu, D., Zheng, Y., Zhang, H., Sun, S., Xie, F., Shi, J., Jiang, Z.: Informative retrieval framework for histopathology whole slides images based on deep hashing network. In: 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), pp. 244–248 (2020)
Kalra, S., Tizhoosh, H., Choi, C., Shah, S., Diamandis, P., Campbell, C.J., Pantanowitz, L.: Yottixel - An image search engine for large archives of histopathology whole slide images. Med. Image Anal. 65, 101757 (2020)
Van der Laak, J., Ciompi, F., Litjens, G.: No pixel-level annotations needed. Nat. Biomed. Eng. 3(11), 855–856 (2019)
Li, R., Yao, J., Zhu, X., Li, Y., Huang, J.: Graph CNN for survival analysis on whole slide pathological images. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11071, pp. 174–182. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00934-2_20
Li, Z., Zhang, X., Müller, H., Zhang, S.: Large-scale retrieval for medical image analytics: A comprehensive review. Med. Image Anal. 43, 66–84 (2018)
Liu, Q., Zeng, Y., Mokhosi, R., Zhang, H.: Stamp: Short-term attention/memory priority model for session-based recommendation. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1831–1839 (2018)
Peng, T., Boxberg, M., Weichert, W., Navab, N., Marr, C.: Multi-task learning of a deep k-nearest neighbour network for histopathological image classification and retrieval. In: Shen, D., Liu, T., Peters, T.M., Staib, L.H., Essert, C., Zhou, S., Yap, P.-T., Khan, A. (eds.) MICCAI 2019. LNCS, vol. 11764, pp. 676–684. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32239-7_75
Sapkota, M., Shi, X., Xing, F., Yang, L.: Deep convolutional hashing for low-dimensional binary embedding of histopathological images. IEEE J. Biomed. Health Inform. 23(2), 805–816 (2018)
Shi, X., Sapkota, M., Xing, F., Liu, F., Cui, L., Yang, L.: Pairwise based deep ranking hashing for histopathology image classification and retrieval. Pattern Recogn. 81, 14–22 (2018)
Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019)
Jimenez-del-Toro, O., Otálora, S., Atzori, M., Müller, H.: Deep multimodal case–based retrieval for large histopathology datasets. In: Wu, G., Munsell, B.C., Zhan, Y., Bai, W., Sanroma, G., Coupé, P. (eds.) Patch-MI 2017. LNCS, vol. 10530, pp. 149–157. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67434-6_17
Wang, X., Shi, Y., Kitani, K.M.: Deep supervised hashing with triplet labels. In: Lai, S.-H., Lepetit, V., Nishino, K., Sato, Y. (eds.) ACCV 2016. LNCS, vol. 10111, pp. 70–84. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-54181-5_5
Wu, S., Tang, Y., Zhu, Y., Wang, L., Xie, X., Tan, T.: Session-based recommendation with graph neural networks. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 346–353 (2019)
Yan, R., et al.: Breast cancer histopathological image classification using a hybrid deep neural network. Methods 173, 52–60 (2019)
Ying, Z., You, J., Morris, C., Ren, X., Hamilton, W., Leskovec, J.: Hierarchical graph representation learning with differentiable pooling. In: Advances in Neural Information Processing Systems, pp. 4800–4810 (2018)
Zheng, Y., Jiang, B., Shi, J., Zhang, H., Xie, F.: Encoding histopathological WSIs using GNN for scalable diagnostically relevant regions retrieval. In: Shen, D., Liu, T., Peters, T.M., Staib, L.H., Essert, C., Zhou, S., Yap, P.-T., Khan, A. (eds.) MICCAI 2019. LNCS, vol. 11764, pp. 550–558. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32239-7_61
Zheng, Y., et al.: Histopathological whole slide image analysis using context-based CBIR. IEEE Trans. Med. Imag. 37(7), 1641–1652 (2018)
Zheng, Y., et al.: Size-scalable content-based histopathological image retrieval from database that consists of WSIS. IEEE J. Biomed. Health Inform. 22(4), 1278–1287 (2018)
Acknowledgment
This work was partly supported by the National Natural Science Foundation of China (Grant No. 61901018, 61771031, and 61906058), partly by China Postdoctoral Science Foundation (No. 2019M650446) and partly by Motic-BUAA Image Technology Research Center.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Zheng, Y., Jiang, Z., Zhang, H., Xie, F., Shi, J. (2020). Tracing Diagnosis Paths on Histopathology WSIs for Diagnostically Relevant Case Recommendation. In: Martel, A.L., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science(), vol 12265. Springer, Cham. https://doi.org/10.1007/978-3-030-59722-1_44
Download citation
DOI: https://doi.org/10.1007/978-3-030-59722-1_44
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59721-4
Online ISBN: 978-3-030-59722-1
eBook Packages: Computer ScienceComputer Science (R0)
