Abstract
The digitization of histopathology Whole Slide Images (WSIs) enables deep learning to be applied in pathology research and medical computer-aided diagnosis. Due to the high-resolution of WSIs, multiple-instance learning (MIL) has become a powerful tool for addressing weakly supervised WSI classification tasks. However, most of the existing MIL approaches ignore the dependency of instances, limiting the adequate extraction of global WSI representation. Further, MIL approaches are often based on attention mechanisms, usually with huge time and space complexity. To tackle these problems, we propose an RWKV-based MIL, adapted from an RNN-like NLP model. Its instance mixing mechanism can model the long-range dependency of instances. And it can work parallelly with lower time and space complexity. As a result, our method can efficiently learn global representations from high-resolution WSIs. Compared with the original RWKV, we propose various new designs to achieve the best performance in WSI classification. Experiments and ablation studies are conducted on two public WSI datasets with four baselines to demonstrate the better performance of our RWKV-based MIL. The AUC results of our proposed method on CMELYON16 and TCGA-BRCA are 0.7964 and 0.8971, gaining an improvement of 7.7% and 0.8% over existing methods, respectively.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Cornish, T.C., Swapp, R.E., Kaplan, K.J.: Whole-slide imaging: routine pathologic diagnosis. Adv. Anat. Pathol. 19(3), 152–159 (2012)
Pantanowitz, L., et al.: Review of the current state of whole slide imaging in pathology. J. Pathol. Inform. 2(1), 36 (2011)
Madabhushi, A.: Digital pathology image analysis: opportunities and challenges. Imaging Med 1(1), 7 (2009)
Campanella, G., et al.: Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat. Med. 25(8), 1301–1309 (2019)
Litjens, G., et al.: A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60–88 (2017)
Li, B., Keikhosravi, A., Loeffler, A.G., Eliceiri, K.W.: Single image super-resolution for whole slide image using convolutional neural networks and self-supervised color normalization. Med. Image Anal. 68, 101938 (2021). https://doi.org/10.1016/j.media.2020.101938
Sirinukunwattana, K., et al.: Gland segmentation in colon histology images: The glas challenge contest. Med. Image Anal. 35, 489–502 (2017)
Wang, X., et al.: Weakly supervised deep learning for whole slide lung cancer image analysis. IEEE Trans. Cybernet. 50(9), 3950–3962 (2019)
Srinidhi, C.L., Ciga, O., Martel, A.L.: Deep neural network models for computational histopathology: A survey. Med. Image Anal. 67, 101813 (2021)
Dietterich, T.G., Lathrop, R.H., Lozano-Pérez, T.: Solving the multiple instance problem with axis-parallel rectangles. Artif. Intell. 89(1–2), 31–71 (1997)
Ilse, M., Tomczak, J., Welling, M.: Attention-based deep multiple instance learning. In: International Conference on Machine Learning, pp. 2127–2136. PMLR (July 2018)
Pham, A.T., Raich, R., Fern, X.Z., Wong, W.K., Guan, X.: Discriminative probabilistic framework for generalized multi-instance learning. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2281–2285. IEEE (April 2018)
Liu, G., Wu, J., Zhou, Z.H:. Key instance detection in multi-instance learning. In: Asian Conference on Machine Learning, pp. 253–268. PMLR (November 2012)
Ramon, J., De Raedt, L.: Multi instance neural networks. In: Proceedings of the ICML-2000 Workshop on Attribute-value and Relational Learning, pp. 53–60 (2000)
Maron, O., Lozano-Pérez, T.: A framework for multiple-instance learning. Adv. Neural Inform. Proc. Syst. 10 (1997)
Wang, X., Yan, Y., Tang, P., Bai, X., Liu, W.: Revisiting multiple instance neural networks. Pattern Recogn. 74, 15–24 (2018)
Kanavati, F., et al.: Weakly-supervised learning for lung carcinoma classification using deep learning. Sci. Rep. 10(1), 9297 (2020)
Li, B., Li, Y., Eliceiri, K.W.: Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14318–14328 (2021)
Tu, M., Huang, J., He, X., Zhou, B.: Multiple instance learning with graph neural networks (2019). arXiv preprint arXiv:1906.04881
Shao, Z., Bian, H., Chen, Y., Wang, Y., Zhang, J., Ji, X.: Transmil: transformer based correlated multiple instance learning for whole slide image classification. Adv. Neural. Inf. Process. Syst. 34, 2136–2147 (2021)
Vaswani, A., et al.: Attention is all you need. Adv. Neural. Inf. Process. Syst. 30 (2017)
Zhai, S., et al.: An attention free transformer. arXiv preprint arXiv:2105.14103 (2021)
Peng, B., et al.: RWKV: Reinventing RNNs for the Transformer Era. arXiv preprint arXiv:2305.13048 (2023)
Zaheer, M., Kottur, S., Ravanbakhsh, S., Poczos, B., Salakhutdinov, R. R., & Smola, A. J. (2017). Deep sets. Advances in neural information processing systems, 30
Xu, G., et al.: Camel: a weakly supervised learning framework for histopathology image segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10682–10691 (2019)
Lerousseau, M., Vakalopoulou, M., Classe, M., Adam, J., Battistella, E., Carré, A., Estienne, T., Henry, T., Deutsch, E., Paragios, N.: Weakly supervised multiple instance learning histopathological tumor segmentation. In: Martel, A.L., Abolmaesumi, P., Stoyanov, D., Mateus, D., Zuluaga, M.A., Kevin Zhou, S., Racoceanu, D., Joskowicz, L. (eds.) MICCAI 2020. LNCS, vol. 12265, pp. 470–479. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59722-1_45
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014)
Bradbury, J., Merity, S., Xiong, C., Socher, R.: Quasi-recurrent neural networks. arXiv preprint arXiv:1611.01576 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Ji, G., Liu, P. (2024). RNN-Based Multiple Instance Learning for the Classification of Histopathology Whole Slide Images. In: Su, R., Zhang, YD., Frangi, A.F. (eds) Proceedings of 2023 International Conference on Medical Imaging and Computer-Aided Diagnosis (MICAD 2023). MICAD 2023. Lecture Notes in Electrical Engineering, vol 1166. Springer, Singapore. https://doi.org/10.1007/978-981-97-1335-6_29
Download citation
DOI: https://doi.org/10.1007/978-981-97-1335-6_29
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-1334-9
Online ISBN: 978-981-97-1335-6
eBook Packages: Computer ScienceComputer Science (R0)