Skip to main content

RNN-Based Multiple Instance Learning for the Classification of Histopathology Whole Slide Images

  • Conference paper
  • First Online:
Proceedings of 2023 International Conference on Medical Imaging and Computer-Aided Diagnosis (MICAD 2023) (MICAD 2023)

Part of the book series: Lecture Notes in Electrical Engineering ((LNEE,volume 1166))

  • 473 Accesses

Abstract

The digitization of histopathology Whole Slide Images (WSIs) enables deep learning to be applied in pathology research and medical computer-aided diagnosis. Due to the high-resolution of WSIs, multiple-instance learning (MIL) has become a powerful tool for addressing weakly supervised WSI classification tasks. However, most of the existing MIL approaches ignore the dependency of instances, limiting the adequate extraction of global WSI representation. Further, MIL approaches are often based on attention mechanisms, usually with huge time and space complexity. To tackle these problems, we propose an RWKV-based MIL, adapted from an RNN-like NLP model. Its instance mixing mechanism can model the long-range dependency of instances. And it can work parallelly with lower time and space complexity. As a result, our method can efficiently learn global representations from high-resolution WSIs. Compared with the original RWKV, we propose various new designs to achieve the best performance in WSI classification. Experiments and ablation studies are conducted on two public WSI datasets with four baselines to demonstrate the better performance of our RWKV-based MIL. The AUC results of our proposed method on CMELYON16 and TCGA-BRCA are 0.7964 and 0.8971, gaining an improvement of 7.7% and 0.8% over existing methods, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Cornish, T.C., Swapp, R.E., Kaplan, K.J.: Whole-slide imaging: routine pathologic diagnosis. Adv. Anat. Pathol. 19(3), 152–159 (2012)

    Google Scholar 

  2. Pantanowitz, L., et al.: Review of the current state of whole slide imaging in pathology. J. Pathol. Inform. 2(1), 36 (2011)

    Google Scholar 

  3. Madabhushi, A.: Digital pathology image analysis: opportunities and challenges. Imaging  Med 1(1), 7 (2009)

    Google Scholar 

  4. Campanella, G., et al.: Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat. Med. 25(8), 1301–1309 (2019)

    Google Scholar 

  5. Litjens, G., et al.: A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60–88 (2017)

    Google Scholar 

  6. Li, B., Keikhosravi, A., Loeffler, A.G., Eliceiri, K.W.: Single image super-resolution for whole slide image using convolutional neural networks and self-supervised color normalization. Med. Image Anal. 68, 101938 (2021). https://doi.org/10.1016/j.media.2020.101938

  7. Sirinukunwattana, K., et al.: Gland segmentation in colon histology images: The glas challenge contest. Med. Image Anal. 35, 489–502 (2017)

    Google Scholar 

  8. Wang, X., et al.: Weakly supervised deep learning for whole slide lung cancer image analysis. IEEE Trans. Cybernet. 50(9), 3950–3962 (2019)

    Google Scholar 

  9. Srinidhi, C.L., Ciga, O., Martel, A.L.: Deep neural network models for computational histopathology: A survey. Med. Image Anal. 67, 101813 (2021)

    Google Scholar 

  10. Dietterich, T.G., Lathrop, R.H., Lozano-Pérez, T.: Solving the multiple instance problem with axis-parallel rectangles. Artif. Intell. 89(1–2), 31–71 (1997)

    Article  Google Scholar 

  11. Ilse, M., Tomczak, J.,  Welling, M.: Attention-based deep multiple instance learning. In: International Conference on Machine Learning, pp. 2127–2136. PMLR (July 2018)

    Google Scholar 

  12. Pham, A.T., Raich, R., Fern, X.Z., Wong, W.K., Guan, X.: Discriminative probabilistic framework for generalized multi-instance learning. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2281–2285. IEEE (April 2018)

    Google Scholar 

  13. Liu, G., Wu, J.,  Zhou, Z.H:.  Key instance detection in multi-instance learning. In: Asian Conference on Machine Learning, pp. 253–268. PMLR (November 2012)

    Google Scholar 

  14. Ramon, J.,  De Raedt, L.: Multi instance neural networks. In: Proceedings of the ICML-2000 Workshop on Attribute-value and Relational Learning, pp. 53–60  (2000)

    Google Scholar 

  15. Maron, O.,  Lozano-Pérez, T.: A framework for multiple-instance learning. Adv. Neural Inform. Proc. Syst. 10 (1997)

    Google Scholar 

  16. Wang, X., Yan, Y., Tang, P., Bai, X., Liu, W.: Revisiting multiple instance neural networks. Pattern Recogn. 74, 15–24 (2018)

    Article  Google Scholar 

  17. Kanavati, F., et al.: Weakly-supervised learning for lung carcinoma classification using deep learning. Sci. Rep. 10(1), 9297 (2020)

    Google Scholar 

  18. Li, B., Li, Y., Eliceiri, K.W.: Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14318–14328 (2021)

    Google Scholar 

  19. Tu, M., Huang, J., He, X.,  Zhou, B.:  Multiple instance learning with graph neural networks (2019). arXiv preprint arXiv:1906.04881

  20. Shao, Z., Bian, H., Chen, Y., Wang, Y., Zhang, J., Ji, X.: Transmil: transformer based correlated multiple instance learning for whole slide image classification. Adv. Neural. Inf. Process. Syst. 34, 2136–2147 (2021)

    Google Scholar 

  21. Vaswani, A., et al.:  Attention is all you need. Adv. Neural. Inf. Process. Syst. 30 (2017)

    Google Scholar 

  22. Zhai, S., et al.: An attention free transformer. arXiv preprint arXiv:2105.14103 (2021)

  23. Peng, B., et al.:  RWKV: Reinventing RNNs for the Transformer Era. arXiv preprint arXiv:2305.13048 (2023)

  24. Zaheer, M., Kottur, S., Ravanbakhsh, S., Poczos, B., Salakhutdinov, R. R., & Smola, A. J. (2017). Deep sets. Advances in neural information processing systems, 30

    Google Scholar 

  25. Xu, G., et al.:  Camel: a weakly supervised learning framework for histopathology image segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10682–10691 (2019)

    Google Scholar 

  26. Lerousseau, M., Vakalopoulou, M., Classe, M., Adam, J., Battistella, E., Carré, A., Estienne, T., Henry, T., Deutsch, E., Paragios, N.: Weakly supervised multiple instance learning histopathological tumor segmentation. In: Martel, A.L., Abolmaesumi, P., Stoyanov, D., Mateus, D., Zuluaga, M.A., Kevin Zhou, S., Racoceanu, D., Joskowicz, L. (eds.) MICCAI 2020. LNCS, vol. 12265, pp. 470–479. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59722-1_45

  27. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Google Scholar 

  28. Cho, K., et al.:  Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014)

  29. Bradbury, J., Merity, S., Xiong, C.,  Socher, R.: Quasi-recurrent neural networks. arXiv preprint arXiv:1611.01576 (2016)

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Gaoyuan Ji .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ji, G., Liu, P. (2024). RNN-Based Multiple Instance Learning for the Classification of Histopathology Whole Slide Images. In: Su, R., Zhang, YD., Frangi, A.F. (eds) Proceedings of 2023 International Conference on Medical Imaging and Computer-Aided Diagnosis (MICAD 2023). MICAD 2023. Lecture Notes in Electrical Engineering, vol 1166. Springer, Singapore. https://doi.org/10.1007/978-981-97-1335-6_29

Download citation

  • DOI: https://doi.org/10.1007/978-981-97-1335-6_29

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-97-1334-9

  • Online ISBN: 978-981-97-1335-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics