Abstract
One challenge of training deep neural networks with gigapixel whole-slide images (WSIs) is the lack of annotation at pixel level or patch (instance) level due to the high cost and time-consuming labeling effort. Multiple instance learning (MIL) as a typical weakly supervised learning method aimed to resolve this challenge by using only the slide-level label without needing patch labels. Not all patches/instances are predictive of the outcome. The attention-based MIL method leverages this fact to enhance the performance by weighting the instances based on their contribution to predicting the outcome. A WSI typically contains hundreds of thousands of image patches. Training a deep neural network with thousands of image patches per slide is computationally expensive and requires a lot of time for convergence. One way to alleviate this issue is to sample a subset of instances/patches from the available instances within each bag for training. While the benefit of sampling strategies for decreasing computing time might be evident, there is a lack of effort to investigate their performances. This project proposes and compares an adaptive sampling strategy with other sampling strategies. Although all sampling strategies substantially reduce computation time, their performance is influenced by the number of selected instances. We show that if we are limited to only select a few instances (e.g., in order of 1\(\sim \)10 instances), the adaptive sampling outperforms other sampling strategies. However, if we are allowed to select more instances (e.g., in order of 100\(\sim \)1000 instances), the random sampling outperforms other sampling strategies.
Genentech Personalized Healthcare Digital Pathology Program.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Fraggetta, F., Garozzo, S., Zannoni, G.F., Pantanowitz, L., Rossi, E.D.: Routine digital pathology workflow: the Catania experience. J. Pathol. Inform. 8(51), 1–6 (2017)
Epstein, J.I.: An update of the Gleason grading system. J. Urol. 183(2), 433–440 (2010)
Otálora, S., Marini, N., Müller, H., Atzori, M.: Combining weakly and strongly supervised learning improves strong supervision in Gleason pattern classification. BMC Med. Imaging 21(77), 1–14 (2021)
Brunyé, T.T., Mercan, E., Weaver, D.L., Elmore, J.G.: Accuracy is in the eyes of the pathologist: the visual interpretive process and diagnostic accuracy with digital whole slide images. J. Biomed. Info. 66, 171–179 (2010)
Tarkhan, A., Simon, N., Bengtsson, T., Nguyen, K., Dai, J.: Survival prediction using deep learning. In: Proceedings of AAAI Spring Symposium on Survival Prediction - Algorithms, Challenges, and Applications 2021. Proceedings of Machine Learning Research, vol. 146, pp. 207–214. PMLR, 22–24 Mar 2021
Cui, M., Zhang, D.Y.: Artificial intelligence and computational pathology. Lab. Invest. 101, 412–422 (2021)
Quellec, G., Cazuguel, G., Cochener, B., Lamard, M.: Multiple-instance learning for medical image and video analysis. IEEE Rev. Biomed. Eng. 10, 213–234 (2017)
Dietterich, T.G., Lathrop, R.H., Lozano-Pérez, T.: Solving the multiple instance problem with axis-parallel rectangles. Artif. Intell. 89(1), 31–71 (1997)
Maron, O., Lozano-Pérez, T.: A framework for multiple-instance learning. In: Jordan, M., Kearns, M., Solla, S. (eds.) Advances in Neural Information Processing Systems, vol. 10. MIT Press (1998)
Liu, G., Wu, J., Zhou, Z.-H.: Key instance detection in multi-instance learning. In: Proceedings of the Asian Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 25, pp. 253–268. PMLR (2012)
Raffel, C., Ellis, D.P.W.: Feed-forward networks with attention can solve some long-term memory problems (2016)
Ramon, J., Raedt, L.D.: Multi instance neural networks. In: ICML Workshop on Attribute-Value and Relational Learning, pp. 53–60 (2000)
Ilse, M., Tomczak, J.M., Welling, M.: Attention-based deep multiple instance learning (2018)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition (2015)
Zhu, X., Yao, J., Huang, J.: Deep convolutional neural network for survival analysis with pathological images. In: 2016 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 544–547 (2016)
Wulczyn, E.: Deep learning-based survival prediction for multiple cancer types using histopathology images. PLOS ONE 15(6), e0233678 (2020)
Li, R., Yao, J., Zhu, X., Li, Y., Huang, J.: Graph CNN for survival analysis on whole slide pathological images. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11071, pp. 174–182. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00934-2_20
Lu, M.Y., Williamson, D.F.K., Chen, T.Y., Chen, R.J., Barbieri, M., Mahmood, F.: Data-efficient and weakly supervised computational pathology on whole-slide images. Nat. Biomed. Eng. 5, 555–570 (2021)
Lu, M.Y., Chen, R.J., Wang, J., Dillon, D., Mahmood, F.: Semi-supervised histology classification using deep multiple instance learning and contrastive predictive coding (2019)
Dehaene, O., Camara, A., Moindrot, O., de Lavergne, A., Courtiol, P.: Self-supervision closes the gap between weak and strong supervision in histology (2020)
Katharopoulos, A., Fleuret, F.: Processing megapixel images with deep attention-sampling models (2019)
Tarkhan, A., Nguyen, T.K., Simon, N., Bengtsson, T., Ocampo, P., Dai, J.: Attention-based deep multiple instance learning with adaptive instance sampling. In: 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI), pp. 1–5 (2022)
Campanella, G., et al.: Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat. Med. 25, 1–9 (2019)
Sharmay, Y., Ehsany, L., Syed, S., Brown, D.E.: HistoTransfer: understanding transfer learning for histopathology. In: 2021 IEEE EMBS International Conference on Biomedical and Health Informatics (BHI), pp. 1–4 (2021)
Zuley, M.L., et la.: Radiology data from the cancer genome atlas prostate adenocarcinoma [TCGA-PRAD] collection. Cancer Imaging Arch (2016)
Gleason, D.F., Mellinger, G.T.: Prediction of prognosis for prostatic adenocarcinoma by combined histological grading and clinical staging. J. Urol. 111(1), 58–64 (1974)
NCCN: NCCN guidelines: prostate cancer (version 4.2018) (2018). https://www2.tri-kobe.org/nccn/guideline/archive/urological2018/english/prostate.pdf. Accessed 11 Nov 2021
Bejnordi, B.E.: Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA 318(22), 2199–2210 (2017)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization (2017)
Prechelt, L.: Early stopping — but when? In: Montavon, G., Orr, G.B., Müller, K.-R. (eds.) Neural Networks: Tricks of the Trade. LNCS, vol. 7700, pp. 53–67. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-35289-8_5
Bishop, C.M.: Training with noise is equivalent to Tikhonov regularization. Neural Comput. 7(1), 108–116 (1995)
Tarkhan, A., Simon, N.: bigSurvSGD: big survival data analysis via stochastic gradient descent. eprint arXiv:2003.00116 [math, stat] (2020)
Yao, J., Zhu, X., Jonnagaddala, J., Hawkins, N., Huang, J.: Whole slide images based cancer survival prediction using attention guided deep multiple instance learning networks. Med. Image Anal. 65, 101789 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Tarkhan, A., Nguyen, T.K., Simon, N., Dai, J. (2022). Investigation of Training Multiple Instance Learning Networks with Instance Sampling. In: Xu, X., Li, X., Mahapatra, D., Cheng, L., Petitjean, C., Fu, H. (eds) Resource-Efficient Medical Image Analysis. REMIA 2022. Lecture Notes in Computer Science, vol 13543. Springer, Cham. https://doi.org/10.1007/978-3-031-16876-5_10
Download citation
DOI: https://doi.org/10.1007/978-3-031-16876-5_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16875-8
Online ISBN: 978-3-031-16876-5
eBook Packages: Computer ScienceComputer Science (R0)
