Abstract
Soft-tissue sarcomas are heterogeneous cancers of the mesenchymal lineage that can develop anywhere in the body. A precise prediction of sarcomas patients’ prognosis is critical for clinicians to define an adequate treatment plan. In this paper, we proposed an end-to-end Deep learning framework via Multiple Instance Learning (MIL), Deep Attention-MIL framework, for the survival predictions: Overall survival (OS), Metastasis-free survival (MFS), and Local-recurrence free survival (LRFS) of sarcomas patients, by studying the features from Whole Slide Images (WSIs) of their tumors. The Deep Attention-MIL framework consists of three steps: tiles selection from the WSIs to choose the relevant tiles for the study; tiles feature extraction by using a pre-trained deep learning model; and a Deep Attention-MIL model to predict the risk score for each patient via MIL approach. The risk scores outputted from the Deep Attention-MIL model are used to divide the patients into low/high-risk groups and predict survival time. The framework was trained and validated on a local dataset including 220 patients, then it was used to predict the survival for 48 patients in an external validation dataset. The experiments showed the proposed framework yielded satisfactory and promising results and contributed to accurate cancer survival predictions on both the validation and external testing datasets: By using the WSIs feature only, we obtained an average C-index (of 5-fold cross-validation) of 0.6901, 0.7179, and 0.6211 for OS, MFS, and LRFS tasks on the validation dataset, respectively. On the external testing dataset, these scores are 0.6294, 0.682, and 0.76 for the three tasks (OS, MFS, LRFS), respectively. By adding the clinical features, these scores have been improved both on validation and external testing datasets. We obtained an average C-index of 0.7835/0.6378, 0.7389/0.6885, and 0.6883/0.7272 for the three tasks (OS, MFS, LRFS) on validation/external testing datasets.
O. Saut and F. Le-Loarer—Co-directed the research.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Casali, P.G., et al.: Soft tissue and visceral sarcomas: Esmo-euracan clinical practice guidelines for diagnosis, treatment and follow-up. Ann. Oncol. 29, iv51–iv67 (2018)
Coindre, J.-M., Terrier, P., Bui, N.B., et al.: Prognostic factors in adult patients with locally controlled soft tissue sarcoma: a study of 546 patients from the French federation of cancer centers sarcoma group. Journal of Clinical Oncology 14(3), 869–877 (1996)
Callegaro, D., et al.: Development and external validation of two nomograms to predict overall survival and occurrence of distant metastases in adults after surgical resection of localised soft-tissue sarcomas of the extremities: a retrospective analysis. Lancet Oncol. 17(5), 671–680 (2016)
Herrera, F., et al.: Multiple Instance Learning. Springer, Heidelberg (2016). https://doi.org/10.1007/978-3-319-47759-6
Dietterich, T.G., Lathrop, R.H., Lozano-Pérez, T.: Solving the multiple instance problem with axis-parallel rectangles. Artif. Intell. 89(1), 31–71 (1997)
Ilse, M., Tomczak, J., Welling, W.: Attention-based deep multiple instance learning. In International Conference on Machine Learning, pp. 2127–2136. PMLR (2018)
Yao, J., Zhu, X., Jonnagaddala, J., Hawkins, N., Huang, J.: Whole slide images based cancer survival prediction using attention guided deep multiple instance learning networks. Med. Image Anal. 65, 101789 (2020)
Chopra, S., Hadsell, R., LeCun, Y.: Learning a similarity metric discriminatively, with application to face verification. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), vol. 1, pp. 539–546. IEEE (2005)
MacQueen, J., et al.: Some methods for classification and analysis of multivariate observations. In Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Oakland, CA, USA, vol. 1, pp. 281–297 (1967)
Courtiol, P., et al.: Deep learning-based classification of mesothelioma improves prediction of patient outcome. Nat. Med. 25(10), 1519–1525 (2019)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Coindre, J.M., et al.: Predictive value of grade for metastasis development in the main histologic types of adult soft tissue sarcomas: a study of 1240 patients from the French federation of cancer centers sarcoma group. Cancer: Interdisc. Int. J. Am. Cancer Soc. 91(10), 1914–1926 (2001)
Kaplan, E.L., Meier, P.: Nonparametric estimation from incomplete observations. J. Am. Stat. Assoc. 53, 457–481 (1958)
Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vision (IJCV) 115(3), 211–252 (2015)
Courtiol, P., Tramel, E.W., Sanselme, M., Wainrib, G.: Classification and disease localization in histopathology using only global labels: a weakly-supervised approach. preprint arXiv:1802.02212 (2018)
Rony, J., Belharbi, S., Dolz, J., Ayed, I.B., McCaffrey, L., Granger, E.: Deep weakly-supervised learning methods for classification and localization in histology images: a survey. arXiv preprint arXiv:1909.03354 (2019)
Wang, D., Khosla, A., et al.: Deep learning for identifying metastatic breast cancer. arXiv preprint arXiv:1606.05718 (2016)
Hou, L., Samaras, D., Kurç, T.M., Gao, Y., Davis, J.E., Saltz, J.: Efficient multiple instance convolutional neural networks for gigapixel resolution image classification, vol. 7, pp. 174–182 (2015). preprint arXiv:1504.07947
He, K., Zhang, X., et al.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1026–1034 (2015)
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)
Paszke, A., et al.: Automatic differentiation in pytorch. In: NIPS-W (2017)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv:1412.6980 (2014)
Cox, D.R.: Regression models and life-tables. J. Royal Stat. Soc. Ser. B (Methodological) 34(2):187–202 (1972)
Acknowledgements
This work was supported by grants from the SIRIC Bordeaux and the Fondation Bergonié. This study is carried out in collaboration between the MONC team, Inria Bordeaux Sud-Ouest, and the BRIC U1312.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Le, VL. et al. (2023). A Deep Attention-Multiple Instance Learning Framework to Predict Survival of Soft-Tissue Sarcoma from Whole Slide Images. In: Ali, S., van der Sommen, F., van Eijnatten, M., Papież, B.W., Jin, Y., Kolenbrander, I. (eds) Cancer Prevention Through Early Detection. CaPTion 2023. Lecture Notes in Computer Science, vol 14295. Springer, Cham. https://doi.org/10.1007/978-3-031-45350-2_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-45350-2_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-45349-6
Online ISBN: 978-3-031-45350-2
eBook Packages: Computer ScienceComputer Science (R0)
