Abstract
Segmentation of gliomas into distinct sub-regions can help guide clinicians in tasks such as surgical planning, prognosis, and treatment response assessment. Manual delineation is time-consuming and prone to inter-rater variability. In this work, we propose a deep learning based automatic segmentation method that takes T1-pre, T1-post, T2, and FLAIR MRI as input and outputs a segmentation map of the sub-regions of interest (enhancing tumor (ET), whole tumor (WT), and tumor core (TC)). Our U-Net based architecture incorporates a modified selective kernel block to enable the network to adjust its receptive field via an attention mechanism, enabling more robust segmentation of gliomas of all appearances, shapes, and scales. Using this approach on the official BraTS 2020 testing set, we obtain Dice scores of .822, .889, and .834, and Hausdorff distances (95%) of 11.588, 4.812, and 21.984 for ET, WT, and TC, respectively. For prediction of overall survival, we extract deep features from the bottleneck layer of this network and train a Cox Proportional Hazards model, obtaining .495 accuracy. For uncertainty prediction, we achieve AUCs of .850, .914, and .854 for ET, WT, and TC, respectively, which earned us third place for this task.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abadi, M., et al.: TensorFlow: a system for large-scale machine learning. In: 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2016), pp. 265–283 (2016). https://www.usenix.org/system/files/conference/osdi16/osdi16-abadi.pdf
Bakas, S., et al.: Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features. Sci. Data 4, 170117 (2017). https://doi.org/10.1038/sdata.2017.117
Bakas, S., et al.: Segmentation labels and radiomic features for the pre-operative scans of the TCGA-LGG collection [Data Set]. Cancer Imaging Arch. (2017). https://doi.org/10.7937/K9/TCIA.2017.GJQ7R0EF
Bakas, S., et al.: Segmentation labels for the pre-operative scans of the TCGA-GBM collection [Data set]. Cancer Imaging Arch. (2017). https://doi.org/10.7937/K9/TCIA.2017.KLXWJJ1Q
Bakas, S., et al.: Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge (2018)
Beers, A., et al.: High-resolution medical image synthesis using progressively grown generative adversarial networks (2018)
Beers, A., et al.: DeepNeuro: an open-source deep learning toolbox for neuroimaging. Neuroinformatics (2020). https://doi.org/10.1007/s12021-020-09477-5
Beers, A., Chang, K., Brown, J., Gerstner, E., Rosen, B., Kalpathy-Cramer, J.: Sequential neural networks for biologically informed glioma segmentation. In: Angelini, E.D., Landman, B.A. (eds.) Medical Imaging 2018: Image Processing, vol. 10574, pp. 807–816. International Society for Optics and Photonics, SPIE (2018). https://doi.org/10.1117/12.2293941
Beig, N., et al.: Radiogenomic analysis of hypoxia pathway is predictive of overall survival in Glioblastoma. Sci. Rep. 8(1), 7 (2018). https://doi.org/10.1038/s41598-017-18310-0
Chang, K., et al.: Residual convolutional neural network for the determination of IDH status in low- and high-grade gliomas from MR imaging. Clin. Cancer Res. 24(5), 1073–1081 (2018). https://doi.org/10.1158/1078-0432.CCR-17-2236, https://clincancerres.aacrjournals.org/content/24/5/1073
Chang, K., et al.: Automatic assessment of glioma burden: a deep learning algorithm for fully automated volumetric and bidimensional measurement. Neuro-Oncology 21(11), 1412–1422 (2019). https://doi.org/10.1093/neuonc/noz106
Eckel-Passow, J.E., et al.: Glioma groups based on 1p/19q, IDH, and TERT promoter mutations in tumors. New Engl. J. Med. 372(26), 2499–2508 (2015). https://doi.org/10.1056/NEJMoa1407279, pMID: 26061753
Gerstner, E.R., et al.: Bevacizumab reduces permeability and concurrent temozolomide delivery in a subset of patients with recurrent glioblastoma. Clin. Cancer Res. 26(1), 206–212 (2020). https://doi.org/10.1158/1078-0432.CCR-19-1739, https://clincancerres.aacrjournals.org/content/26/1/206
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016)
Hendrycks, D., Gimpel, K.: A baseline for detecting misclassified and out-of-distribution examples in neural networks (2018)
Ioffe, S., Szegedy, C.: Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift (2015)
Isensee, F., Kickingereder, P., Wick, W., Bendszus, M., Maier-Hein, K.H.: No new-net. CoRR abs/1809.10483 (2018), http://arxiv.org/abs/1809.10483
Jiang, Z., Ding, C., Liu, M., Tao, D.: Two-stage cascaded u-net: 1st place solution to brats challenge 2019 segmentation task. In: Crimi, A., Bakas, S. (eds.) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, pp. 231–241. Springer International Publishing, Cham (2020). https://doi.org/10.1007/978-3-030-11726-9
Kayalibay, B., Jensen, G., van der Smagt, P.: CNN-based segmentation of medical imaging data (2017)
Lao, J., et al.: A deep learning-based radiomics model for prediction of survival in glioblastoma multiforme. Sci. Rep. 7(1), 10353 (2017). https://doi.org/10.1038/s41598-017-10649-8
Li, X., Wang, W., Hu, X., Yang, J.: Selective Kernel Networks (2019)
Loshchilov, I., Hutter, F.: Decoupled Weight Decay Regularization (2017)
Mazzara, G.P., Velthuizen, R.P., Pearlman, J.L., Greenberg, H.M., Wagner, H.: Brain tumor target volume determination for radiation treatment planning through automated MRI segmentation. Int. J. Radiat. Oncol. Biol. Phys. 59(1), 300–312 (2004). https://doi.org/10.1016/j.ijrobp.2004.01.026
Menze, B.H., et al.: The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) 34(10), 1993–2024 (2015). https://doi.org/10.1109/TMI.2014.2377694
Milletari, F., Navab, N., Ahmadi, S.A.: V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation (2016)
Myronenko, A.: 3D MRI brain tumor segmentation using autoencoder regularization (2018)
Nair, V., Hinton, G.: Rectified linear units improve restricted Boltzmann machines. In: Proceedings of ICML, vol. 27, pp. 807–814 (2010)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: Convolutional Networks for Biomedical Image Segmentation. CoRR abs/1505.0 (2015), http://arxiv.org/abs/1505.04597
Tustison, N.J., et al.: N4ITK: improved N3 bias correction. IEEE Trans. Med. Imaging 29(6), 1310–1320 (2010). https://doi.org/10.1109/TMI.2010.2046908, https://www.ncbi.nlm.nih.gov/pubmed/20378467
Woo, S., Park, J., Lee, J.Y., Kweon, I.S.: CBAM: Convolutional Block Attention Module (2018)
Wu, Y., He, K.: Group Normalization (2018)
Zhao, Y., Zhang, Y.M., Liu, C.L.: Bag of Tricks for 3D MRI Brain Tumor Segmentation, pp. 210–220, May 2020. https://doi.org/10.1007/978-3-030-46640-4_20
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Patel, J. et al. (2021). Segmentation, Survival Prediction, and Uncertainty Estimation of Gliomas from Multimodal 3D MRI Using Selective Kernel Networks. In: Crimi, A., Bakas, S. (eds) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. BrainLes 2020. Lecture Notes in Computer Science(), vol 12659. Springer, Cham. https://doi.org/10.1007/978-3-030-72087-2_20
Download citation
DOI: https://doi.org/10.1007/978-3-030-72087-2_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-72086-5
Online ISBN: 978-3-030-72087-2
eBook Packages: Computer ScienceComputer Science (R0)
