Abstract
We apply an ensemble of modified TransBTS, nnU-Net, and a combination of both for the segmentation task of the BraTS 2021 challenge. We change the original architecture of the TransBTS model by adding Squeeze-and-Excitation blocks, increasing the number of CNN layers, replacing positional encoding in the Transformer block with a learnable Multilayer Perceptron (MLP) embeddings, which makes Transformer adjustable to any input size during inference. With these modifications, we can improve TransBTS performance largely. Inspired by a nnU-Net framework, we decided to combine it with our modified TransBTS by changing the architecture inside nnU-Net to our custom model. On the Validation set of BraTS 2021, the ensemble of these approaches achieves 0.8496, 0.8698, 0.9256 Dice score and 15.72, 11.057, 3.374 HD95 for enhancing tumor, tumor core, and whole tumor, correspondingly. On test set we get Dice score 0.8789, 0.8759, 0.9279, and HD95: 10.426, 17.203, 4.93. Our code is publicly available. (Implementation is available at https://github.com/ucuapps/BraTS2021_Challenge).
M. Dobko, D.-I. Kolinko, O. Viniavskyi and Y. Yelisieiev—These authors contributed equally to the work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Baid, U., et al.: The RSNA-ASNR-MICCAI brats 2021 benchmark on brain tumor segmentation and radiogenomic classification. arXiv preprint arXiv:2107.02314 (2021)
Bakas, S., et al.: Segmentation labels for the pre-operative scans of the TCGA-GBM collection (2017). https://doi.org/10.7937/K9/TCIA.2017.KLXWJJ1Q, https://wiki.cancerimagingarchive.net/x/KoZyAQ
Bakas, S., et al.: Segmentation labels for the pre-operative scans of the TCGA-LGG collection (2017). https://doi.org/10.7937/K9/TCIA.2017.GJQ7R0EF, https://wiki.cancerimagingarchive.net/x/LIZyAQ
Bakas, S., et al.: Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features. Nat. Sci. Data 4(1) (2017). https://doi.org/10.1038/sdata.2017.117
Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, June 2018. https://doi.org/10.1109/cvpr.2018.00745
Isensee, F., Jaeger, P.F., Kohl, S.A.A., Petersen, J., Maier-Hein, K.H.: nnU-net: a self-configuring method for deep learning-based biomedical image segmentation. Nature Methods 18(2), 203–211 (2020). https://doi.org/10.1038/s41592-020-01008-z
Isensee, F., Jäger, P.F., Full, P.M., Vollmuth, P., Maier-Hein, K.H.: nnU-Net for brain tumor segmentation. In: Crimi, A., Bakas, S. (eds.) BrainLes 2020. LNCS, vol. 12659, pp. 118–132. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-72087-2_11
Isensee, F., et al.: Abstract: nnU-Net: self-adapting framework for u-net-based medical image segmentation. In: Bildverarbeitung für die Medizin 2019. I, pp. 22–22. Springer, Wiesbaden (2019). https://doi.org/10.1007/978-3-658-25326-4_7
Jieneng, C., et al.: Transunet: transformers make strong encoders for medical image segmentation. arXiv preprint arXiv:2102.04306 (2021)
Karimi, D., Salcudean, S.E.: Reducing the hausdorff distance in medical image segmentation with convolutional neural networks. IEEE Trans. Med. Imaging 39(2), 499–513 (2020). https://doi.org/10.1109/tmi.2019.2930068
Menze, B.H., et al.: The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans. Med. Imaging 34(10), 1993–2024 (2015). https://doi.org/10.1109/tmi.2014.2377694
Micikevicius, P., et al.: Mixed precision training. CoRR abs/1710.03740 (2017). http://arxiv.org/abs/1710.03740
Paszke, A., et al.: Pytorch: an imperative style, high-performance deep learning library, pp. 8024–8035 (2019). http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf
Wang, Wenxuan, Chen, Chen, Ding, Meng, Yu, Hong, Zha, Sen, Li, Jiangyun: TransBTS: multimodal brain tumor segmentation using transformer. In: de Bruijne, Marleen, et al. (eds.) MICCAI 2021. LNCS, vol. 12901, pp. 109–119. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87193-2_11
Wang, Y., et al.: Modality-pairing learning for brain tumor segmentation. In: Crimi, A., Bakas, S. (eds.) BrainLes 2020. LNCS, vol. 12658, pp. 230–240. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-72084-1_21
Acknowledgements
Authors thank Avenga, Eleks, and Ukrainian Catholic University for providing necessary computing resources. We also express gratitude to Marko Kostiv and Dmytro Fishman for their help and support in the last week of competition.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Dobko, M., Kolinko, DI., Viniavskyi, O., Yelisieiev, Y. (2022). Combining CNNs with Transformer for Multimodal 3D MRI Brain Tumor Segmentation. In: Crimi, A., Bakas, S. (eds) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. BrainLes 2021. Lecture Notes in Computer Science, vol 12963. Springer, Cham. https://doi.org/10.1007/978-3-031-09002-8_21
Download citation
DOI: https://doi.org/10.1007/978-3-031-09002-8_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-09001-1
Online ISBN: 978-3-031-09002-8
eBook Packages: Computer ScienceComputer Science (R0)
