Abstract
Early diagnosis and grading of gliomas are crucial for determining therapy and the prognosis of brain cancer. For this purpose, magnetic resonance (MR) studies of brain tumours are widely used in the therapy process. Due to the overlap between the intensity distributions of healthy, enhanced, non-enhancing, and edematous areas, automated segmentation of tumours is a complicated task. Convolutional neural networks (CNNs) have been utilized as the dominant deep learning method for segmentation tasks. However, they suffer from the inability to capture and learn long-range dependencies and global features due to their limited kernels. Vision transformers (ViTs) were introduced recently to tackle these limitations. Although ViTs are capable of capturing long-range features, their segmentation performance falls as the variety of tumour sizes increases. In this matter, ViT’s patch size plays a significant role in the learning process of a network, and finding an optimal patch size is a challenging and time-consuming task. In this paper, we propose a framework to find the optimal ViT patch size for the brain tumour segmentation task, particularly for segmenting smaller tumours. We validated our proposed framework on the BraTS’21 dataset. Our proposed framework, could improve the segmentation dice performance for 0.97%, 1.14%, and 2.05% for enhancing tumour, tumour core, and whole tumour, respectively, in comparison with default ViT (ViT-base). This research lays the groundwork for future research on the semantic segmentation of tumour segmentation and detection using vision transformer-based networks for optimal outcomes. The implementation source code is available at: https://github.com/Ramtin-Mojtahedi/BRATS_OVTPS.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Banu, Z.: Glioblastoma multiforme: a review of its pathogenesis and treatment. Int. Res. J. Pharm. 9, 7–12 (2019)
Ribalta Lorenzo, P., et al.: Segmenting brain tumors from FLAIR MRI using fully convolutional neural networks. Comput. Methods Programs Biomed. 176, 135–148 (2019)
Soleymanifard, M., Hamghalam, M.: Multi-stage glioma segmentation for tumour grade classification based on multiscale fuzzy C-means. Multimedia Tools Appl. 81, 8451–8470 (2022)
Hamghalam, M., Lei, B., Wang, T.: Brain tumor synthetic segmentation in 3D multimodal MRI scans. In: Crimi, A., Bakas, S. (eds.) BrainLes 2019. LNCS, vol. 11992, pp. 153–162. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-46640-4_15
Hamghalam, M., Lei, B., Wang, T.: Convolutional 3D to 2D patch conversion for pixel-wise glioma segmentation in MRI scans. In: Crimi, A., Bakas, S. (eds.) BrainLes 2019. LNCS, vol. 11992, pp. 3–12. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-46640-4_1
Menze, B.H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., et al.: The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans. Med. Imaging 34(10), 1993–2024 (2015). https://doi.org/10.1109/TMI.2014.2377694
Akinyelu, A.A., Zaccagna, F., Grist, J.T., Castelli, M., Rundo, L.: Brain tumor diagnosis using machine learning, convolutional neural networks, capsule neural networks and vision transformers, applied to MRI: a survey. J. Imaging 8, 205 (2022)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Zhou, Z., Rahman Siddiquee, M.M., Tajbakhsh, N., Liang, J.: UNet++: a nested U-Net architecture for medical image segmentation. In: Stoyanov, D., et al. (eds.) DLMIA/ML-CDS -2018. LNCS, vol. 11045, pp. 3–11. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00889-5_1
Zhang, Z., Liu, Q., Wang, Y.: Road extraction by deep residual U-Net. IEEE Geosci. Remote Sens. Lett. 15, 749–753 (2018)
Isensee, F., Jaeger, P.F., Kohl, S.A., Petersen, J., Maier-Hein, K.H.: nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat. Methods 18, 203–211 (2020)
Hu, H., Zhang, Z., Xie, Z., Lin, S.: Local relation networks for image recognition. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV) (2019)
Dosovitskiy, A., et al.: An image is worth \(16\times 16\) words: transformers for image recognition at scale. In: ICLR 2021 (2021)
Hatamizadeh, A., et al.: UNETR: transformers for 3D medical image segmentation. In: 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) (2022)
Mojtahedi, R., Hamghalam, M., Do, R.K.G., Simpson, A.L.: Towards optimal patch size in vision transformers for tumor segmentation. In: Li, X., Lv, J., Huo, Y., Dong, B., Leahy, R.M., Li, Q. (eds.) MMMI 2022. LNCS, vol. 13594, pp. 110–120. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-18814-5_11
Milletari, F., Navab, N., Ahmadi, S.-A.: V-net: fully convolutional neural networks for volumetric medical image segmentation. In: 2016 Fourth International Conference on 3D Vision (3DV) (2016)
Baid, U., et al.: The RSNA-ASNR-MICCAI BraTS 2021 benchmark on brain tumor segmentation and radiogenomic classification. arXiv:2107.02314 (2021)
Bakas, S., Akbari, H., Sotiras, A., Bilello, M., Rozycki, M., Kirby, J.S., et al.: Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features. Nat. Sci. Data 4, 170117 (2017). https://doi.org/10.1038/sdata.2017.117
Bakas, S., Akbari, H., Sotiras, A., Bilello, M., Rozycki, M., Kirby, J., et al.: Segmentation labels and radiomic features for the pre-operative scans of the TCGA-GBM collection. The Cancer Imaging Archive (2017). https://doi.org/10.7937/K9/TCIA.2017.KLXWJJ1Q
Bakas, S., Akbari, H., Sotiras, A., Bilello, M., Rozycki, M., Kirby, J., et al.: Segmentation labels and radiomic features for the pre-operative scans of the TCGA-LGG collection. The Cancer Imaging Archive (2017). https://doi.org/10.7937/K9/TCIA.2017.GJQ7R0EF
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Mojtahedi, R., Hamghalam, M., Simpson, A.L. (2023). Multi-modal Brain Tumour Segmentation Using Transformer with Optimal Patch Size. In: Bakas, S., et al. Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. BrainLes 2022. Lecture Notes in Computer Science, vol 13769. Springer, Cham. https://doi.org/10.1007/978-3-031-33842-7_17
Download citation
DOI: https://doi.org/10.1007/978-3-031-33842-7_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-33841-0
Online ISBN: 978-3-031-33842-7
eBook Packages: Computer ScienceComputer Science (R0)
