Abstract
Accurate segmentation and grading of brain tumors from multi-modal magnetic resonance imaging (MRI) play a vital role in the diagnosis and treatment of brain tumors. The gene expression in glioma also influences the selection of treatment strategies and assessment of patient survival, such as the gene mutation status of isocitrate dehydrogenase (IDH), the co-deletion status of 1p/19q, and the value of Ki67. However, obtaining medical image annotations is both time-consuming and expensive, and it is challenging to perform tasks such as brain tumor segmentation, grading, and genotype prediction directly using label-deprived multi-modal MRI. We proposed a comprehensive multi-modal domain adaptative aid (CMDA) framework building on hospital datasets from multiple centers to address this issue, which can effectively relieve distributional differences between labeled source datasets and unlabeled target datasets. Specifically, a comprehensive diagnostic module is proposed to simultaneously accomplish the tasks of brain tumor segmentation, grading, genotyping, and glioma subtype classification. Furthermore, to learn the data distribution between labeled public datasets and unlabeled local hospital datasets, we consider the semantic segmentation results as the output capturing the similarity between different data sources, and we employ adversarial learning to facilitate the network in learning domain knowledge. Experimental results show that our end-to-end CMDA framework outperforms other methods based on direct transfer learning and other state-of-the-art unsupervised methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Parsons, D.W.: An integrated genomic analysis of human glioblastoma multiforme. Science 321(5897), 1807–1812 (2008)
Oronsky, B.: A review of newly diagnosed glioblastoma. Front. Oncol. 10, 574012 (2021)
Komori, T.: Grading of adult diffuse gliomas according to the 2021 WHO Classification of Tumors of the Central Nervous System. Lab. Invest. 102(2), 126–133 (2022)
Menze, B.H.: The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans. Med. Imaging 34(10), 1993–2024 (2014)
Fu, J.: Dual attention network for scene segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3146–3154 (2019)
Menze, B.H.: A generative model for brain tumor segmentation in multi-modal images. In: Jiang, T., Navab, N., Pluim, J.P.W., Viergever, M.A. (eds.) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2010. MICCAI 2010. LNCS, vol. 6362. Springer, Berlin, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15745-5_19
Havaei, M.: Brain tumor segmentation with deep neural networks. Med. Image Anal. 35, 18–31 (2017)
Long, J.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440. (2015)
Dong, H.: Automatic brain tumor detection and segmentation using U-Net based fully convolutional networks. In: Hernández, M.V., González-Castro, V. (eds.) MIUA 2017. CCIS, vol. 723, pp. 506–517. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-60964-5_44
Cui, S.: Automatic semantic segmentation of brain gliomas from MRI images using a deep cascaded neural network. J. Healthcare Eng. 2018 (2018)
Vu, T.H.: Advent: adversarial entropy minimization for domain adaptation in semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2517–2526. (2019)
Wang, H.: Classes matter: a fine-grained adversarial approach to cross-domain semantic segmentation. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12359, pp. 642–659. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58568-6_38
Tsai, Y.H.: Learning to adapt structured output space for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7472–7481. (2018)
Zhang, Y.: Task driven generative modeling for unsupervised domain adaptation: application to X-ray image segmentation. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11071, pp. 599–607. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00934-2_67
Chen, C.: Deep learning for cardiac image segmentation: a review. Front. Cardiovasc. Med. 7, 25 (2020)
Chen, C.: Synergistic image and feature adaptation: towards cross-modality domain adaptation for medical image segmentation. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 865–872 (2019)
He, K.: Cross-Modality Brain Tumor Segmentation via Bidirectional Global-to-Local Unsupervised Domain Adaptation. arXiv preprint arXiv:2105.07715 (2021)
Zhou, Z.: Unet++: redesigning skip connections to exploit multiscale features in image segmentation. IEEE Trans. Med. Imaging 39(6), 1856–1867 (2019)
Zhu, J.Y.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232 (2017)
Chen, L.C.: Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2017)
Oktay, O.: Attention U-Net: Learning Where to Look for the Pancreas. arXiv preprint arXiv:1804.03999 (2018)
Ronneberger, O.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W., Frangi, A. (eds.) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. MICCAI 2015. LNCS, vol. 9351. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Acknowledgments
This work was supported in part by the National Key R&D Program of China under Grant 2022YFC2402102, and in part by the National Natural Science Foundation of China under grant numbers 82071913 and 22161142024.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Chu, W., Zhou, Y., Cai, S., Chen, Z., Cai, C. (2024). A Comprehensive Multi-modal Domain Adaptative Aid Framework for Brain Tumor Diagnosis. In: Liu, Q., et al. Pattern Recognition and Computer Vision. PRCV 2023. Lecture Notes in Computer Science, vol 14437. Springer, Singapore. https://doi.org/10.1007/978-981-99-8558-6_32
Download citation
DOI: https://doi.org/10.1007/978-981-99-8558-6_32
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8557-9
Online ISBN: 978-981-99-8558-6
eBook Packages: Computer ScienceComputer Science (R0)