Abstract
Breast cancer is a major health issue, causing millions of deaths each year worldwide. Magnetic Resonance Imaging (MRI) is an effective tool for detecting and diagnosing breast tumors, with various MRI sequences providing comprehensive information on tumor morphology. However, existing methods for segmenting tumors from multi-parametric MRI have limitations, including the lack of considering inter-modality relationships and exploring task-informative modalities. To address these limitations, we propose the Modality-Specific Information Disentanglement (MoSID) framework, which extracts both intra- and inter-modality attention maps as prior knowledge to guide tumor segmentation from multi-parametric MRI. This is achieved by disentangling modality-specific information that provides complementary clues to the segmentation task and generating modality-specific attention maps in a synthesis manner. The modality-specific attention maps are further used to guide modality selection and inter-modality evaluation. Experiment results on a large breast dataset show that the MoSID achieves superior performance over other state-of-the-art multi-modality segmentation methods, and works reasonably well even with missing modalities.
J. Zhang and Q. Chen–Equal Contribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Avants, B.B., Tustison, N.J., Song, G., Cook, P.A., Klein, A., Gee, J.C.: A reproducible evaluation of ants similarity metric performance in brain image registration. Neuroimage 54(3), 2033–2044 (2011)
Chen, C., Dou, Q., Jin, Y., Chen, H., Qin, J., Heng, P.-A.: Robust multimodal brain tumor segmentation via feature disentanglement and gated fusion. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11766, pp. 447–456. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32248-9_50
Ding, Y., Yu, X., Yang, Y.: Rfnet: region-aware fusion network for incomplete multi-modal brain tumor segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3975–3984 (2021)
Gal, Y., Ghahramani, Z.: Dropout as a bayesian approximation: representing model uncertainty in deep learning. In: International Conference on Machine Learning, pp. 1050–1059. PMLR (2016)
Han, Z., Yang, F., Huang, J., Zhang, C., Yao, J.: Multimodal dynamics: dynamical fusion for trustworthy multimodal classification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20707–20717 (2022)
Huang, L., Denoeux, T., Vera, P., Ruan, S.: Evidence fusion with contextual discounting for multi-modality medical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 401–411. Springer, Heidelberg (2022). https://doi.org/10.1007/978-3-031-16443-9_39
Li, C., Sun, H., Liu, Z., Wang, M., Zheng, H., Wang, S.: Learning cross-modal deep representations for multi-modal MR image segmentation. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11765, pp. 57–65. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32245-8_7
Oktay, O., et al.: Attention u-net: learning where to look for the pancreas. arXiv preprint arXiv:1804.03999 (2018)
Venturelli, S., Leischner, C., Helling, T., Renner, O., Burkard, M., Marongiu, L.: Minerals and cancer: overview of the possible diagnostic value. Cancers 14(5), 1256 (2022)
Xing, Z., Yu, L., Wan, L., Han, T., Zhu, L.: Nestedformer: nested modality-aware transformer for brain tumor segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 140–150. Springer, Heidelberg (2022). https://doi.org/10.1007/978-3-031-16443-9_14
Zhang, G., et al.: Cross-modal prostate cancer segmentation via self-attention distillation. IEEE J. Biomed. Health Inf. 26, 5298–5309 (2021)
Zhang, J., et al.: A robust and efficient AI assistant for breast tumor segmentation from DCE-MRI via a spatial-temporal framework. Patterns 4(9), 1–14 (2023)
Zhang, Y., et al.: mmformer: multimodal medical transformer for incomplete multimodal learning of brain tumor segmentation. arXiv preprint arXiv:2206.02425 (2022)
Zhang, Y., et al.: Modality-aware mutual learning for multi-modal medical image segmentation. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12901, pp. 589–599. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87193-2_56
Acknowledgments
This work was supported in part by The Key R &D Program of Guangdong Province, China (grant number 2021B0101420006), National Natural Science Foundation of China (grant number 62131015), and Science and Technology Commission of Shanghai Municipality (STCSM) (grant number 21010502600).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, J. et al. (2023). MoSID: Modality-Specific Information Disentanglement from Multi-parametric MRI for Breast Tumor Segmentation. In: Ali, S., van der Sommen, F., van Eijnatten, M., Papież, B.W., Jin, Y., Kolenbrander, I. (eds) Cancer Prevention Through Early Detection. CaPTion 2023. Lecture Notes in Computer Science, vol 14295. Springer, Cham. https://doi.org/10.1007/978-3-031-45350-2_8
Download citation
DOI: https://doi.org/10.1007/978-3-031-45350-2_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-45349-6
Online ISBN: 978-3-031-45350-2
eBook Packages: Computer ScienceComputer Science (R0)
