Abstract
In neuroscience research, automatic segmentation of macaque brain tissues in magnetic resonance imaging (MRI) is crucial for understanding brain structure and function during development and evolution. Acquisition of multimodal information is a key enabler of accurate tissue segmentation, especially in early-developing macaques with extremely low contrast and dynamic myelination. However, many MRI scans of early-developing macaques are acquired only in a single modality. While various generative adversarial networks (GAN) have been developed to impute missing modality data, current solutions treat modality generation and image segmentation as two independent tasks, neglecting their inherent relationship and mutual benefits. To address these issues, this study proposes a novel Collaborative Segmentation-Generation Framework (CSGF) that enables joint missing modality generation and tissue segmentation of macaque brain MR images. Specifically, the CSGF consists of a modality generation module (MGM) and a tissue segmentation module (TSM) that are trained jointly by a cross-module feature sharing (CFS) and transferring generated modality. The training of the MGM under the supervision of the TSM enforces anatomical feature consistency, while the TSM learns multi-modality information related to anatomical structures from both real and synthetic multi-modality MR images. Experiments show that the CSGF outperforms the conventional independent-task mode on an early-developing macaque MRI dataset with 155 scans, achieving superior quality in both missing modality generation and tissue segmentation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Roelfsema, P.R., et al.: Basic neuroscience research with nonhuman primates: a small but indispensable component of biomedical research. Neuron 82(6), 1200–1204 (2014)
Zhong, T., et al.: DIKA-Nets: Domain-invariant knowledge-guided attention networks for brain skull stripping of early developing macaques. Neuroimage 227, 117649 (2021)
Li, G., et al.: Computational neuroanatomy of baby brains: a review. Neuroimage 185, 906–925 (2019)
Isensee, F., et al.: nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat. Methods 18(2), 203–211 (2021)
Yu, B., et al.: Medical image synthesis via deep learning. Deep Learn. Med. Image Anal. 23–44 (2020)
Zhou, B., Liu, C., Duncan, J.S.: Anatomy-constrained contrastive learning for synthetic segmentation without ground-truth. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12901, pp. 47–56. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87193-2_5
Zheng, H., et al.: Phase Collaborative Network for Two-Phase Medical Imaging Segmentation. arXiv:1811.11814 (2018)
Zhou, T., et al.: A review: deep learning for medical image segmentation using multi-modality fusion. Array 3, 100004 (2019)
Pan, Y., Chen, Y., Shen, D., Xia, Y.: Collaborative image synthesis and disease diagnosis for classification of neurodegenerative disorders with incomplete multi-modal neuroimages. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12905, pp. 480–489. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87240-3_46
Young, J.T., et al.: The UNC-Wisconsin rhesus macaque neurodevelopment database: a structural MRI and DTI database of early postnatal development. Front. Neurosci. 11, 29 (2017). https://pubmed.ncbi.nlm.nih.gov/28210206/
Xie, G., et al.: A Survey of Cross-Modality Brain Image Synthesis. arXiv preprint arXiv:2202.06997 (2022)
Yu, Z., Zhai, Y., Han, X., Peng, T., Zhang, X.-Y.: MouseGAN: GAN-based multiple MRI modalities synthesis and segmentation for mouse brain structures. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12901, pp. 442–450. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87193-2_42
Goodfellow, I., et al.: Generative adversarial networks. Commun. ACM 63(11), 139–144 (2020)
Isola, P., et al.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017), pp. 1125–1134. IEEE(2017)
Zhu, J., et al.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017), pp. 2223–2232. IEEE (2017)
Zhang, X., et al.: PTNet3D: a 3D high-resolution longitudinal infant brain MRI synthesizer based on transformers. IEEE Trans. Med. Imaging 41(10), 2925–2940 (2022)
Wang, L., et al.: Links: learning-based multi-source integration framework for segmentation of infant brain images. NeruoImage 108, 160–172 (2015)
Zhong, T., et al.: Longitudinal brain atlases of early developing cynomolgus macaques from birth to 48 months of age. NeruoImage 247, 118799 (2022)
Wang, F., et al.: Developmental topography of cortical thickness during infancy. Proc. Natl. Acad. Sci. 116(32), 15855–15860 (2019)
Jenkinson, M., et al.: FSL. Neuroimage 62(2), 782–790 (2012)
Yushkevich, P.A., et al.: User-guided 3D active contour segmentation of anatomical structures: significantly improved efficiency and reliability. Neuroimage 31(3), 1116–1128 (2006)
National Centre for the Replacement, Reduction and Refinement of Animals in Research. https://macaques.nc3rs.org.uk/about-macaques/life-history
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
UW-Madison Rhesus MRI dataset. https://fcon_1000.projects.nitrc.org/indi/PRIME/uwmadison.html
Tustison, N.J., et at.: N4ITK: improved N3 bias correction. IEEE Trans. Med. Imaging 29(6), 1310–1320 (2010)
Acknowledgements
This work was supported in part by the National Natural Science Foundation of China #61971213, and #U22A20350, the National Natural Science Foundation of Youth Science Foundation project of China #62201246, and #62001206, and the Guangzhou Basic and Applied Basic Research Project #2023A04J2262.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wu, X., Zhong, T., Liang, S., Wang, L., Li, G., Zhang, Y. (2023). Collaborative Modality Generation and Tissue Segmentation for Early-Developing Macaque Brain MR Images. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14223. Springer, Cham. https://doi.org/10.1007/978-3-031-43901-8_45
Download citation
DOI: https://doi.org/10.1007/978-3-031-43901-8_45
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43900-1
Online ISBN: 978-3-031-43901-8
eBook Packages: Computer ScienceComputer Science (R0)
