Towards Accurate Fetal Brain Parcellation via Hierarchical Network and Loss

Abstract

Automatic fetal brain parcellation on Magnetic Resonance (MR) images is increasingly being used to assess prenatal brain growth and development. Despite their progress, existing methods are limited due to ignoring of the hierarchical nature of segmentation labels and the rich complementary information among hierarchical labels. To address these limitations, we propose a novel deep-learning model to segment the whole fetal brain into 87 fine-grained regions hierarchically. Specifically, we design a hierarchical network with adjustable levels and define a three-level structure. These levels are dedicated, respectively, to predicting 8 types of brain tissues, 36 more detailed brain regions, and ultimately 87 brain regions according to developing Human Connectome Project (dHCP) labels. The coarse-level network is capable of providing prior features to the fine-level network for fine-grained brain parcellation. This design involves decomposing complex problems into simpler ones and addresses intricate issues with the priors for resolving simple problems. Furthermore, we design a data augmentation module to simulate variations in scanning parameters, enabling precise segmentation of fetal brain images across diverse domains. Finally, we integrate this data augmentation module into a semi-supervised paradigm to alleviate the shortage of high-quality labeled data and enhance the generalizability of our model. Thanks to these designs, our model can obtain fine-grained and multi-scale brain segmentation with high robustness to variations in MR scanners and imaging protocols. Extensive experiments on 558 dHCP and 176 fetal brain MR images demonstrate that our model achieves state-of-the-art segmentation performance across multi-site datasets. Our code is publicly available at https://github.com/sj-huang/HieraParceNet.