White Matter Tract Segmentation with Self-supervised Learning

Abstract

White matter tract segmentation based on diffusion magnetic resonance imaging (dMRI) plays an important role in brain analysis. Deep learning based methods of white matter tract segmentation have been proposed to improve the segmentation accuracy. However, manual delineations of white matter tracts for network training are especially difficult to obtain. Therefore, in this paper, we explore how to improve the performance of deep learning based white matter tract segmentation when the number of manual tract delineations is limited. Specifically, we propose to exploit the abundant unannotated data using a self-supervised learning approach, where knowledge about image context can be learned in a well designed pretext task that does not require manual annotations. The knowledge can then be transferred to the white matter tract segmentation task, so that when manual tract delineations for training are scarce, the performance of the network can be improved. To allow the image context knowledge to be relevant to white matter tracts, the pretext task in this work is designed to predict the density map of fiber streamlines, where training data can be obtained using tractography without manual efforts. The model pretrained for the pretext task is then fine-tuned by the small number of tract annotations for the target segmentation task. In addition, we explore the possibility of combining self-supervised learning with a complementary pseudo-labeling strategy of using unannotated data. We validated the proposed approach using dMRI scans from the Human Connectome Project dataset, where the benefit of the proposed method is shown when tract annotations are scarce.