Keywords

1 Introduction

1.1 Overview of Cortical Sulci Recognition Approaches

The surface of the human brain cortex is divided into gyri, separated by fissures called sulci. The largest sulci are good indicators of the localization of functional areas and the morphometry of the sulci geometry is used to quantify brain development and neurodegenerative processes. Automatic recognition is mandatory to exploit the large databases yielded by recent neuroimaging projects. However, automatic sulci recognition remains a challenging problem because of the lack of understanding of the nature of the interindividual variability of the folding patterns. During the last twenty years, many different approaches have been proposed to handle this problem.

The approaches based on atlas registration yield reasonable identification rate for major sulci. For instance, in [12], a labeled brain is elastically deformed to fit a new brain, and in [6], a multiresolution strategy is designed to improve the results by progressively registering the largest folds to the smallest folds. This single template strategy however has strong limitations with regard to the numerous folding configurations incompatible with the atlas geometry, which occurs even for the largest sulci.

Modeling intersubject variability seems mandatory to increase robustness, which was tackled using a variety of frameworks ranging from PCA to Bayesian approaches [1, 5, 7, 9]. A large amount of strategies rely on graph-based representations, which provide a flexible way to model the spatial relationships between the sulci in addition to their shape and localization in a normalized space [3, 10, 11, 13,14,15]. Comparing these approaches is difficult in the absence of benchmark. Furthermore, all of them except the one in BrainVISA are restricted to a small set of sulci, a small training database and are not distributed. Therefore, in this paper, we will only compare our new model to the results of BrainVISA current package.

1.2 The Current BrainVISA Model: Principles, Advantages and Inconvenients

The current BrainVISA model relies on a coherent Bayesian framework based on a probabilistic atlas (a model made of a mixture of Statistical Probabilistic Anatomy Maps (SPAM)). This approach performs simultaneously sulcus recognition and local alignment with the template of SPAMs [9]. Unfortunately, this approach performs poorly with unusual folding patterns, which depart from the main modes of the probabilistic atlas. This is a classical limitation of single template strategies. For instance, the reconfiguration of the folding patterns induced by interruptions of the large sulci can lead to inconsistent sulcus recognition. Furthermore, we have observed disturbing mistakes that did not occur with the older graph-based approach also proposed in BrainVISA [10]. For instance a large sulcus can be locally duplicated because the current Bayesian framework does not model the relationships between folds, while the previous graph-based approach could learn alternative configurations thanks to a neural network (NN) based memory. Unfortunately, this NN strategy could probably not be trained with a large enough manually labelled database to reach the robustness of the Bayesian framework.

1.3 MAS Strategy: A Solution to the Limits of the Current Model?

In biomedical image analysis, segmentation is the process of tagging image voxels with biologically meaningful labels. The MAS strategy has become one of the most widely-used and successful solution. The idea of this technique is to use for segmentation the entire dataset of “atlases” (i.e. training images) rather than one model-based average kind of representation, so that it better captures the anatomical variability. A power of this approach is its pragmatism: it can be applied for a problem where a real understanding of the nature of the interindividual variability is lacking, which fits the current status of the neuroscience world with regard to the cortical folding.

Most of the MAS techniques include four main steps: (1) Atlas generation: the design of atlases from the training images; (2) Registration: alignment of each atlas onto the new image to be segmented; (3) Label propagation: from each aligned atlas to the new subject, (4) Label fusion: combining the propagated labels to achieve the final segmentation. One of the most recent MAS techniques developed by Coupé et al. [4] performs a patch-based label fusion, which leads to very efficient implementations.

The problem of sulci recognition differs from standard MAS applications by several aspects. First, there are many more anatomical structures to be labeled: each brain contains up to 125 cortical sulci in the nomenclature of BrainVISA. Second and more importantly, grey level intensities and textures surrounding the sulci do not provide discriminative information except for a few sulci like central sulcus, because of a specific local myelin content. Therefore, the key discriminative information is folding geometry, which requires larger voxel-based patches than in usual implementations. Hence, for the sake of efficiency, in the following we will operate at a larger intermediate scale of representation of the folding patterns, where the entities to be labelled are predefined sets of voxels corresponding to the most elementary folds (about 500 such entities in a standard brain) [8].

The new model proposed in this article is inspired by the patch-based methodology proposed by Coupé et al. [4] but the patches are not defined as local groups of voxels but as local groups of sulci. Note that this strategy could be extended to all the domains where an intermediate level of representation of the data could be useful for the alignment of the atlases to the unknown subject. Hence, we have to revisit the different stages of the mainstream strategy to take into account the heterogeneity of the representations of the subjects. In the following, we design a dedicated patch generation strategy, a geometry-based measure of similarity to perform the patch-based alignment, and a propagation strategy for 3D points clouds.

2 Database

In order to compare our results to those of [9] in the same conditions, we use the exact same database of 62 healthy subjects selected from different heterogeneous databases. Most of the subjects are right-handed male persons, between 25 and 35 years old. The elementary folds of each brain were manually labelled according to the sulcus nomenclature following a long process leading to achieve a consensus across a set of several experts of the morphology of the cortex. Each fold representation is a set of voxels corresponding to the medial surface of the cerebrospinal fluid filling up the fold. Hence, these fold representations are elementary pieces of a negative mould of the brain. As mentioned above, the sulcus recognition method described in this paper will be forced to give one unique label to the set of voxels representing a fold. The training base is composed of 62 brains labelled with a model containing 62 sulci for the right hemisphere and 63 for the left.

3 Method: A New Model Inspired by MAS Algorithms

As the new model is inspired by the MAS strategy, the same steps are used and will be described in this section (Fig. 1). In the following, the atlas brains and the new brain are all represented as a set of elementary folds. All the atlas folds have been manually labelled with the sulcus nomenclature. All the brains have been affinely aligned with the Talairach standard space.

Fig. 1.
figure 1

Patch-based segmentation approach for automatic sulci recognition.

Full size image

3.1 Patch Generation

In this paper, we experiment with a reasonable but relatively naive strategy, which will be refined in the future using machine learning. The goal is to define from the atlas dataset a library of local patches embedding enough geometrical information to minimize ambiguities when searching for a high similarity hit in the unknown subject morphology. Note that the shape of small sulci is not informative enough to prevent spurious hits. Hence the idea is to aggregate a few sulci to create discriminative local shapes. In the following, we define a type of patches for each pair of sulci that are neighbors in the brain (Fig. 1). Practically, a pair of sulci is selected if the instances of the two sulci are neighbors at least in one brain of the atlas dataset, for the topology provided by the BrainVISA pipeline yielding the folds. This pipeline endows the list of folds with a graph structure corresponding to either direct connections or to the fact that two folds are separated by a piece of gyrus. Finally, for a given sulcus S, with neighbors \(S_{1}, S_{2}, ..., S_{N}\), N patches are generated from each atlas of the dataset, each patch corresponding to a different type of patch \(SS_{1}, SS_{2},..., SS_{N}\). Note that if the neighbor or S is missing in the atlas, the patch is not included in the library.

3.2 Registration

For the registration step, the set of folds of the unknown brain and the patches of the library are represented by point clouds. The well-known iterative closest points algorithm [2] is applied to find an optimal alignment of each patch into the point cloud made up of the unlabelled folds of the unknown brain. To build the measure used to rank the matches, the nearest points in the new brain of each patch point are saved as activated points. The measure corresponds to the sum of quadratic distances of the patch points and their corresponding activated points, divided by the number of different activated points.

3.3 Label Propagation

For each patch type, the ten instances leading to the smallest distances are selected to participate to the propagation of the two parent sulci. The number of instances selected has been set to ten arbitrary for a first implementation and should be optimized later on. Note that some sulcus instances are selected several times, because they win the competition for several patch types. But their multiple contributions will be associated with slightly different alignments. Hence, sulcus instances maximizing regional similarity to the unknown subject get more weight. For each selected patch after the ICP registration to the unknown subject, each point gives its label to its nearest neighbor in the target brain. To consider the patch structure, each connected set of points in the patch should correspond to a unique connected set in the target brain: the smallest non-connected sets are excluded. Hence, each point of the target brain can be labeled several times from several patches and only the most propagated label is saved.

Fig. 2.
figure 2

Mean error rates per sulcus. The graph on the left and the graph on the right present the mean error rates for the sulci on the left hemisphere and on the right hemisphere, respectively. The Brainvisa model is represented in violet and the new model is represented in pink. The significative differences (\(p_{value}<0.05\)) are marked with a star.

Full size image

3.4 Label Fusion

The label fusion process is performed on a fold by fold basis. For each fold of the unknown subject, the sulcus label the most represented in its points is given to the fold. Without activated points in the fold, the “unknown” label is kept.

4 Results and Comparison with the BrainVISA Model

The proposed approach and the Bayesian BrainVISA approach are compared using a Leave-One-Out strategy on the database described above. Many different error measures are possible to evaluate the model. The measures used to evaluate the current BrainVISA model are the \(E_{local}\) for each sulcus and \(E_{SI}\) for the final labeling. Here is their definitions for a given subject:

$$\begin{aligned} E_{local}(l) = \frac{FP(l) + FN(l)}{FP(l) + FN(l) + TP(l)} \; , \end{aligned}$$
(1)
$$\begin{aligned} E_{SI} = \sum \limits _{l \in L} w_{l} \frac{FP(l) + FN(l)}{FP(l) + FN(l) + 2TP(l)} \; , \end{aligned}$$
(2)

with L the ensemble of sulcus labels, FP(l), FN(l) and TP(l) respectively the size of the set of voxels false positive, false negative and true positive for the label l and \(w_{l} = \frac{s_{l}}{\sum \limits _{l \in L} s_{l}}\) with \(s_{l} = FN(l)+TP(l)\) the true size of the sulcus with the label l.

The advantages of \(E_{SI}\) is that it is sensible to local labeling and it takes into account the shared errors between labels [9]. With this measure, we deduct the main recognition rate for the two compared models: the current BrainVISA model obtains 85.53% (+/– 5.80%) for the left hemisphere and 86.27% (+/– 6.12%) for the right hemisphere while the new model obtains 86.76% (+/– 5.16%) and 88.04% (+/– 5.70%).

In order to compare the two models, we calculate the error rates for each subject in the database and compare the set of measures with a T-test. The \(E_{SI}\) comparison shows that the new model is significantly better than the Brainvisa model (\(p_{value}=4.14e-8\)). Moreover, by comparing the \(E_{local}\) per sulcus, more than 20 sulci are found significantly better (Fig. 2).

5 Conclusion

This paper describes a first attempt at casting the cortical sulci recognition problem in a MAS-based framework. While some choices are still ad hoc and will require further developments, the comparison with the existing Bayesian approach is full of promise. The main contribution of our work is the extension of the MAS framework to a high level representation of the data dedicated to our pattern recognition problem. This original setting is calling for a more sophisticated inference of the library of patches, which should probably be developed with an optimization strategy trying to maximize the recognition result while keeping the library as small as possible. This strategy shall pick the selected patch types from a larger combinatorial set aggregating more than two sulci. Another improvement opportunity lies in the patch selection strategy, for example by learning the optimal cut-off for each type of patches. The label fusion could be largely improved by the introduction of weights, in order to balance the contribution of each patch according to its similarity measure. Finally, as the new strategy involves a labeling per point, it will allow the method to question the initial split of the cortex morphology into elementary folds to detect under-segmentation issues. This is an essential element to overcome the weakness of the current Bayesian framework, which is stuck to the high level representation yielded by the preprocessing stage. This possibility can be considered as a top-down feature embedded in the global recognition system.