Graph Convolutions on Spectral Embeddings for Cortical Surface Parcellation

Highlights

• Novel spectral graph convolutional approach for cortical parcellation.

• Direct learning of surface data using trainable spectral filters over intrinsic embeddings.

• New spectral scheme for graph convolutional approaches with a direct surface representation.

• Training of spectral filters across multiple mesh geometries of various graph structures.

• State-of-the-art performance for cortical surface parcellation with graph convolutions on the largest publicly available manually labeled dataset.

Abstract

Neuronal cell bodies mostly reside in the cerebral cortex. The study of this thin and highly convoluted surface is essential for understanding how the brain works. The analysis of surface data is, however, challenging due to the high variability of the cortical geometry. This paper presents a novel approach for learning and exploiting surface data directly across multiple surface domains. Current approaches rely on geometrical simplifications, such as spherical inflations, a popular but costly process. For instance, the widely used FreeSurfer takes about 3 hours to parcellate brain surfaces on a standard machine. Direct learning of surface data via graph convolutions would provide a new family of fast algorithms for processing brain surfaces. However, the current limitation of existing state-of-the-art approaches is their inability to compare surface data across different surface domains. Surface bases are indeed incompatible between brain geometries. This paper leverages recent advances in spectral graph matching to transfer surface data across aligned spectral domains. This novel approach enables direct learning of surface data across compatible surface bases. It exploits spectral filters over intrinsic representations of surface neighborhoods. We illustrate the benefits of this approach with an application to brain parcellation. We validate the algorithm over 101 manually labeled brain surfaces. The results show a significant improvement in labeling accuracy over recent Euclidean approaches while gaining a drastic speed improvement over conventional methods.

Introduction

Neuroimage analysis consists of studying functional and anatomical information over the brain geometry. Various aspects of the brain are investigated using different imaging modalities, such as magnetic resonance imaging (MRI) data. Structural MRI provides notably the geometry of the cortex. The thin outer layer of the brain cerebrum is of particular interest due to its vital role in cognition, vision, and perception. Statistical frameworks on surfaces are, therefore, highly sought for studying various aspects of the brain. For instance, variations in surface data can reveal new biomarkers as well as possible relations with disease processes (Arbabshirani et al., 2017). The challenge consists of learning surface data over highly complex and convoluted surfaces and across different subjects.

The goal of separating the cerebral cortex into distinct regions based on structure or function is known as parcellation. Initially, automated parcellation techniques used clustering based on local regional statistics (Craddock et al., 2012). For instance, a semi-supervised technique (Glasser et al., 2016) delineated the cortical boundary from sharp changes in multimodal MRI data. Most research works use a cortical surface based feature to find surface correspondence. BrainVisa (Cointepas, Geffroy, Souedet, Denghien, Rivière, 2010, Rivière, Régis, Cointepas, Papadopoulos-Orfanos, Cachia, Mangin, 2003, Auzias, Lefèvre, Le Troter, Fischer, Perrot, Régis, Coulon, 2013) uses sulcal features defined by the cortical folding patterns to find correspondence between brain surfaces. Features like sulcal pits or sulcal lines (Lohmann, Von Cramon, Colchester, 2007, Auzias, Brun, Deruelle, Coulon, 2015) are other existing features used for estimating surface correspondences. Conventional approaches typically rely on geometrical simplifications, such as spherical inflation and slow mesh deformations (Tustison, Cook, Klein, Song, Das, Duda, Kandel, van Strien, Stone, Gee, Avants, 2014, Styner, Oguz, Xu, Brechbühler, Pantazis, Levitt, Shenton, Gerig, 2006, Yeo, Sabuncu, Vercauteren, Ayache, Fischl, Golland, 2010), a popular but costly process. For instance, the widely used FreeSurfer (Fischl et al., 2004) takes around 3 hours to parcellate brain surfaces by slowly deforming brain models towards labeled atlases.

Convolutional Neural Networks (CNNs) (Lecun et al., 1998) have the potential to offer a drastic speed advantage over traditional surface-based methods. CNNs are mostly used in neuroimage analysis for segmentation (Wachinger et al., 2017) or finding structural abnormalities (Valverde et al., 2017). The network architecture is either fixed for various segmentation applications (Ronneberger et al., 2015) or tailored to particular problems (Kamnitsas et al., 2017). Fundamentally, current statistical frameworks exploit spatial information mostly derived from the Euclidean domain, for instance, based on vector fields, image or volumetric coordinates (Zhang, Davatzikos, 2011, Hua, Hibar, Ching, Boyle, Rajagopalan, Gutman, Leow, Toga, Jack, Harvey, Weiner, Thompson, Alzheimer’s Disease Neuroimaging Initiative, 2013, Dolz, Desrosiers, Ben Ayed, 2017, Kamnitsas, Ledig, Newcombe, Simpson, Kane, Menon, Rueckert, Glocker, 2017). Such information is highly variable across brain geometries and severally hinders the training of modern machine learning algorithms.

Geometric deep learning (Bronstein et al., 2017) recently proposed to use convolutional filters on irregular graphs. To handle the neural network on a graph, Scarselli et al. (2009) proposes to map and learn graph data in a high-dimensional Euclidean space. Lately, Bruna et al. (2014) formulates the convolution theorem from Fourier space to spectral domains over graphs. Chebyshev polynomials are also used to avoid the explicit computation of graph Laplacian eigenvectors (Defferrard et al., 2016). The main concern of these methods is their inability to compare surface data across different surface domains (Bronstein, Glashoff, Loring, Kovnatsky, Bronstein, Bronstein, Glashoff, Kimmel, 2013, Ovsjanikov, Ben-Chen, Solomon, Butscher, Guibas, 2012, Eynard, Kovnatsky, Bronstein, Glashoff, Bronstein, 2015). Laplacian eigenbases are indeed incompatible across multiple geometries. Alternatively, Masci et al. (2015) and Boscaini et al. (2016) proposed a graph convolution approach in the spatial domain. These approaches map local graph information onto geodesic patches and use conventional spatial convolution as template matching. For instance, Monti et al. (2017) obtains geodesic patches with local parametric constructions of tangent planes to the surface. Another prominent spatial approach Veličković et al. (2018) proposes to include self-attentional layers in which neighborhoods are used to avoid an explicit computation of a graph Laplacian. This attentional approach reduces to a particular formulation of Monti et al. (2017). A related work (Simonovsky and Komodakis, 2017) also conditions convolutional filter weights on specific edge labels over neighborhoods rather than on graph nodes. Applications of graph convolution networks in neuroimaging remain yet limited. Existing work includes the use of graph convolutions over population graphs for predicting brain disorders and learning distance metrics in functional brain networks (Parisot, Ktena, Ferrante, Lee, Guerrero, Glocker, Rueckert, 2018, Ktena, Parisot, Ferrante, Rajchl, Lee, Glocker, Rueckert, 2017). A recent work (Cucurull et al., 2018) proposes to parcellate the cerebral cortex into three parcels using an attention-based method (Veličković et al., 2018). Brain meshes are, however, constrained within a unique graph structure, limiting all meshes to use the same mesh geometry. Fundamentally, these methods either lack the capability to process multiple surface domains (Bronstein, Glashoff, Loring, Kovnatsky, Bronstein, Bronstein, Glashoff, Kimmel, 2013, Ovsjanikov, Ben-Chen, Solomon, Butscher, Guibas, 2012, Eynard, Kovnatsky, Bronstein, Glashoff, Bronstein, 2015) or have spatial representations of surface data defined in a Euclidean space, which ignore the complex geometry of the surface. They rely, for instance, on polar representations of mesh vertices (Boscaini, Masci, Rodolà, Bronstein, 2016, Masci, Boscaini, Bronstein, Vandergheynst, 2015, Monti, Boscaini, Masci, Rodolà, Svoboda, Bronstein, 2017, Veličković, Cucurull, Casanova, Romero, Liò, Bengio, 2018).

This paper leverages recent advances in spectral graph matching to transfer surface data across aligned spectral domains (Lombaert et al., 2015a). The transfer of spectral coordinates across domains provides a robust formulation for spectral methods that naturally handles differences across Laplacian eigenvectors, including sign flips, ordering, and mixing of eigenvectors in higher frequencies. This spectral alignment strategy was exploited to learn surface data (Lombaert et al., 2015b) within the random forest framework. Spectral forests are operating in a spectral domain and use the first spectral coordinates as well as sulcal depth of each cortical point. This approach is, however, limited to only pointwise information, ignoring local patterns within surface neighborhoods. Our approach consists of leveraging spectral coordinates within graph convolutional networks to bridge a gap between learning algorithms and geometrical representations. To the best of our knowledge, this is the first attempt at intrinsically learning surface data via spectral graph convolutions in neuroimaging. This novel approach enables a direct learning of surface data across compatible surface bases by exploiting spectral filters over intrinsic representations of surface neighborhoods.

The main contributions of our work are:

• A novel spectral graph convolutional approach for cortical parcellation;

• A direct learning of surface data using trainable spectral filters over surface embeddings;

• The training of spectral filters across multiple mesh geometries of various graph structures;

• The leverage of spectral filters to exploit local patterns of data within surface neighborhoods;

• An evaluation on the largest publicly available dataset of manually labeled brain surfaces (Klein et al., 2017);

• An improved state-of-the-art performance for cortical surface parcellation with graph convolutions;

In this work, we propose a surface learning algorithm. We illustrate the learning capabilities of this approach with an application to brain parcellation. We choose cortical parcellation since it provides established benchmarks with publicly available datasets of manual labelings. The validation over the largest publicly available dataset of manually labeled brain surfaces (Klein et al., 2017), with 101 subjects, demonstrates a significant improvement in using spectral graph convolutions over Euclidean approaches. This change of paradigm indeed improves the parcellation accuracy when using graph convolutions, from a Dice score of 53% to 85%. Our approach is at least at par with the well established FreeSurfer algorithm (Fischl et al., 2004) when benchmarking over a large dataset (Klein et al., 2017), while gaining a drastic speed improvement in the order of seconds. The next section details the fundamentals of our spectral approach, followed by experiments evaluating the impact of our spectral strategy over standard Euclidean approaches for graph convolutions.