Template-based graph registration network for boosting the diagnosis of brain connectivity disorders

https://doi.org/10.1016/j.compmedimag.2022.102140Get rights and content

Highlights

  • We propose the first template-based graph registration neural network to boost downstream learning tasks.

  • We propose a novel joint graph registration and classification loss function.

  • Our plug-in GRN is a generic model. It can be trained using any type of graph templates.

  • The GRN source code is available on GitHub.

Abstract

Brain graphs are powerful representations to explore the biological roadmaps of the human brain in its healthy and disordered states. Recently, a few graph neural networks (GNNs) have been designed for brain connectivity synthesis and diagnosis. However, such non-Euclidean deep learning architectures might fail to capture the neural interactions between different brain regions as they are trained without guidance from any prior biological template—i.e., template-free learning. Here we assume that using a population-driven brain connectional template (CBT) that captures well the connectivity patterns fingerprinting a given brain state (e.g., healthy) can better guide the GNN training in its downstream learning task such as classification or regression. To this aim we design a plug-in graph registration network (GRN) that can be coupled with any conventional graph neural network (GNN) so as to boost its learning accuracy and generalizability to unseen samples. Our GRN is a graph generative adversarial network (gGAN), which registers brain graphs to a prior CBT. Next, the registered brain graphs are used to train typical GNN models. Our GRN can be integrated into any GNN working in an end-to-end fashion to boost its prediction accuracy. Our experiments showed that GRN remarkably boosted the prediction accuracy of four conventional GNN models across four neurological datasets.

Introduction

Deep Learning (DL) has dominated the research field of clinical decision making including brain disease diagnosis (Suzuki, 2017, Lee et al., 2017). Recent works (Asiri et al., 2019) designing computer-aided diagnosis (CAD) systems have integrated DL diagnosis models to be more robust and powerful in discriminating between disordered and healthy patients. DL has an outstanding ability to learn multi-level representation of medical imaging data such as Magnetic Resonance Imaging (MRI) or resting-state functional MRI (rs-fMRI) (Suzuki, 2017, Lee et al., 2017). Such non-invasive imaging technologies provide anatomical features such as gray matter volumes and cortical thickness as potential clinical biomarkers of particular neurological disorders—to mention just a few. However, the brain is a compound, highly and internally connected system. Thus, such simple features might fail to capture the brain interconnectedness (van den Heuvel and Sporns, 2019, Fornito et al., 2015). Thanks to their ability to represent connections between different entities, graphs are powerful representations to exhibit the relational information between different anatomical regions of interest (ROIs) in the brain.

Several studies on brain graph analysis focused on node classification, link prediction, and graph classification using particular machine learning (ML) methods (Richiardi et al., 2013, Du et al., 2018). For instance, Khazaee et al. (2015) constructed connectivity matrices derived from rs-fMRI data and performed statistical analysis using ANOVA and forward sequential feature selection to obtain discriminative feature vectors. Next, they trained a support vector machine (SVM) classifier based on these discriminative feature vectors to distinguish between Alzheimer’s (AD), Mild Cognitive Impairment (MCI), and healthy patients. A very recent paper (Bilgen et al., 2020) examined a diverse pool of machine learning pipelines in classifying cortical brain networks. Nonetheless, traditional ML methods merely work for a specific task or a dataset and rarely generalize to other datasets and tasks which indicates that they lack the ability of generalizability. Therefore, Graph Neural Networks (GNN) come forward with their higher generalizability to non-Euclidean data and powerful performance. However, there are very limited works on Graph Neural Networks (GNNs) for brain graph classification as reported in a recent review paper (Bessadok et al., 2022).

For graph classification and graph representation, Geometric Deep Learning (GDL) has become a leading focal point in various areas such as social science (Hamilton et al., 2018), e-commerce networks (Li et al., 2020), natural science (biology networks) (Bove et al., 2020), and traffic networks (Diehl et al., 2019, Mallick et al., 2020). GNNs root in several fundamental concepts (Zhou et al., 2018), which are Convolutional Neural Networks (CNNs) and graph embedding—to mention just a few. CNNs with multi-layer architectures extract high-order representations from raw input data to map them into the desired output. The multi-layer architecture helps attain valuable information from hierarchical patterns, which refer to non-Euclidean data of graphs. Also, CNNs acknowledge regional connections in images, which are the fundamental data source of CNNs, by reconstructing their localized spatial features. Therefore, CNN is a strong concept to adapt for GNNs. Accordingly, Kipf and Welling (2017) proposed Graph Convolutional Network (GCN), which applies convolutional operations to graphs to learn a hidden layer that captures both local graph structures and node features and scales the graph edges to a desired output feature map. Another work (Veličković et al., 2018) introduced Graph Attentional Network (GAT), an attention-based architecture that generates hidden representations for each node in a graph by calculating the importance of each neighbor. These GNN methods reproduce features from the graph edges linearly and overlook to learn the hierarchical representations of graphs, which might be questionable over the graph classification task where the aim is to try to predict the label of an entire graph. To overcome this limitation, Ying et al. (2019) proposed DiffPool, a differentiable graph pooling module for hierarchical graph representation. DiffPool embeds graphs using GNNs and clusters these embeddings as nodes to generate a new graph at each hierarchical layer. The number of layers is a hyperparameter and the last layer outputs the classified label. Lastly, Gao and Ji (2019) came up with an encoder–decoder architecture for graphs and introduced graph U-Nets (g-U-Nets), which consists of graph pooling (gPool) and graph un-pooling (gUnpool) blocks.

Although such conventional models deliver powerful performance in several graph applications, they have some limitations when the task domain is network neuroscience due to the complexity and richness of brain connectivity graphs (Bessadok et al., 2022). In fact, brain connectomes are biological roadmaps of brain connections between different anatomical ROIs. It is hence crucial to preserve the topological soundness of such biological roadmaps in downstream learning tasks such as brain synthesis, classification or regression (Bessadok et al., 2022). Besides, existing GNN architectures might fail to capture the neural interactions between different brain regions as they are trained without guidance from any prior biological template—i.e., template-free learning (Bessadok et al., 2022). The concept of template-based learning is commonly used on Euclidean data such as images. In fact, image-based brain atlases or templates are commonly used to register an input image to a prior image-based template (e.g., MRI) for abnormality detection, disorder diagnosis, brain mapping and evolution trajectory prediction—among other purposes (Fan et al., 2006, Davatzikos et al., 2011, Kim et al., 2016, Min et al., 2014, Liu et al., 2015, Gafuroglu and Rekik, 2019, Li et al., 2019). Simply put, image registration aims to apply a geometric transformation to input images for aligning them onto a fixed prior image. The minimization of the difference between the aligned image and the fixed image template determines the success in image registration. A considerable amount of studies worked on image registration in medical imaging for various tasks (Sokooti et al., 2017, Hu et al., 2018, Balakrishnan et al., 2019, de Vos et al., 2019). However, while there is an abundance of image-based registration methods in the neuroscience and neuroimaging literature, graph-based registration remains an uncharted territory.

To fill this gap and motivated by the outperformance template-guided or atlas-guided learning tasks over the template-free methods, we set out to generalize the concept of image registration to graphs and demonstrate its value in boosting classification accuracy of brain states. Here, we assume that using a population-driven brain connectional template (CBT) (Rekik et al., 2017, Dhifallah and Rekik, 2020, Gurbuz and Rekik, 2020, Gürbüz and Rekik, 2021) that captures well the connectivity patterns fingerprinting a given brain state (e.g., healthy) can better guide the GNN training in its downstream learning task such as classification or regression. Specifically, we drive inspiration from the recent works on integrational network neuroscience where CBTs are derived from input brain graph populations (Dhifallah and Rekik, 2020, Gurbuz and Rekik, 2020) and propose a Graph Registration Network (GRN). Our GRN acts as a plug-in network that can be coupled with any conventional graph neural network in an end-to-end fashion to boost its learning accuracy and generalizability to unseen samples. Particularly, our GRN is a graph generative adversarial network (gGAN), which registers brain graphs to a prior CBT. Training in an end-to-end fashion makes each step act dependently with each other and send feedback to each other, which helps optimize the learning process. Next, the registered brain graphs are used to train typical GNN models.

Fig. 1 illustrates the main concept of our brain graph registration and classification framework. First, we register brain graphs with respect to a graph-based template (here CBT) and obtain registered brain graphs that preserve the common interactional neural patterns. Registering the input brain graphs to a prior CBT might help preserve the brain topology and also better bring out the unique features of individual brain graphs. In fact, such graph registration step will enable to tease apart the individuality of each brain connectome by simply comparing its registered version with the prior population CBT. The registered brain graphs are then inputted to the conventional GNNs as they better capture the individual traits of their topology for the target learning task (e.g., classification). Below, we articulate the main contributions of our work at different levels:

  • 1.

    We propose a plug-in network that can eventually work with other GNN architecture and be combined with other downstream learning tasks.

  • 2.

    We propose a network that works in an end-to-end manner by optimizing a joint loss function of both registration and classification networks at each iteration.

  • 3.

    Our GRN generalizes the registration concept to graphs to boost the classification performance of any GNN architecture.

  • 4.

    Our plug-in GRN is a generic model. It can be trained using any type of graph templates (e.g., genomic and any omic template).

Section snippets

Methodology

In this section, we detail our GRN for boosting GNN based classifiers for neurological disorders diagnosis. Table 1 displays the mathematical notations that we use throughout this paper. We denote the matrices as boldface capital letters, e.g., X, and scalars as lowercase letters, e.g., n. We illustrate in Fig. 2 the two proposed steps: (1) registration of input graphs into the population CBT and (2) classification of the registered brain graphs by the GRN plug-in.

Results and discussion

Connectomic dataset. We evaluated our GRN plug-in on two datasets, where each subject is represented by a morphological brain graph. The first dataset (300) (ASD/NC dataset) is collected from the Autism Brain Imaging Data Exchange ABIDE I public dataset2 and consists of 300 subjects: 150 normal controls (NC) and 150 subjects with autism spectrum disorder (ASD) (Martino et al., 2013, Soussia and Rekik, 2018). The second dataset (LMCI/AD dataset)

Conclusion

In this paper, we proposed a template-based graph registration network that can be used as a plug-in to boost the performance of graph neural network architectures. Our model is composed of two major parts, respectively a graph registration network (GRN) and a GNN. GRN is a gGAN, which registers each input brain graph to a prior connectional brain template (CBT) to enhance their individual connectivity features. Both networks are trained in an end-to-end fashion while optimizing a joint loss

Code availability

An open-source Python implementation of GRN is available on GitHub at https://github.com/basiralab/GRN. The release includes a demo using simulated data and notes regarding Python packages, which need to be installed. Information regarding input format can be also found in the same repository.

CRediT authorship contribution statement

Zeynep Gürler: Methodology, Formal analysis, Validation, Visualization, Writing – original draft, Writing – review & editing. Mohammed Amine Gharsallaoui: Formal analysis, Writing – original draft. Islem Rekik: Conceptualization, Supervision, Methodology, Resources, Writing – review & editing, Funding acquisition.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This work was funded by generous grants from the European H2020 Marie Sklodowska-Curie action (grant no. 101003403, http://basira-lab.com/normnets/) to I.R. and the Scientific and Technological Research Council of Turkey to I.R. under the TUBITAK 2232 Fellowship for Outstanding Researchers (no. 118C288, http://basira-lab.com/reprime/). However, all scientific contributions made in this project are owned and approved solely by the authors. M.A.G is supported by the same TUBITAK 2232 Fellowship.

References (72)

  • YalçinA. et al.

    A diagnostic unified classification model for classifying multi-sized and multi-modal brain graphs using graph alignment

    J. Neurosci. Methods

    (2021)
  • BalakrishnanG. et al.

    VoxelMorph: A learning framework for deformable medical image registration

    IEEE Trans. Med. Imaging

    (2019)
  • BankaA. et al.

    Multi-view brain HyperConnectome AutoEncoder for brain state classification

  • BessadokA. et al.

    Graph neural networks in network neuroscience

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2022)
  • BirdG. et al.

    Empathic brain responses in insula are modulated by levels of alexithymia but not autism

    Brain

    (2010)
  • BoisgueheneucF.d. et al.

    Functions of the left superior frontal gyrus in humans: a lesion study

    Brain

    (2006)
  • BoveP. et al.

    Prediction of dynamical properties of biochemical pathways with graph neural networks

    Bioinformatics

    (2020)
  • ChaariN. et al.

    Estimation of gender-specific connectional brain templates using joint multi-view cortical morphological network integration

    Brain Imaging Behav.

    (2020)
  • ChironC. et al.

    SPECT of the brain in childhood autism: evidence for a lack of normal hemispheric asymmetry

    Dev. Med. Child Neurol.

    (1995)
  • DavatzikosC. et al.

    Multi-parametric analysis and registration of brain tumors: constructing statistical atlases and diagnostic tools of predictive value

  • Di MartinoA. et al.

    Functional brain correlates of social and nonsocial processes in autism spectrum disorders: an activation likelihood estimation meta-analysis

    Biol. Psychiatry

    (2009)
  • Di PaolaM. et al.

    When, where, and how the corpus callosum changes in MCI and AD

    Neurology

    (2010)
  • Diehl, F., Brunner, T., Le, M.T., Knoll, A., 2019. Graph Neural Networks for Modelling Traffic Participant Interaction....
  • Doyle-ThomasK.A. et al.

    The effect of diagnosis, age, and symptom severity on Cortical Surface Area in the cingulate cortex and insula in autism spectrum disorders

    J. Child Neurol.

    (2012)
  • DrzezgaA. et al.

    Cerebral metabolic changes accompanying conversion of mild cognitive impairment into Alzheimer’s disease: a PET follow-up study

    Eur. J. Nucl. Med. Mol. Imaging

    (2003)
  • DuY. et al.

    Classification and prediction of brain disorders using functional connectivity: Promising but challenging

    Front. Neurosci.

    (2018)
  • FanY. et al.

    Diagnosis of brain abnormality using both structural and functional MR images

  • FischlB.

    Automatically parcellating the human cerebral cortex

    Cerebral Cortex

    (2004)
  • FornitoA. et al.

    The connectomics of brain disorders

    Nat. Rev. Neurosci.

    (2015)
  • GafurogluC. et al.

    Image evolution trajectory prediction and classification from baseline using learning-based patch atlas selection for early diagnosis

    (2019)
  • GaoH. et al.

    Graph u-nets

  • GoodfellowI. et al.

    Generative adversarial nets

    Adv. Neural Inf. Process. Syst.

    (2014)
  • GurbuzM.B. et al.

    Deep graph normalizer: A geometric deep learning approach for estimating connectional brain templates

  • GürbüzM.B. et al.

    MGN-net: A multi-view graph normalizer for integrating heterogeneous biological network populations

    Med. Image Anal.

    (2021)
  • GurlerZ. et al.

    Foreseeing brain graph evolution over time using deep adversarial network normalizer

    (2020)
  • HamiltonW.L. et al.

    Inductive representation learning on large graphs

    (2018)
  • Cited by (0)

    View full text