Elsevier

NeuroImage

Volume 117, 15 August 2015, Pages 67-79
NeuroImage

Optimization of rs-fMRI Pre-processing for Enhanced Signal-Noise Separation, Test-Retest Reliability, and Group Discrimination

https://doi.org/10.1016/j.neuroimage.2015.05.015Get rights and content

Highlights

  • We examine hundreds of permutations of preprocessing parameters.

  • We measure how these parameters affect scan noise, reliability, and group discriminability.

  • We validate our findings on multiple datasets collected both by us and other groups.

  • Parameter selection influences noise, reliability, and group discriminability.

  • These metrics can be optimized simultaneously with correct parameter selection.

Abstract

Resting-state functional magnetic resonance imaging (rs-fMRI) has become an increasingly important tool in mapping the functional networks of the brain. This tool has been used to examine network changes induced by cognitive and emotional states, neurological traits, and neuropsychiatric disorders. However, noise that remains in the rs-fMRI data after preprocessing has limited the reliability of individual-subject results, wherein scanner artifacts, subject movements, and other noise sources induce non-neural temporal correlations in the blood oxygen level-dependent (BOLD) timeseries. Numerous preprocessing methods have been proposed to isolate and remove these confounds; however, the field has not coalesced around a standard preprocessing pipeline. In comparisons, these preprocessing methods are often assessed with only a single metric of rs-fMRI data quality, such as reliability, without considering other aspects in tandem, such as signal-to-noise ratio and group discriminability. The present study seeks to identify the data preprocessing pipeline that optimizes rs-fMRI data across multiple outcome measures. Specifically, we aim to minimize the noise in the data and maximize result reliability, while retaining the unique features that characterize distinct groups. We examine how these metrics are influenced by bandpass filter selection and noise regression in four datasets, totaling 181 rs-fMRI scans and 38 subject-driven memory scans. Additionally, we perform two different rs-fMRI analysis methods – dual regression and region-of-interest based functional connectivity – and highlight the preprocessing parameters that optimize both approaches. Our results expand upon previous reports of individual-scan reliability, and demonstrate that preprocessing parameter selection can significantly change the noisiness, reliability, and heterogeneity of rs-fMRI data. The application of our findings to rs-fMRI data analysis should improve the validity and reliability of rs-fMRI results, both at the individual-subject level and the group level.

Introduction

Resting-state functional magnetic resonance imaging (rs-fMRI) has been an important tool in mapping the functional networks of the brain, and in examining network changes induced by cognitive and emotional states, traits, and disorders. Many findings related to resting-state networks, including the spatial distribution of functional networks and their cognitive correlates (Biswal et al., 1995, Hampson et al., 2002, Greicius et al., 2003, Kiviniemi et al., 2003, Beckmann et al., 2005, Bellec et al., 2006, Damoiseaux et al., 2006, Seeley et al., 2007, Kiviniemi et al., 2009, Smith et al., 2009), changes in network connectivity induced by cognitive states (Richiardi et al., 2010, Shirer et al., 2012), and group differences in network connectivity due to brain disorders (Greicius et al., 2004, Greicius et al., 2007, Hedden et al., 2009, Seeley et al., 2009, Sheline et al., 2010, Zhou et al., 2010, Johnson et al., 2013), are well-replicated at the group level. However, reliability at the individual level remains rather limited (Damoiseaux and Greicius, 2009, Honey et al., 2009). Previous studies that have evaluated the consistency of resting-state networks in individuals have mainly used the intra-class correlation coefficient (ICC) as a measure of test-retest reliability, which is scaled from 0 to 1—where 0 represents no reliability, and 1 represents perfect reliability. These studies have reported values on the order of 0.5 (Shehzad et al., 2009) to 0.6 (Zuo et al., 2010, Thomason et al., 2011, Guo et al., 2012), which reflect only modest test-retest reliability, thereby diminishing confidence in group contrasts, undermining generalizability of classification approaches, and confounding interpretation of subject-level results. The low test-retest reliability of rs-fMRI data is largely attributable to noise that remains in the data after preprocessing, wherein scanner artifacts, subject movements, and other noise sources induce non-neural temporal correlations in the blood oxygen level-dependent (BOLD) signal. Numerous preprocessing methods have been proposed to isolate and remove these confounds (Fox et al., 2009, Murphy et al., 2009, Weissenbacher et al., 2009, Braun et al., 2012, Friston et al., 1996; Power et al., 2012, Power et al., 2013, Power et al., 2014, Satterthwaite et al., 2013, Siegel et al., 2014, Yan et al., 2013a).

The present study seeks to comprehensively and simultaneously compare various preprocessing strategies, and assess their effects on multiple outcome measures: signal-noise separation, test-retest reliability, and group discriminability. Previous studies attempting to optimize rs-fMRI data have focused exclusively on one of these metrics without consideration as to how other aspects of data quality are affected. In particular, studies examining test-retest reliability of rs-fMRI data do not quantify how much noise remains in the data, or how maximizing this measure affects the ability to distinguish different groups of subjects (Shehzad et al., 2009, Zuo et al., 2010, Thomason et al., 2011, Guo et al., 2012, Yan et al., 2013b).

We focus our analyses on the effects of common preprocessing steps, such as global signal regression (GS) (Weissenbacher et al., 2009, Shirer et al., 2012); removal of cerebrospinal fluid (CSF) and white matter (WM) confounds (Shirer et al., 2012); noise regression of motion parameters estimated during motion correction (Friston et al., 1996, Power et al., 2012, Power et al., 2013, Satterthwaite et al., 2013, Yan et al., 2013a); and temporal filtering at various frequency bands reported in the literature (Achard et al., 2006, Ko et al., 2011, Guo et al., 2012). To maximize generalizability of our results, we elect not to examine preprocessing techniques that are unavailable at some imaging centers, such as removal of physiological noise using RETROICOR or RVHRCOR (Chang and Glover, 2009).

We analyze the rs-fMRI data using two measures of functional connectivity (FC): Pearson’s correlation between regions of interest (ROI) pairs, and ICA (dual regression, or DR). We then assess the quality of the outcome data based on three metrics. First, we propose a novel method of estimating signal-noise separation (SNS) in rs-fMRI data. Here, we calculate the functional connectivity between brain areas within well-established networks, and contrast this with spurious connectivity between these areas and non-neural regions outside the brain. This procedure is performed for each preprocessing permutation to identify the preprocessing parameters that produce the greatest ratio of network connectivity to noise. Second, we assess the impact of preprocessing parameters on the test-retest reliability (TRT) of whole-brain connectivity. We do so by measuring the TRT of network connectivity produced by each preprocessing pipeline, using the same intra-class correlation (ICC) metric as reported in previous rs-fMRI test-retest reliability studies (Shehzad et al., 2009, Zuo et al., 2010, Thomason et al., 2011, Guo et al., 2012). The effects of preprocessing on SNS and TRT are first evaluated on data collected by us, and then examined on an independent dataset that has been widely used in previous test-retest reliability studies for comparison (Shehzad et al., 2009, Zuo et al., 2010, Thomason et al., 2011, Guo et al., 2012). Finally, we assess the clinical relevance of different preprocessing pipelines by examining group discriminability (GD), wherein we compare different pipelines’ accuracy in classification of Alzheimer’s disease (AD) patients and controls. To ensure that the preprocessing strategies most successful for classifying disease states are not simply due to non-neural artifacts (e.g. differences in the amount of movement between groups), we also explore the effect of different preprocessing pipelines on the ability to classify different cognitive states within individual subjects.

Section snippets

Stanford University data

Data were compiled from three datasets, resulting in a total of thirty-eight healthy right-handed subjects (21 females) aged 18-78, and fifteen subjects with mild AD (8 females) aged 51-86 who participated in this study. The first Stanford healthy control dataset (S-HC1) contained twenty-three healthy right-handed subjects (15 females) aged 56-76. The second Stanford healthy control dataset (S-HC2) contained fifteen healthy right-handed subjects (6 females) aged 22-46. The third dataset, S-AD,

Results

A multiple regression analysis across 120 pipeline combinations revealed a significant effect of preprocessing parameters on all three metrics of data quality. This was true for both types of functional connectivity analyses (Table 1A).

Discussion

The present study examines numerous common preprocessing strategies, synthesizing the unique and aggregate effects of these techniques across multiple metrics of rs-fMRI data quality. Many previous studies assessing rs-fMRI data quality focus on a single preprocessing strategy, such as motion artifact removal (Power et al., 2014), or a single metric of data quality, such as reliability (Shehzad et al., 2009). However, some degree of variability may be attributable to biologically meaningful

Conflict of interest

The authors declare no conflict of interest.

Acknowledgements

This work was supported by the National Institutes of Health (NS073498). We thank Jonas Richiardi, Andre Altmann, and Jeffrey Bernstein for helpful discussions and advice.

References (61)

  • K.A. Johnson et al.

    Functional connectivity in Alzheimer's disease: measurement and meaning

    Biol. Psychiatry

    (2013)
  • V. Kiviniemi et al.

    Independent component analysis of nondeterministic fMRI signal sources

    NeuroImage

    (2003)
  • L. Lemieux et al.

    Modelling large motion events in fMRI studies of patients with epilepsy

    Magn. Reson. Imaging

    (2007)
  • K. Murphy et al.

    The impact of global signal regression on resting state correlations: Are anti-correlated networks introduced?

    NeuroImage

    (2009)
  • V. Perlbarg et al.

    CORSICA: correlation of structured noise in fMRI by automatic identification of ICA components

    Magn. Reson. Imaging

    (2007)
  • J.D. Power et al.

    Spurious but systematic correlations in functional connectivity MRI networks arise from subject motion

    NeuroImage

    (2012)
  • J.D. Power et al.

    Steps toward optimizing motion artifact removal in functional connectivity MRI; a reply to Carp

    NeuroImage

    (2013)
  • J.D. Power et al.

    Methods to detect, characterize, and remove artifact in resting-state fMRI

    NeuroImage

    (2014)
  • G. Salimi-Khorshidi et al.

    Automatic denoising of functional MRI data: combining independent component analysis and hierarchical fusion of classifiers

    NeuroImage

    (2014)
  • T.D. Satterthwaite et al.

    An improved framework for confound regression and filtering for control of motion artifact in the preprocessing of resting-state functional connectivity data

    NeuroImage

    (2013)
  • W.W. Seeley et al.

    Neurodegenerative diseases target large-scale human brain networks

    Neuron

    (2009)
  • S.C. Strother et al.

    The quantitative evaluation of functional neuroimaging experiments: the NPAIRS data analysis framework

    NeuroImage

    (2002)
  • M.E. Thomason et al.

    Resting-state fMRI can reliably map neural networks in children

    NeuroImage

    (2011)
  • A. Weissenbacher et al.

    Correlations and anticorrelations in resting-state functional connectivity MRI: A quantitative comparison of preprocessing strategies

    NeuroImage

    (2009)
  • C.G. Yan et al.

    A comprehensive assessment of regional variation in the impact of head micromovements on functional connectomics

    NeuroImage

    (2013)
  • C.G. Yan et al.

    Standardizing the intrinsic brain: towards robust measurement of inter-individual variation in 1000 functional connectomes

    NeuroImage

    (2013)
  • X.N. Zuo et al.

    Reliable intrinsic connectivity networks: Test-retest evaluation using ICA and dual regression approach

    NeuroImage

    (2010)
  • S. Achard et al.

    A resilient, low-frequency, small-world human brain functional network with highly connected association cortical hubs

    J. Neurosci.

    (2006)
  • J.L. Andersson et al.

    Non-linear registration, aka Spatial normalisation

  • C.F. Beckmann et al.

    Investigations into resting-state connectivity using independent component analysis

    Philos. Trans. R. Soc. Lond. B Biol. Sci.

    (2005)
  • Cited by (149)

    • Statistical power in network neuroscience

      2023, Trends in Cognitive Sciences
    View all citing articles on Scopus
    1

    Authors contributed equally to this work.

    View full text