Elsevier

NeuroImage

Volume 42, Issue 2, 15 August 2008, Pages 778-786
NeuroImage

The validity of the face-selective ERP N170 component during simultaneous recording with functional MRI

https://doi.org/10.1016/j.neuroimage.2008.04.168Get rights and content

Abstract

Despite the wide interest in the neural mechanisms of face processing and numerous event-related potential (ERP) and functional MRI (fMRI) studies of face-selective neural responses, no study, to date, has collected these two measures simultaneously. The main reason for the absence of such an investigation is that MRI data acquisition generates major artifacts, which completely conceals the EEG signal. Recently, artifact removal algorithms have been developed. Our goal was to examine the validity of the face-selective ERP component N170 and its functional effects such as category selectivity and hemispherical laterality, when recorded simultaneously with functional MRI. In our experiment, half of the scans were collected during fMRI acquisition and half without fMRI acquisition. The validity of the N170 was then measured for its amplitude, latency, face selectivity (the difference between the amplitude to faces and objects), laterality (the difference between the amplitude to faces over the right and the left hemispheres) and the laterality of the face selectivity effect, by correlating these measures across subjects between data collected without fMRI and with fMRI data acquisition, after applying artifact removal procedures. We found high validity coefficients for all N170 measures. Furthermore, ERP data collected outside the scanner on a different day were highly correlated with data collected during MR acquisition for the N170 amplitude, latency, and selectivity index but moderate for laterality indices. Our study demonstrates that face-selective ERP effects are preserved in simultaneous recording with fMRI. These findings will hopefully encourage researchers to combine the two complementary neuroimaging techniques in future research.

Introduction

Faces elicit a robust and reliable neural response in temporal-occipital regions of the human brain. Specifically, event-related potential (ERP) studies have revealed an early temporal-occipital component that peaks around 170 ms after stimulus onset, which shows larger amplitude for faces than objects (Bentin et al., 1996; for an extensive review see Rossion and Jacques (2008)). Functional MRI studies have revealed regions in the occipital–temporal cortex that show a much higher response for faces than objects (Kanwisher et al., 1997, McCarthy et al., 1997; for review see Kanwisher and Yovel (2006)). These findings from human neuroimaging are consistent with single cell recording studies with monkeys, which have reported neurons in the inferior temporal cortex and superior temporal sulcus that show a highly face-selective response (Desimone et al., 1984, Tsao et al., 2006).

The two most prevalent human neuroimaging techniques – functional MRI (fMRI) and ERPs – provide complementary information about spatial and temporal aspects of the neural response, respectively. In particular, functional MRI reveals the brain regions that show a strong face-selective response with relatively high spatial resolution (millimeters) but very poor temporal resolution due to the slow hemodynamic response. In contrast, ERPs measured over the scalp provide an electrophysiological signal of high temporal resolution (milliseconds), but poor information about the exact location of its neural sources. Surprisingly, despite the hundreds of studies that have been published on the neural response to faces with fMRI and ERP and the complementary information they provide about the brain response to faces, only a few studies, to date, have attempted to assess the correlations between the ERP and fMRI response to faces when collected in different sessions (Henson et al., 2003, Horovitz et al., 2004) and, to the best of our knowledge, no study has measured both face-related signals simultaneously.

Simultaneous EEG–fMRI data collection is advantageous to separate recording sessions in several ways. (i) The environments of fMRI and EEG experimental settings are fundamentally different and may generate differences in the brain response to the stimulus or task manipulation that will decrease the correlations between them. (ii) The subject response to the task may differ across sessions and if one is interested in the correlations between the signal and the subject trial-by-trial response, simultaneous recording is mandatory. (iii) If an identical task is used in the two sessions, learning effects may generate different brain responses across the two sessions. (iv) Direct correlations between the task-induced modulation of the EEG and fMRI signal that are time-locked to one another (for review see Debener et al. (2007)) will be maximal when they are collected simultaneously. Thus, more robust and valid correlations between the two signals can be revealed when they are collected simultaneously.

The main reason for the absence of such simultaneous fMRI–EEG investigation is that artifacts generated by the MRI mask the EEG signal. In particular, MR gradient switching produces high amplitude artifacts, which completely conceal the EEG signal (see Fig. 1A). Furthermore, because of the high magnetic field, additional artifacts are induced by heart pulses – ballistocardiogram (BCG) artifacts – and further distort the EEG signal (see Fig. 1B). To avoid the artifacts of the MR gradient on the EEG signal, some simultaneous EEG–fMRI studies have used the method of interleaved fMRI data acquisition (e.g. Bonmassar et al., 1999, Bonmassar et al., 2001, Mulert et al., 2005, Otzenberger et al., 2005). Because the hemodynamic response peaks about 6 s after stimulus onset, whereas relevant ERP data can be collected within 1 to 2 s after stimulus onset, fMRI data collection can be done starting 2 s after stimulus onset and still provide event-related signals for the same events. The main disadvantage of this method is that the interstimulus interval should not exceed the time course of the hemodynamic response (10–16 s). Thus, the rate of stimulus presentation is very low and the number of stimuli that can be presented in a given recording session is limited. Because the signal-to-noise ratio of fMRI and ERP data increases as the number of trials increases, rapid event-related data collection, which allows many more trials to be presented in a given session is more desirable. Such rapid presentation, however, requires continuous EEG–fMRI recording and an effective method to remove the gradient artifacts that are generated during MRI data acquisition.

Fortunately, MR gradients elicit a predicted and relatively constant pattern of distortion in the EEG signal. Effective algorithms for MR artifact removal (Fig. 1B) as well as algorithms for detection and removal of BCG artifacts (Fig. 1C) have been recently developed, which allow recovery of the EEG signal offline (Allen et al., 2000, Iannetti et al., 2005, Niazy et al., 2005). For example, a recent study reported high validity for the amplitude and latency of the P1 and N1 response to colored words after the removal of gradient and BCG artifacts in a 1.5 T MR scanner (Comi et al., 2005). However, no study, to date, has examined whether the functional characteristics or the experimental effects associated with an ERP component are preserved after the removal of these artifacts. In particular, before embarking on an investigation of the relationship between face-related neural mechanisms using simultaneous ERP–fMRI methods, it is critical to determine that the known functional characteristics of the face-related ERP component N170, such as its category selectivity and hemispheric asymmetry, can be reliably recovered after artifact removal algorithms are applied.

The present study examines whether the ERP response to faces that is recorded simultaneously with BOLD acquisition yields reliable evoked responses, which preserve the basic features of the face-selective N170 response. We first asked whether the amplitude and the latency of the N170 are preserved after the application of artifact rejection algorithms. More importantly, we also confirmed that the well-established functional characteristics of an ERP component are preserved. In particular, the face-selective N170 has three main functional characteristics: 1) The N170 amplitude is significantly larger for faces than non-face objects (e.g. Bentin et al., 1996, Rossion and Jacques, 2008). 2) The N170 amplitude is larger over the right than the left temporal–occipital electrodes (e.g. Bentin et al., 1996, Yovel et al., 2003). 3) The magnitude of the N170 face selectivity is larger over the right than the left hemisphere (Bentin et al., 1996, Rossion et al., 2003).

To assess the validity of the face-selective N170, ERPs for faces and non-face objects (chairs) were recorded while subjects lay in a 3 T MR scanner. Most previous studies of simultaneous EEG–MRI recording were done in 1.5 T scanners (e.g. Becker et al., 2005, Garreffa et al., 2004, Garreffa et al., 2003, Laufs et al., 2003, Lazeyras et al., 2001) so validation of ERP data in a 3 T scanner, which provides better signal-to-noise fMRI data, but also produces twice as large BCG artifacts in the EEG signal, is important. On half of the scans, ERPs were collected during fMRI acquisition and, on the other half, without fMRI acquisition, when subjects were lying in the static magnetic field. We then performed correlational analyses across subjects between data collected during fMRI scanning and without fMRI scanning to assess the validity of each of the five N170 measures. In addition, our subjects returned for another session outside the magnet and we assessed the correlations with data we collected inside the MRI. To the best of our knowledge, the correlations between the N170 measures that were recorded on separate sessions have never been reported, even for data collected in standard settings.

Section snippets

Subjects

Eleven subjects (age: 22–30, eight females, two left handed) participated in a simultaneous recording of ERP and fMRI. Left-handed subjects were not excluded from our sample as is often the case in face perception studies, in order to increase the variance in our laterality measures and avoid low correlations due to restricted range. Two right-handed subjects, one female and one male, were excluded due to technical problems in recordings. Eight of the nine remaining subjects returned for

Amplitude

To assess the pattern of the N170 response for faces and chairs in temporal–occipital electrodes with and without MR acquisition we performed a three-way repeated measure ANOVA with MR condition (MR, no-MR), Hemisphere and Category (face, chair) as repeated measures and amplitude or latency as dependent measures. Our analyses reveal no difference between the pattern of response to faces and chairs at the right and left occipito-temporal electrodes during MR recording and without MR recording.

Discussion

We report the results of the first study of the face-selective ERP component (N170) that was collected simultaneously with functional MRI. To determine whether the well-established face-selective characteristics of the N170 are preserved during simultaneous fMRI data acquisition, we examined the validity of the N170 peak amplitude, latency, selectivity to faces, laterality to faces, and laterality of its face selectivity measure. Our data reveal high validity scores for all five N170 measures

Acknowledgments

This research was supported by the Marie Currie IRG-046448 and Adams Super Center for Brain Research to GY. The Israeli Scientific Foundation bikura program, Israeli Scientific Foundation converging technologies program, US-Israel Bi-national Scientific Foundation psychobiology smith's center grants to TH.

References (31)

Cited by (37)

  • Characterizing the neural signature of face processing in Williams syndrome via multivariate pattern analysis and event related potentials

    2020, Neuropsychologia
    Citation Excerpt :

    Interestingly an earlier N170 peak for faces compared to houses was observed in WS but not in TD. An earlier N170 component for faces compared to other objects has previously been reported in adults (Itier and Taylor, 2004), albeit not consistently (Sadeh et al., 2008), suggesting that this effect is not robust. Few studies have previously investigated face-selective processing using EEG in WS, and not specifically the N170 component (Key and Dykens, 2016).

  • Using Event-Related Potentials and Startle to Evaluate Time Course in Anxiety and Depression

    2018, Biological Psychiatry: Cognitive Neuroscience and Neuroimaging
  • Cross-modal reorganization in cochlear implant users: Auditory cortex contributes to visual face processing

    2015, NeuroImage
    Citation Excerpt :

    This finding could not be replicated in the present study. However, a correlation analysis using the face-selectivity index (Sadeh et al., 2008), which expresses relation between the amplitude for faces and non-face objects, identified a clear relationship to face recognition abilities in the predicted direction, namely, higher face-selectivity was associated with better face recognition. Interestingly, this relationship was observed for NH controls but not for CI users.

  • Let's face it, from trial to trial: Comparing procedures for N170 single-trial estimation

    2012, NeuroImage
    Citation Excerpt :

    A well-known class- or condition-selective ERP is the N170 component, which is systematically related in amplitude and latency to the processing of visually-presented faces (Bentin et al., 1996; Sadeh et al., 2010). Importantly, the face-selectivity of the N170 ERP has been found to be valid (Sadeh et al., 2008), although little is known about its ST modulation as the majority of previous studies investigating N170 characteristics have exclusively focused on the trial-averaged ERP. Recently, Rousselet et al. (2007, 2009) presented the first ST face studies, but the authors did not focus on ST estimates to similar stimuli, but rather on global effects based on a large range of ‘ST’ features.

View all citing articles on Scopus
View full text