Pattern classification precedes region-average hemodynamic response in early visual cortex

Highlights

• High temporal resolution fMRI was applied while subjects viewed houses and faces.

• Temporal dynamics of average hemodynamic response (HR) and MVPA was compared.

• In V1, stimuli could be classified using MVPA before average HR left baseline.

• In V1, classification reached peak accuracy earlier than average HR.

• Early classification effects were restricted to early visual cortex.

Abstract

How quickly can information about the neural response to a visual stimulus be detected in the hemodynamic response measured using fMRI? Multi-voxel pattern analysis (MVPA) uses pattern classification to detect subtle stimulus-specific information from patterns of responses among voxels, including information that cannot be detected in the average response across a given brain region. Here we use MVPA in combination with rapid temporal sampling of the fMRI signal to investigate the temporal evolution of classification accuracy and its relationship to the average regional hemodynamic response. In primary visual cortex (V1) stimulus information can be detected in the pattern of voxel responses more than a second before the average hemodynamic response of V1 deviates from baseline, and classification accuracy peaks before the peak of the average hemodynamic response. Both of these effects are restricted to early visual cortex, with higher level areas showing no difference or, in some cases, the opposite temporal relationship. These results have methodological implications for fMRI studies using MVPA because they demonstrate that information can be decoded from hemodynamic activity more quickly than previously assumed.

Introduction

Although the neural activity in visual cortex associated with a visual task such as category recognition begins within a few hundred milliseconds after stimulus presentation (Rust and Dicarlo, 2010), associated change in blood flow measured with functional magnetic resonance imaging (fMRI), known as the hemodynamic response (HR), begins seconds later, and evolves over the course of several seconds, originating in a constrained spatial region, and spreading outward from that point while rising in amplitude (Shmuel et al., 2007). The complex spatiotemporal dynamics of the HR make it difficult to predict when the maximum amount of information about neural activity should be recoverable from the signal. Moreover, the time at which maximal information about brain activity is recoverable may not be the same in different types of analyses or different brain regions. Because knowledge of when maximal information is recoverable from fMRI data is of great utility for optimizing experimental designs and analyses, we undertook a systematic investigation of the timecourse of information availability in the HR.

There are two types of analyses commonly performed on fMRI data: univariate and multivariate. In univariate analyses, a general linear model (GLM) is applied to each voxel individually. By contrast, multivariate pattern analyses (MVPA) of fMRI data take into account relationships in the activity of multiple voxels. Several recent reports have begun to investigate the timecourse of MVPA classification accuracy during a range of cognitive tasks, showing that although the MVPA timecourse roughly tracks the region-average HR timecourse, classification accuracy can have temporal dynamics that differ from the region-average HR (Bode and Haynes, 2009, Greenberg et al., 2010). In fact, under certain conditions, accurate classification can persist even after the region-average HR has returned to baseline (Harrison and Tong, 2009).

We hypothesized that the reverse might also be the case, namely that HR patterns between voxels in a region would be able to support MVPA classification prior to a significant rise in the region-average HR. We will call this the “early onset” hypothesis. This hypothesis could be true if the HRs of individual voxels deviated reliably from baseline early in the timecourse, without being uniform enough to cause the region-wide average to deviate from baseline. Similarly, we hypothesized that peak classification would not always occur at the same time as the peak of the region-average HR. We will refer to this as the “early peak” hypothesis. In univariate analyses, peak region-average HR will by definition yield the largest difference between conditions and hence the largest effect size, but this is not the case for MVPA, where HR patterns could potentially contain more information about experimental conditions at timepoints before (or after) the peak region-average HR. This could be the case, for example, if the patterns were made less informative by the spatial spreading of blood through the capillary bed that may occur as the HR approaches its peak.

We tested early onset and early peak hypotheses in a number of functionally defined regions in visual cortex. In order to characterize the timecourse of the region-average HR and classification accuracy as precisely as possible, we collected fMRI data over the occipital and temporal lobes at a high temporal resolution (one acquisition = 739 ms), using a slow event-related design, while participants viewed pictures of faces and houses. We analyzed stimulus category classification at each acquisition and compared classification accuracy to the region-average HR within each predefined ROI. We find that both the onset (first above-chance classification) and peak (most statistically significant classification) of the MVPA analysis precede the onset (first significant deviation from baseline) and peak (most statistically significant increase from baseline) of the region-average HR in V1, but not in other visual areas.