Neural representation of object orientation: A dissociation between MVPA and Repetition Suppression
Introduction
Representing an object's orientation is important in a variety of circumstances. For example, picking up a hammer requires an accurate representation of the hammer's orientation. In addition, the orientations of objects may be crucial for interpreting a scene (e.g., a chair upright versus on its side), and orientation can also influence judgments about an object's stability (Cholewiak et al., 2013) and center of mass (Barnett-Cowan et al., 2011).
Behavioral evidence concerning the tendency to confuse orientations with one another may offer insights into how the brain represents object orientation. In particular, mirror image views of objects are especially prone to confusion (e.g., Bradshaw et al., 1976, Corballis and Beale, 1976, Farrell, 1979, Mello, 1965, Sekuler and Houlihan, 1968, Sutherland, 1957, Wolff, 1971). Mirror-image confusion is typically conceptualized as a tendency to confuse an image with its left–right reflection—that is, with its reflection across a vertical axis (Fig. 1A). However, recent behavioral research (Gregory and McCloskey, 2010, Gregory et al., 2011) has challenged this conception, arguing that in most studies object-based axes have been confounded with extrinsic (i.e., non-object-based) vertical axes. In Fig. 1A, for example, the object's primary axis of elongation is aligned with the Extrinsic Vertical Axis. Given this confounding, confusions involving reflection across an Extrinsic Vertical Axis (EVA reflection) could equally be described as reflections across the object's primary axis (OPA reflections; Fig. 1B).
Recent studies have dissociated EVA reflections from OPA reflections by presenting object stimuli at oblique orientations (Fig. 1C & D). Under these conditions, adult participants rarely made left–right EVA reflection errors (Fig. 1C), and instead made OPA reflections (Fig. 1D) more often than any other type of error (Gregory and McCloskey, 2010, Gregory et al., 2011). These results indicate the importance of distinguishing different mirror image relationships: OPA reflections are highly confusable, whereas EVA reflections are not. In addition, the findings raise new questions about the neural representation of object orientation. Does the greater behavioral confusability for OPA reflections relative to EVA reflections stem from the way object orientation is represented in the brain?
In this article we examine the neural representation of object orientation in the object-selective lateral occipital complex (LOC, Malach et al., 1995). In particular, we investigate the hypothesis that the behavioral confusability of orientations reflects the similarity of their representations in LOC. Recent work on the neural representation of mirror images is broadly consistent with this view. Neuroimaging studies have shown that cortical regions comprising the LOC represent mirror image views of objects and faces similarly (Dilks et al., 2011, Kietzmann et al., 2012, Axelrod and Yovel, 2012). These results are bolstered by comparable findings in Macaque IT, the putative homolog of LOC (Freiwald and Tsao, 2010, Rollenhagen and Olson, 2000). While potentially consistent with the view that confusable orientations are represented similarly in object-selective cortex, these studies either presented stimuli exclusively in “upright” orientations, or only used extrinsic (vertical or horizontal) axes of reflection. Such designs do not allow differentiation of OPA and EVA reflections, which differ in their degree of behavioral confusability.
In two fMRI experiments we investigated the neural representation of OPA reflections, EVA reflections, and a range of other orientation relationships in object-selective cortex (LOC). Experiment 1 used Repetition Suppression (RS) to ask whether OPA reflections are represented more similarly than EVA reflections. Experiment 2 investigated the representation of object orientation across a wider range of orientation relationships, including but not limited to OPA and EVA reflections. Using a continuous carry-over design (Aguirre, 2007), we measured the similarity of neural representations using both RS and multi-voxel pattern similarity (MVP-similarity). We compared these neural similarity measures to the behavioral confusability rates for those same orientations.
The use of both RS and MVP-similarity measures also afforded an opportunity to ask whether these methods capture different aspects of neural similarity. Although both methods are frequently used to address questions of representational similarity, previous studies have found that RS and MVPA do not always lead to the same conclusions (Drucker and Aguirre, 2009, Epstein and Morgan, 2012, Moore et al., 2013, Ward et al., 2013), raising pressing questions about what conclusions should be drawn in cases where the two methods diverge. In the General discussion we propose that RS and MVP-similarity are sensitive to different aspects of neural response, arguing that this hypothesis can explain our results as well as other dissociations between MVPA and RS.
Section snippets
Experiment 1
This experiment used RS to ask whether object-selective cortex differentiates OPA and EVA reflections. On each trial, participants viewed two object images involving 1) an identical repetition; 2) an OPA reflection; 3) an EVA reflection; or 4) two different objects (Fig. 2A). All objects were presented in oblique orientations (Fig. 2B) so that OPA reflections were always distinct from EVA reflections. If the behavioral confusability of orientations is related to the degree of RS, we expect
Experiment 2
Experiment 1 focused on determining whether and to what extent OPA and EVA reflections elicit RS. The design was not, however, suitable for MVP-similarity analyses, the other method of interest in the present study. In Experiment 1 we could not obtain multi-voxel patterns for individual stimulus orientations, because two stimulus images were presented on each trial, and the estimated beta values therefore reflected the response to two stimuli. Experiment 2 was designed to allow MVP-similarity
General discussion
Previous studies finding similar neural representations for mirror images (Axelrod and Yovel, 2012, Dilks et al., 2011, Freiwald and Tsao, 2010, Kietzmann et al., 2012, Rollenhagen and Olson, 2000) raised the possibility that object-selective cortex is sensitive to the behavioral confusability of different orientations of an object. The present study explored this possibility systematically. Using RS (Experiments 1 and 2) and MVP-similarity (Experiment 2), we assessed the similarity of neural
Conclusion
The present study found a clear dissociation between Repetition Suppression (RS) and multi-voxel pattern similarity (MVP-similarity) in region LO. This dissociation can be explained by a simple working hypothesis: MVP-similarity is sensitive to whether stimuli activate anatomically clustered neuronal populations, whereas RS is sensitive to whether they activate the same neuronal populations. This hypothesis not only accounts for our results but may also account for other apparent discrepancies
Acknowledgements
This work was supported by research funding from the Johns Hopkins University to S.P. We thank David Rothlein for providing the code used for the MVPA searchlight analysis, Jung Uk Kang and Harry Ngai for their assistance with data collection and analysis, and Kristen Johannes for constructive comments on the paper.
References (64)
Continuous carry-over designs for fMRI
NeuroImage
(2007)- et al.
The representation of object viewpoint in human visual cortex
NeuroImage
(2009) - et al.
Human parietal cortex in action
Curr. Opin. Neurobiol.
(2006) - et al.
What do differences between multi-voxel and univariate analysis mean? How subject-, voxel-, and trial-level variance impact fMRI analysis
NeuroImage
(2014) - et al.
Neural responses to visual scenes reveals inconsistencies between fMRI adaptation and multivoxel pattern analysis
Neuropsychologia
(2012) - et al.
fMR-adaptation reveals a distributed representation of inanimate objects and places in human visual cortex
NeuroImage
(2005) - et al.
Mirror-image confusions: implications for representation and processing of object orientation
Cognition
(2010) - et al.
fMR-adaptation: a tool for studying the functional properties of human cortical neurons
Acta Psychol.
(2001) - et al.
Differential processing of objects under various viewing conditions in the human lateral occipital complex
Neuron
(1999) - et al.
Repetition and the brain: neural models of stimulus-specific effects
Trends Cogn. Sci.
(2006)
Differential effects of viewpoint on object-driven activation in dorsal and ventral streams
Neuron
The ventral visual pathway: an expanded neural framework for the processing of object quality
Trends Cogn. Sci.
Targeting the functional properties of cortical neurons using fMR-adaptation
NeuroImage
Beyond the edges of a view: boundary extension in human scene-selective visual cortex
Neuron
The similarity structure of distributed neural responses reveals the multiple representations of letters
NeuroImage
How distributed is visual category information in human occipito-temporal cortex? An fMRI study
Neuron
Selectivity of inferior temporal neurons for realistic pictures predicted by algorithms for image database navigation
J. Neurophysiol.
Hierarchical processing of face viewpoint in human visual cortex
J. Neurosci.
Perceived object stability depends on multisensory estimates of gravity
PLoS One
Orientation-specific adaptation in human visual cortex
J. Neurosci.
The perception and identification of mirror-reversed patterns
Q. J. Exp. Psychol.
The psychophysics toolbox
Spat. Vis.
Visual perception of the physical stability of asymmetric three-dimensional objects
J. Vis.
The Psychology of Left and Right
Neural mechanisms for visual memory and their role in attention
Proc. Natl. Acad. Sci.
Mirror-image sensitivity and invariance in object and scene processing pathways
J. Neurosci.
Different spatial scales of shape similarity representation in lateral and ventral LOC
Cereb. Cortex
BOLD repetition decreases in object-responsive ventral visual areas depend on spatial attention
J. Neurophysiol.
Two kinds of fMRI repetition suppression? Evidence for dissociable neural mechanisms
J. Neurophysiol.
Orientation-tuned fMRI adaptation in human visual cortex
J. Neurophysiol.
Coding left and right
J. Exp. Psychol. Hum. Percept. Perform.
Functional compartmentalization and viewpoint generalization within the macaque face-processing system
Science
Cited by (20)
Skeletal representations of shape in the human visual cortex
2022, NeuropsychologiaIntegration and differentiation of hippocampal memory traces
2020, Neuroscience and Biobehavioral ReviewsFunctions of ventral visual cortex after bilateral medial temporal lobe damage
2020, Progress in NeurobiologyCitation Excerpt :This adaptation phenomenon (also called repetition suppression or repetition attenuation) can be considered a form of rapid learning and a signature of perceptual memory for previously viewed stimuli. Adaptation has been observed in many visual regions including the lateral occipital cortex (LOC) for repeated presentations of objects (e.g., Li et al., 1993; Grill-Spector et al., 1999; Grill-Spector and Malach, 2001; Anderson et al., 2008; Hatfield et al., 2016; Kim et al., 2009; Konen and Kastner, 2008) and in the parahippocampal place area (PPA) for repeated presentations of scenes (Epstein et al., 1999, 2008; Kim et al., 2015). Adaptation occurs not only when a stimulus is repeated immediately (with no intervening stimuli), but also after a time lag of several minutes or longer during which intervening stimuli are presented (e.g., van Turennout et al., 2000, 2003; Henson et al., 2000, 2004; Weiner et al., 2010; Brozinsky et al., 2005).
Representation of viewed and recalled film clips in patterns of brain activity in a person with developmental amnesia
2020, NeuropsychologiaCitation Excerpt :In a group of healthy participants presented repeatedly with visual scenes, Ward and colleagues (Ward et al., 2013) also reported a trade-off between repetition suppression and successful pattern classification in the fusiform and parahippocampal gyrus. Although they can be dissociated in some conditions, repetition suppression and pattern classification are also both sensitive to overlapping patterns of neural representations (Hatfield et al., 2016), and are used to characterize the sensitivities of cortical regions involved in perceptual and conceptual cognitive processes (Grill-Spector and Malach, 2001). Thus, repetition suppression and pattern similarity during perception are linked, and—as shown in our results—are both influenced by amnesia; however, their respective relationship with encoding and subsequent memory performance is not straightforward.
Representational differences between line drawings and photographs of natural scenes: A dissociation between multi-voxel pattern analysis and repetition suppression
2018, NeuropsychologiaCitation Excerpt :Park and Park (2017) found a signal for scene texture in the parahippocampal place area (PPA) using MVPA but not using RS. Recently, Hatfield et al. (2016) identified neural signatures of object orientations using MVPA but not RS in lateral occipital cortex, and proposed a framework to characterize the divergent results provided by RS and MVPA in light of the underlying neural activity thought to drive each effect. Specifically, they proposed that significant RS reflects co-activation of the same neural populations across stimulus repetitions while significant MVPA reflects activity in either identical populations of neurons or reliably clustered but not necessarily identical populations of neurons.