Elsevier

NeuroImage

Volume 139, 1 October 2016, Pages 136-148
NeuroImage

Neural representation of object orientation: A dissociation between MVPA and Repetition Suppression

https://doi.org/10.1016/j.neuroimage.2016.05.052Get rights and content

Highlights

  • MVP-similarity in LO is sensitive to confusability of object orientations.

  • RS in LO is not affected by the confusability of different orientations.

  • Results are interpreted with new hypothesis about the neural bases of MVPA and RS.

Abstract

How is object orientation represented in the brain? Behavioral error patterns reveal systematic tendencies to confuse certain orientations with one another. Using fMRI, we asked whether more confusable orientations are represented more similarly in object selective cortex (LOC). We compared two widely-used measures of neural similarity: multi-voxel pattern similarity (MVP-similarity) and Repetition Suppression. In LO, we found that multi-voxel pattern similarity was predicted by the confusability of two orientations. By contrast, Repetition Suppression effects in LO were unrelated to the confusability of orientations. To account for these differences between MVP-similarity and Repetition Suppression, we propose that MVP-similarity reflects the topographical distribution of neural populations, whereas Repetition Suppression depends on repeated activation of particular groups of neurons. This hypothesis leads to a unified interpretation of our results and may explain other dissociations between MVPA and Repetition Suppression observed in the literature.

Introduction

Representing an object's orientation is important in a variety of circumstances. For example, picking up a hammer requires an accurate representation of the hammer's orientation. In addition, the orientations of objects may be crucial for interpreting a scene (e.g., a chair upright versus on its side), and orientation can also influence judgments about an object's stability (Cholewiak et al., 2013) and center of mass (Barnett-Cowan et al., 2011).

Behavioral evidence concerning the tendency to confuse orientations with one another may offer insights into how the brain represents object orientation. In particular, mirror image views of objects are especially prone to confusion (e.g., Bradshaw et al., 1976, Corballis and Beale, 1976, Farrell, 1979, Mello, 1965, Sekuler and Houlihan, 1968, Sutherland, 1957, Wolff, 1971). Mirror-image confusion is typically conceptualized as a tendency to confuse an image with its left–right reflection—that is, with its reflection across a vertical axis (Fig. 1A). However, recent behavioral research (Gregory and McCloskey, 2010, Gregory et al., 2011) has challenged this conception, arguing that in most studies object-based axes have been confounded with extrinsic (i.e., non-object-based) vertical axes. In Fig. 1A, for example, the object's primary axis of elongation is aligned with the Extrinsic Vertical Axis. Given this confounding, confusions involving reflection across an Extrinsic Vertical Axis (EVA reflection) could equally be described as reflections across the object's primary axis (OPA reflections; Fig. 1B).

Recent studies have dissociated EVA reflections from OPA reflections by presenting object stimuli at oblique orientations (Fig. 1C & D). Under these conditions, adult participants rarely made left–right EVA reflection errors (Fig. 1C), and instead made OPA reflections (Fig. 1D) more often than any other type of error (Gregory and McCloskey, 2010, Gregory et al., 2011). These results indicate the importance of distinguishing different mirror image relationships: OPA reflections are highly confusable, whereas EVA reflections are not. In addition, the findings raise new questions about the neural representation of object orientation. Does the greater behavioral confusability for OPA reflections relative to EVA reflections stem from the way object orientation is represented in the brain?

In this article we examine the neural representation of object orientation in the object-selective lateral occipital complex (LOC, Malach et al., 1995). In particular, we investigate the hypothesis that the behavioral confusability of orientations reflects the similarity of their representations in LOC. Recent work on the neural representation of mirror images is broadly consistent with this view. Neuroimaging studies have shown that cortical regions comprising the LOC represent mirror image views of objects and faces similarly (Dilks et al., 2011, Kietzmann et al., 2012, Axelrod and Yovel, 2012). These results are bolstered by comparable findings in Macaque IT, the putative homolog of LOC (Freiwald and Tsao, 2010, Rollenhagen and Olson, 2000). While potentially consistent with the view that confusable orientations are represented similarly in object-selective cortex, these studies either presented stimuli exclusively in “upright” orientations, or only used extrinsic (vertical or horizontal) axes of reflection. Such designs do not allow differentiation of OPA and EVA reflections, which differ in their degree of behavioral confusability.

In two fMRI experiments we investigated the neural representation of OPA reflections, EVA reflections, and a range of other orientation relationships in object-selective cortex (LOC). Experiment 1 used Repetition Suppression (RS) to ask whether OPA reflections are represented more similarly than EVA reflections. Experiment 2 investigated the representation of object orientation across a wider range of orientation relationships, including but not limited to OPA and EVA reflections. Using a continuous carry-over design (Aguirre, 2007), we measured the similarity of neural representations using both RS and multi-voxel pattern similarity (MVP-similarity). We compared these neural similarity measures to the behavioral confusability rates for those same orientations.

The use of both RS and MVP-similarity measures also afforded an opportunity to ask whether these methods capture different aspects of neural similarity. Although both methods are frequently used to address questions of representational similarity, previous studies have found that RS and MVPA do not always lead to the same conclusions (Drucker and Aguirre, 2009, Epstein and Morgan, 2012, Moore et al., 2013, Ward et al., 2013), raising pressing questions about what conclusions should be drawn in cases where the two methods diverge. In the General discussion we propose that RS and MVP-similarity are sensitive to different aspects of neural response, arguing that this hypothesis can explain our results as well as other dissociations between MVPA and RS.

Section snippets

Experiment 1

This experiment used RS to ask whether object-selective cortex differentiates OPA and EVA reflections. On each trial, participants viewed two object images involving 1) an identical repetition; 2) an OPA reflection; 3) an EVA reflection; or 4) two different objects (Fig. 2A). All objects were presented in oblique orientations (Fig. 2B) so that OPA reflections were always distinct from EVA reflections. If the behavioral confusability of orientations is related to the degree of RS, we expect

Experiment 2

Experiment 1 focused on determining whether and to what extent OPA and EVA reflections elicit RS. The design was not, however, suitable for MVP-similarity analyses, the other method of interest in the present study. In Experiment 1 we could not obtain multi-voxel patterns for individual stimulus orientations, because two stimulus images were presented on each trial, and the estimated beta values therefore reflected the response to two stimuli. Experiment 2 was designed to allow MVP-similarity

General discussion

Previous studies finding similar neural representations for mirror images (Axelrod and Yovel, 2012, Dilks et al., 2011, Freiwald and Tsao, 2010, Kietzmann et al., 2012, Rollenhagen and Olson, 2000) raised the possibility that object-selective cortex is sensitive to the behavioral confusability of different orientations of an object. The present study explored this possibility systematically. Using RS (Experiments 1 and 2) and MVP-similarity (Experiment 2), we assessed the similarity of neural

Conclusion

The present study found a clear dissociation between Repetition Suppression (RS) and multi-voxel pattern similarity (MVP-similarity) in region LO. This dissociation can be explained by a simple working hypothesis: MVP-similarity is sensitive to whether stimuli activate anatomically clustered neuronal populations, whereas RS is sensitive to whether they activate the same neuronal populations. This hypothesis not only accounts for our results but may also account for other apparent discrepancies

Acknowledgements

This work was supported by research funding from the Johns Hopkins University to S.P. We thank David Rothlein for providing the code used for the MVPA searchlight analysis, Jung Uk Kang and Harry Ngai for their assistance with data collection and analysis, and Kristen Johannes for constructive comments on the paper.

References (64)

  • T.W. James et al.

    Differential effects of viewpoint on object-driven activation in dorsal and ventral streams

    Neuron

    (2002)
  • D.J. Kravitz et al.

    The ventral visual pathway: an expanded neural framework for the processing of object quality

    Trends Cogn. Sci.

    (2013)
  • R. Malach

    Targeting the functional properties of cortical neurons using fMR-adaptation

    NeuroImage

    (2012)
  • S. Park et al.

    Beyond the edges of a view: boundary extension in human scene-selective visual cortex

    Neuron

    (2007)
  • D. Rothlein et al.

    The similarity structure of distributed neural responses reveals the multiple representations of letters

    NeuroImage

    (2014)
  • M. Spiridon et al.

    How distributed is visual category information in human occipito-temporal cortex? An fMRI study

    Neuron

    (2002)
  • S. Allred et al.

    Selectivity of inferior temporal neurons for realistic pictures predicted by algorithms for image database navigation

    J. Neurophysiol.

    (2005)
  • V. Axelrod et al.

    Hierarchical processing of face viewpoint in human visual cortex

    J. Neurosci.

    (2012)
  • M. Barnett-Cowan et al.

    Perceived object stability depends on multisensory estimates of gravity

    PLoS One

    (2011)
  • G.M. Boynton et al.

    Orientation-specific adaptation in human visual cortex

    J. Neurosci.

    (2003)
  • J. Bradshaw et al.

    The perception and identification of mirror-reversed patterns

    Q. J. Exp. Psychol.

    (1976)
  • D.H. Brainard

    The psychophysics toolbox

    Spat. Vis.

    (1997)
  • S.A. Cholewiak et al.

    Visual perception of the physical stability of asymmetric three-dimensional objects

    J. Vis.

    (2013)
  • M.C. Corballis et al.

    The Psychology of Left and Right

    (1976)
  • R. Desimone

    Neural mechanisms for visual memory and their role in attention

    Proc. Natl. Acad. Sci.

    (1996)
  • D.D. Dilks et al.

    Mirror-image sensitivity and invariance in object and scene processing pathways

    J. Neurosci.

    (2011)
  • D.M. Drucker et al.

    Different spatial scales of shape similarity representation in lateral and ventral LOC

    Cereb. Cortex

    (2009)
  • E. Eger et al.

    BOLD repetition decreases in object-responsive ventral visual areas depend on spatial attention

    J. Neurophysiol.

    (2004)
  • R.A. Epstein et al.

    Two kinds of fMRI repetition suppression? Evidence for dissociable neural mechanisms

    J. Neurophysiol.

    (2008)
  • F. Fang et al.

    Orientation-tuned fMRI adaptation in human visual cortex

    J. Neurophysiol.

    (2005)
  • W.S. Farrell

    Coding left and right

    J. Exp. Psychol. Hum. Percept. Perform.

    (1979)
  • W.A. Freiwald et al.

    Functional compartmentalization and viewpoint generalization within the macaque face-processing system

    Science

    (2010)
  • Cited by (20)

    • Integration and differentiation of hippocampal memory traces

      2020, Neuroscience and Biobehavioral Reviews
    • Functions of ventral visual cortex after bilateral medial temporal lobe damage

      2020, Progress in Neurobiology
      Citation Excerpt :

      This adaptation phenomenon (also called repetition suppression or repetition attenuation) can be considered a form of rapid learning and a signature of perceptual memory for previously viewed stimuli. Adaptation has been observed in many visual regions including the lateral occipital cortex (LOC) for repeated presentations of objects (e.g., Li et al., 1993; Grill-Spector et al., 1999; Grill-Spector and Malach, 2001; Anderson et al., 2008; Hatfield et al., 2016; Kim et al., 2009; Konen and Kastner, 2008) and in the parahippocampal place area (PPA) for repeated presentations of scenes (Epstein et al., 1999, 2008; Kim et al., 2015). Adaptation occurs not only when a stimulus is repeated immediately (with no intervening stimuli), but also after a time lag of several minutes or longer during which intervening stimuli are presented (e.g., van Turennout et al., 2000, 2003; Henson et al., 2000, 2004; Weiner et al., 2010; Brozinsky et al., 2005).

    • Representation of viewed and recalled film clips in patterns of brain activity in a person with developmental amnesia

      2020, Neuropsychologia
      Citation Excerpt :

      In a group of healthy participants presented repeatedly with visual scenes, Ward and colleagues (Ward et al., 2013) also reported a trade-off between repetition suppression and successful pattern classification in the fusiform and parahippocampal gyrus. Although they can be dissociated in some conditions, repetition suppression and pattern classification are also both sensitive to overlapping patterns of neural representations (Hatfield et al., 2016), and are used to characterize the sensitivities of cortical regions involved in perceptual and conceptual cognitive processes (Grill-Spector and Malach, 2001). Thus, repetition suppression and pattern similarity during perception are linked, and—as shown in our results—are both influenced by amnesia; however, their respective relationship with encoding and subsequent memory performance is not straightforward.

    • Representational differences between line drawings and photographs of natural scenes: A dissociation between multi-voxel pattern analysis and repetition suppression

      2018, Neuropsychologia
      Citation Excerpt :

      Park and Park (2017) found a signal for scene texture in the parahippocampal place area (PPA) using MVPA but not using RS. Recently, Hatfield et al. (2016) identified neural signatures of object orientations using MVPA but not RS in lateral occipital cortex, and proposed a framework to characterize the divergent results provided by RS and MVPA in light of the underlying neural activity thought to drive each effect. Specifically, they proposed that significant RS reflects co-activation of the same neural populations across stimulus repetitions while significant MVPA reflects activity in either identical populations of neurons or reliably clustered but not necessarily identical populations of neurons.

    View all citing articles on Scopus
    View full text