Abstract
How are visual scenes represented in the brain during categorization? We acquired magnetoencephalography (MEG) data from nine healthy subjects who participated in a rapid natural scene categorization task. Scenes were presented in two different perspectives (aerial vs. terrestrial) and two different orientations (upright vs. inverted). We applied multivariate pattern classification to categorize scene categories from computational (spatial envelope (SpEn): [6]) and neural representations (MEG responses). Predictions of both types of classifiers (1) exceeded chance but performed worse than human subjects, and (2) were significantly correlated in their pattern of predictions, suggesting the relevance of low-level visual features during scene categorization. In general, the pattern of predictions and errors were not correlated with behavioral predictions. We also examined the influence of perspective and orientation on neural and computational representations by studying the generalization performance of classifiers across perspective and orientation. We compared within-perspective-and-orientation classifiers (trained and tested on the same perspective and orientation) with across-perspective (trained on one perspective and tested on another) and across-orientation classifiers (trained on one orientation and tested on another). We report several interesting effects on category-level and identity-level (dis)agreement between neural, computational, and behavioral ”views”. To our knowledge, this is the first study to examine natural scene perception across scene perspectives and orientations from neural, computational, and behavioral angles.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Henderson, J.M., Hollingworth, A.: High-level scene perception. Annual Review of Psychology 50, 243–271 (1999)
Potter, M.C.: Meaning in visual scenes. Science 187, 965–966 (1975)
Oliva, A., Schyns, P.: Coarse blobs or fine edges? Evidence that information diagnosticity changes the perception of complex visual stimuli. Cognitive Psychology 34, 72–107 (1997)
Oliva, A., Schyns, P.: Diagnostic colors mediate scene recognition. Cognitive Psychology 41, 176–210 (2000)
Ehinger, K.A., Xiao, J., Torralba, A., Oliva, A.: Estimating scene typicality from human ratings and image features. In: Proceedings of the 33rd Annual Cognitive Science Conference, Boston (2011)
Oliva, A., Torralba, A.: Modeling the shape of the scene: A holistic representation of the spatial envelope. Intl. J. Comp. Vis. 42, 145–175 (2001)
Greene, M.R., Oliva, A.: The briefest of glances: the time course of natural scene understanding. Psychological Science 20, 464–472 (2009)
Loschky, L.C., Larson, A.M.: The natural/man-made distinction is made prior to basic-level distinctions in scene gist processing. Visual Cognition 18, 513–536 (2010)
Taulu, S., Simola, J.: Spatiotemporal signal space separation method for rejecting nearby interference in MEG measurements. Phys. Med. Biol. 41, 1759–1768 (2006)
Loschky, L., Ellis, K., Sears, T., Ringer, R., Davis, J.: Broadening the Horizons of Scene Gist Recognition: Aerial and Ground-based Views. J. Vis. 10, 1238 (2010)
Walther, D.B., Caddigan, E., Fei-Fei, L., Beck, D.M.: Natural scene categories revealed in distributed patterns of activity in the human brain. J. Neurosci. 29, 10573–10581 (2009)
Peelen, M., Fei-Fei, L., Kastner, S.: Neural mechanisms of rapid scene categorization in human visual cortex. Nature 460, 94–97 (2009)
Walther, D.B., Chai, B., Caddigan, E., Beck, D.M., Fei-Fei, L.: Simple line drawings suffice for fMRI decoding of natural scene categories. PNAS USA 108, 9661–9666 (2011)
Kravitz, D.J., Peng, C.S., Baker, C.I.: Real-World Scene Representations in High-Level Visual Cortex: It’s the Spaces More Than the Places. J. Neurosci. 31, 7322–7333 (2011)
Epstein, R.A., Harris, A., Stanley, D., Kanwisher, N.: The parahippocampal place area: recognition, navigation, or encoding? Neuron 23, 115–125 (1999)
Epstein, R.A., Higgins, J.S.: Differential parahippocampal and retrosplenial involvement in three types of visual scene recognition. Cereb. Cortex 17, 1680–1693 (2007)
Park, S., Brady, T.F., Greene, M.R., Oliva, A.: Disentangling scene content from spatial boundary: complementary roles for the parahippocampal place area and lateral occipital complex in representing real-world scenes. J. Neurosci. 31, 1333–1340 (2011)
MacEvoy, S.P., Epstein, R.A.: Constructing scenes from objects in human occipitotemporal cortex. Nat. Neurosci. 14, 1323–1329 (2011)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ramkumar, P., Pannasch, S., Hansen, B.C., Larson, A.M., Loschky, L.C. (2012). How Does the Brain Represent Visual Scenes? A Neuromagnetic Scene Categorization Study. In: Langs, G., Rish, I., Grosse-Wentrup, M., Murphy, B. (eds) Machine Learning and Interpretation in Neuroimaging. Lecture Notes in Computer Science(), vol 7263. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34713-9_12
Download citation
DOI: https://doi.org/10.1007/978-3-642-34713-9_12
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-34712-2
Online ISBN: 978-3-642-34713-9
eBook Packages: Computer ScienceComputer Science (R0)