Abstract
We have built a phenomenological spiking model of the cat early visual system comprising the retina, the Lateral Geniculate Nucleus (LGN) and V1’s layer 4, and established four main results (1) When exposed to videos that reproduce with high fidelity what a cat experiences under natural conditions, adjacent Retinal Ganglion Cells (RGCs) have spike-time correlations at a short timescale (~30 ms), despite neuronal noise and possible jitter accumulation. (2) In accordance with recent experimental findings, the LGN filters out some noise. It thus increases the spike reliability and temporal precision, the sparsity, and, importantly, further decreases down to ~15 ms adjacent cells’ correlation timescale. (3) Downstream simple cells in V1’s layer 4, if equipped with Spike Timing-Dependent Plasticity (STDP), may detect these fine-scale cross-correlations, and thus connect principally to ON- and OFF-centre cells with Receptive Fields (RF) aligned in the visual space, and thereby become orientation selective, in accordance with Hubel and Wiesel (Journal of Physiology 160:106–154, 1962) classic model. Up to this point we dealt with continuous vision, and there was no absolute time reference such as a stimulus onset, yet information was encoded and decoded in the relative spike times. (4) We then simulated saccades to a static image and benchmarked relative spike time coding and time-to-first spike coding w.r.t. to saccade landing in the context of orientation representation. In both the retina and the LGN, relative spike times are more precise, less affected by pre-landing history and global contrast than absolute ones, and lead to robust contrast invariant orientation representations in V1.
Similar content being viewed by others
Abbreviations
- CGC:
-
Contrast Gain Control
- DoG:
-
Difference-of-Gaussian
- EPSP:
-
Excitatory Post-Synaptic Potential
- IPL:
-
Inner Plexiform Layer
- IPSP:
-
Inhibitory Post-Synaptic Potential
- LGN:
-
Lateral Geniculate Nucleus
- LTD:
-
Long Term Depression
- LTP:
-
Long Term Potentiation
- OPL:
-
Outer Plexiform Layer
- PSTH:
-
Post-Stimulus Time Histogram
- RF:
-
Receptive Field
- RGC:
-
Retinal Ganglion Cell
- SRM:
-
Spike Response Model
- STDP:
-
Spike Timing-Dependent Plasticity
- V1:
-
primary visual cortex (a.k.a. area 17).
References
Albrecht, D. G., Geisler, W. S., Frazor, R. A., & Crane, A. M. (2002). Visual cortex neurons of monkeys and cats: temporal dynamics of the contrast response function. Journal of Neurophysiology, 88(2), 888–913.
Babadi, B., Casti, A., Xiao, Y., Kaplan, E., & Paninski, L. (2010). A generalized linear model of the impact of direct and indirect inputs to the lateral geniculate nucleus. Journal of Vision, 10(10), 22.
Bacon-Mace, N., Mace, M. J., Fabre-Thorpe, M., & Thorpe, S. J. (2005). The time course of visual processing: Backward masking and natural scene categorisation. Vision Research, 45(11), 1459–69.
Barlow, H. (1961). Possible principles underlying the transformation of sensory messages. In Sensory communication (pp. 217–234). Cambridge: MIT. wa rosenblith edition.
Bell, A. J., & Sejnowski, T. J. (1997). The “independent components” of natural scenes are edge filters. Vision Research, 37(23), 3327–3338.
Berkes, P., & Wiskott, L. (2005). Slow feature analysis yields a rich repertoire of complex cell properties. Journal of Vision, 5(6), 579–602.
Betsch, B., Einhäuser, W., Körding, K., & König, P. (2004). The world from a cat’s perspective—statistics of natural videos. Biological Cybernetics, 90(1), 41–50.
Brette, R., & Guigon, E. (2003). Reliability of spike timing is a general property of spiking model neurons. Neural Computation, 15(2), 279–308.
Butts, D. A., Weng, C., Jin, J., Yeh, C.-I., Lesica, N. A., Alonso, J.-M., et al. (2007). Temporal precision in the neural code and the timescales of natural vision. Nature, 449(7158), 92–95.
Cai, D., DeAngelis, G. C., & Freeman, R. D. (1997). Spatiotemporal receptive field organization in the lateral geniculate nucleus of cats and kittens. Journal of Neurophysiology, 78(2), 1045–1061.
Caporale, N., & Dan, Y. (2008). Spike timing-dependent plasticity: a hebbian learning rule. Annual Review of Neuroscience, 31, 25–46.
Carandini, M., Horton, J. C., & Sincich, L. C. (2007). Thalamic filtering of retinal spike trains by postsynaptic summation. Journal of Vision, 7(14), 20.1–2011.
Celebrini, S., Thorpe, S., Trotter, Y., & Imbert, M. (1993). Dynamics of orientation coding in area V1 of the awake primate. Visual Neuroscience, 10(5), 811–825.
Chapman, B., Zahs, K. R., & Stryker, M. P. (1991). Relation of cortical cell orientation selectivity to alignment of receptive fields of the geniculocortical afferents that arborize within a single orientation column in ferret visual cortex. Journal of Neuroscience, 11(5), 1347–1358.
Chase, S. M., & Young, E. D. (2007). First-spike latency information in single neurons increases when referenced to population onset. Proceedings of the National Academy of Sciences of the United States of America, 104(12), 5175–5180.
Chung, S., & Ferster, D. (1998). Strength and orientation tuning of the thalamic input to simple cells revealed by electrically evoked cortical suppression. Neuron, 20(6), 1177–1189.
Coppola, D., & Purves, D. (1996). The extraordinarily rapid disappearance of entopic images. Proceedings of the National Academy of Sciences of the United States of America, 93(15), 8001–8004.
Crouzet, S. M., Kirchner, H., & Thorpe, S. J. (2010). Fast saccades toward faces: face detection in just 100 ms. Journal of Vision, 10(4), 1–17.
Delorme, A., Perrinet, L., Thorpe, S., & Samuelides, M. (2001). Networks of integrate-and-fire neurons using rank order coding B: spike timing dependent plasticity and emergence of orientation selectivity. Neurocomputing, 38–40, 539–545.
Delorme, A., & Thorpe, S. J. (2001). Face identification using one spike per neuron: resistance to image degradations. Neural Networks, 14(6–7), 795–803.
Desbordes, G., Jin, J., Weng, C., Lesica, N. A., Stanley, G. B., & Alonso, J.-M. (2008). Timing precision in population coding of natural scenes in the early visual system. PLoS Biology, 6(12), e324.
Einhäuser, W., Kayser, C., König, P., & Körding, K. P. (2002). Learning the invariance properties of complex cells from their responses to natural stimuli. European Journal of Neuroscience, 15(3), 475–486.
Enroth-Cugell, C., Robson, J. G., Schweitzer-Tong, D. E., & Watson, A. B. (1983). Spatio-temporal interactions in cat retinal ganglion cells showing linear spatial summation. The Journal of Physiology, 341, 279–307.
Fabre-Thorpe, M., Richard, G., & Thorpe, S. J. (1998). Rapid categorization of natural images by rhesus monkeys. NeuroReport, 9(2), 303–8.
Ferster, D., Chung, S., & Wheat, H. (1996). Orientation selectivity of thalamic input to simple cells of cat visual cortex. Nature, 380(6571), 249–252.
Földiák, P. (1991). Learning invariance from transformation sequences. Neural Computation, 3, 194–200.
Fukushima, K. (1980). Neocognitron: a self organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological Cybernetics, 36(4), 193–202.
Gawne, T., Kjaer, T., & Richmond, B. (1996). Latency: another potential code for feature binding in striate cortex. Journal of Neurophysiology, 76(2), 1356–1360.
Gerstner, W., Ritz, R., & van Hemmen, J. L. (1993). Why spikes? hebbian learning and retrieval of time-resolved excitation patterns. Biological Cybernetics, 69(5–6), 503–515.
Gilson, M., Masquelier, T., & Hugues, E. (2011). STDP allows fast rate-modulated coding with Poisson-like spike trains. PLoS Computational Biology (in press).
Girard, P., Jouffrais, C., & Kirchner, C. H. (2008). Ultra-rapid categorisation in non-human primates. Animal Cognition, 11(3), 485–493.
Gollisch, T., & Meister, M. (2008). Rapid neural coding in the retina with relative spike latencies. Science, 319(5866), 1108–1111.
Guyonneau, R., VanRullen, R., & Thorpe, S. (2005). Neurons tune to the earliest spikes through STDP. Neural Computation, 17(4), 859–879.
Hubel, D., & Wiesel, T. (1962). Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. Journal of Physiology, 160, 106–154.
Hung, C., Kreiman, G., Poggio, T., & DiCarlo, J. (2005). Fast readout of object identity from macaque inferior temporal cortex. Science, 310(5749), 863–866.
Hyvärinen, A., & Hoyer, P. O. (2001). A two-layer sparse coding model learns simple and complex cell receptive fields and topography from natural images. Vision Research, 41(18), 2413–2423.
Johansson, R. S., & Birznieks, I. (2004). First spikes in ensembles of human tactile afferents code complex spatial fingertip events. Nature Neuroscience, 7(2), 170–177.
Kara, P., Reinagel, P., & Reid, R. C. (2000). Low response variability in simultaneously recorded retinal, thalamic, and cortical neurons. Neuron, 27(3), 635–646.
Keat, J., Reinagel, P., Reid, R. C., & Meister, M. (2001). Predicting every spike: a model for the responses of visual neurons. Neuron, 30(3), 803–817.
Kempter, R., Gerstner, W., & van Hemmen, J. L. (1999). Hebbian learning and spiking neurons. Physical Review E, 59(4), 4498–4514.
Kirchner, H., & Thorpe, S. (2006). Ultra-rapid object detection with saccadic eye movements: visual processing speed revisited. Vision Research, 46(11), 1762–1776.
König, P., Engel, A. K., & Singer, W. (1996). Integrator or coincidence detector? The role of the cortical neuron revisited. Trends in Neurosciences, 19(4), 130–7.
Körding, K., Kayser, C., Einhäuser, W., & König, P. (2004). How are complex cell properties adapted to the statistics of natural stimuli? Journal of Neurophysiology, 91(1), 206–212.
LeCun, Y., & Bengio, Y. (1998). Convolutional networks for images, speech, and time series. In M. A. Arbib (Ed.), The handbook of brain theory and neural networks (pp. 255–258). Cambridge: MIT.
Lichtsteiner, P., Posch, C., & Delbruck, T. (2007). An 128 × 128 120db 15us-latency temporal contrast vision sensor. IEEE Journal of Solid-State Circuits, 43(2), 566–576.
Liu, H., Agam, Y., Madsen, J. R., & Kreiman, G. (2009). Timing, timing, timing: fast decoding of object information from intracranial field potentials in human visual cortex. Neuron, 62(2), 281–290.
Martinez-Conde, S., Macknik, S. L., & Hubel, D. H. (2004). The role of fixational eye movements in visual perception. Nature Reviews Neuroscience, 5(3), 229–240.
Masquelier, T., Guyonneau, R., & Thorpe, S. J. (2008). Spike timing dependent plasticity finds the start of repeating patterns in continuous spike trains. PloS One, 3(1), e1377.
Masquelier, T., Guyonneau, R., & Thorpe, S. J. (2009). Competitive STDP-based spike pattern learning. Neural Computation, 21(5), 1259–1276.
Masquelier, T., Hugues, E., Deco, G., & Thorpe, S. J. (2009). Oscillations, phase-of-firing coding, and spike timing-dependent plasticity: an efficient learning scheme. Journal of Neuroscience, 29(43), 13484–13493.
Masquelier, T., Serre, T., Thorpe, S., & Poggio, T. (2007). Learning complex cell invariance from natural videos: a plausibility proof. Massachusetts Institute of Technology, CBCL Paper #269/MIT-CSAIL-TR #2007-060.
Masquelier, T., & Thorpe, S. J. (2007). Unsupervised learning of visual features through spike timing dependent plasticity. PLoS Computational Biology, 3(2), e31.
Miller, K. D., & MacKay, D. J. C. (1994). The role of constraints in hebbian learning. Neural Computation, 6, 100–126.
Montemurro, M. A., Rasch, M. J., Murayama, Y., Logothetis, N. K., & Panzeri, S. (2008). Phase-of-firing coding of natural visual stimuli in primary visual cortex. Current Biology, 18(5), 375–380.
Olshausen, B. A., & Field, D. J. (1996). Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 381, 607–609.
Oram, M., & Perrett, D. (1992). Time course of neural responses discriminating different views of the face and head. Journal of Neurophysiology, 68(1), 70–84.
Panzeri, S., & Diamond, M. E. (2010). Information carried by population spike times in the whisker sensory cortex can be decoded without knowledge of stimulus time. Frontiers in Synaptic Neuroscience, 2(17), 1–14.
Puchalla, J. L., Schneidman, E., Harris, R. A., & Berry, M. J. (2005). Redundancy in the population code of the retina. Neuron, 46(3), 493–504.
Rathbun, D. L., Warland, D. K., & Usrey, W. M. (2010). Spike timing and information transmission at retinogeniculate synapses. Journal of Neuroscience, 30(41), 13558–13566.
Rehn, M., & Sommer, F. T. (2007). A network that uses few active neurones to code visual input predicts the diverse shapes of cortical receptive fields. Journal of Computational Neuroscience, 22(2), 135–146.
Riesenhuber, M., & Poggio, T. (1999). Hierarchical models of object recognition in cortex. Nature Neuroscience, 2(11), 1019–1025.
Rolls, E., & Milward, T. (2000). A model of invariant object recognition in the visual system: learning rules, activation functions, lateral inhibition, and information-based performance measures. Neural Computation, 12(11), 2547–2572.
Rousselet, G. A., Fabre-Thorpe, M., & Thorpe, S. J. (2002). Parallel processing in high-level categorization of natural images. Nature Neuroscience, 5(7), 629–30.
Serre, T., Oliva, A., & Poggio, T. (2007). A feedforward architecture accounts for rapid categorization. Proc. Nat. Acad. Sci. USA, 104(15).
Singer, W., Tretter, F., & Cynader, M. (1975). Organization of cat striate cortex: a correlation of receptive-field properties with afferent and efferent connections. Journal of Neurophysiology, 38(5), 1080–1098.
Song, S., Miller, K., & Abbott, L. (2000). Competitive hebbian learning through spike-timing-dependent synaptic plasticity. Nature Neuroscience, 3(9), 919–926.
Spratling, M. (2005). Learning viewpoint invariant perceptual representations from cluttered images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(5).
Stevenson, I. H., & Kording, K. P. (2011). How advances in neural recording affect data analysis. Nature Neuroscience, 14(2), 139–142.
Stewart, N., Brown, G. D. A., & Chater, N. (2005). Absolute identification by relative judgment. Psychological Review, 112(4), 881–911.
Stone, J. (1965). A quantitative analysis of the distribution of ganglion cells in the cat’s retina. The Journal of Comparative Neurology, 124(3), 337–352.
Stringer, S., & Rolls, E. (2000). Position invariant recognition in the visual system with cluttered environments. Neural Networks, 13(3), 305–315.
Thorpe, S., Fize, D., & Marlot, C. (1996). Speed of processing in the human visual system. Nature, 381(6582), 520–2.
Thorpe, S., & Gautrais, J. (1998). Rank order coding. In J. M. Bower (Ed.), Computational neuroscience: Trends in research (pp. 113–118). New York: Plenum.
Thorpe, S., & Imbert, M. (1989). Biological constraints on connectionist modelling. In R. Pfeifer, Z. Schreter, F. Fogelman-Soulie, & L. Steels (Eds.), Connectionism in perspective (pp. 63–92). Amsterdam: Elsevier.
Ullman, S., Vidal-Naquet, M., & Sali, E. (2002). Visual features of intermediate complexity and their use in classification. Nature Neuroscience, 5(7), 682–687.
van Hateren, J. H., & Ruderman, D. L. (1998). Independent component analysis of natural image sequences yields spatio-temporal filters similar to simple cells in primary visual cortex. Proceedings. Biological sciences / The Royal Society, 265(1412), 2315–2320.
van Hateren, J. H., & van der Schaaf, A. (1998). Independent component filters of natural images compared with simple cells in primary visual cortex. Proceedings. Biological sciences / The Royal Society, 265(1394), 359–366.
van Rossum, M. C., Bi, G. Q., & Turrigiano, G. G. (2000). Stable hebbian learning from spike timing-dependent plasticity. Journal of Neuroscience, 20(23), 8812–8821.
VanRullen, R., Gautrais, J., Delorme, A., & Thorpe, S. (1998). Face processing using one spike per neurone. Biosystems, 48(1–3), 229–239.
VanRullen, R., & Thorpe, S. (2001). Rate coding versus temporal order coding: what the retinal ganglion cells tell the visual cortex. Neural Computation, 13(6), 1255–1283.
VanRullen, R., & Thorpe, S. (2002). Surfing a spike wave down the ventral stream. Vision Research, 42(23), 2593–2615.
Vinje, W. E., & Gallant, J. L. (2000). Sparse coding and decorrelation in primary visual cortex during natural vision. Science, 287(5456), 1273–1276.
Wallis, G., & Rolls, E. (1997). Invariant face and object recognition in the visual system. Progress in Neurobiology, 51(2), 167–194.
Williams, P. E., Mechler, F., Gordon, J., Shapley, R., & Hawken, M. J. (2004). Entrainment to video displays in primary visual cortex of macaque and humans. Journal of Neuroscience, 24(38), 8278–8288.
Wilson, J. R., & Sherman, S. M. (1976). Receptive-field characteristics of neurons in cat striate cortex: Changes with visual field eccentricity. Journal of Neurophysiology, 39(3), 512–533.
Wiskott, L., & Sejnowski, T. J. (2002). Slow feature analysis: unsupervised learning of invariances. Neural Computation, 14(4), 715–770.
Wohrer, A. (2008). Model and large-scale simulator of a biological retina, with contrast gain control. PhD thesis, University of Nice-Sophia Antipolis.
Wohrer, A., & Kornprobst, P. (2009). Virtual retina: a biological retina model and simulator, with contrast gain control. Journal of Computational Neuroscience, 26(2), 219–249.
Wörgötter, F., Nelle, E., Li, B., & Funke, K. (1998). The influence of corticofugal feedback on the temporal structure of visual responses of cat thalamic relay cells. The Journal of Physiology, 509(Pt 3), 797–815.
Zamarreño-Ramos, C., Camuñas-Mesa, L., Perez-Carrasco, J. A., Masquelier, T., Serrano-Gotarredona, T., & Linares-Barranco, B. (2011). On spike-timing-dependent-plasticity, memristive devices, and building a self-learning visual cortex. Front. Neurosc.—Neuromorph. Eng., 5(26).
Acknowledgements
This research was supported by the Fyssen Foundation and the FP7 European Project Coronet. We thank Adrien Wohrer for having developed the Virtual Retina simulator (Wohrer and Kornprobst 2009), both user-friendly and highly configurable, and for the quality of his support. We also thank Wolfgang Einhäuser for kindly providing the videos used in (Betsch et al. 2004), and Mario Pannunzi for the numerous insightful brainstorms we had.
Author information
Authors and Affiliations
Corresponding author
Additional information
Action Editor: Ken Miller
Rights and permissions
About this article
Cite this article
Masquelier, T. Relative spike time coding and STDP-based orientation selectivity in the early visual system in natural continuous and saccadic vision: a computational model. J Comput Neurosci 32, 425–441 (2012). https://doi.org/10.1007/s10827-011-0361-9
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10827-011-0361-9