Skip to main content
Log in

Relative spike time coding and STDP-based orientation selectivity in the early visual system in natural continuous and saccadic vision: a computational model

  • Published:
Journal of Computational Neuroscience Aims and scope Submit manuscript

Abstract

We have built a phenomenological spiking model of the cat early visual system comprising the retina, the Lateral Geniculate Nucleus (LGN) and V1’s layer 4, and established four main results (1) When exposed to videos that reproduce with high fidelity what a cat experiences under natural conditions, adjacent Retinal Ganglion Cells (RGCs) have spike-time correlations at a short timescale (~30 ms), despite neuronal noise and possible jitter accumulation. (2) In accordance with recent experimental findings, the LGN filters out some noise. It thus increases the spike reliability and temporal precision, the sparsity, and, importantly, further decreases down to ~15 ms adjacent cells’ correlation timescale. (3) Downstream simple cells in V1’s layer 4, if equipped with Spike Timing-Dependent Plasticity (STDP), may detect these fine-scale cross-correlations, and thus connect principally to ON- and OFF-centre cells with Receptive Fields (RF) aligned in the visual space, and thereby become orientation selective, in accordance with Hubel and Wiesel (Journal of Physiology 160:106–154, 1962) classic model. Up to this point we dealt with continuous vision, and there was no absolute time reference such as a stimulus onset, yet information was encoded and decoded in the relative spike times. (4) We then simulated saccades to a static image and benchmarked relative spike time coding and time-to-first spike coding w.r.t. to saccade landing in the context of orientation representation. In both the retina and the LGN, relative spike times are more precise, less affected by pre-landing history and global contrast than absolute ones, and lead to robust contrast invariant orientation representations in V1.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Abbreviations

CGC:

Contrast Gain Control

DoG:

Difference-of-Gaussian

EPSP:

Excitatory Post-Synaptic Potential

IPL:

Inner Plexiform Layer

IPSP:

Inhibitory Post-Synaptic Potential

LGN:

Lateral Geniculate Nucleus

LTD:

Long Term Depression

LTP:

Long Term Potentiation

OPL:

Outer Plexiform Layer

PSTH:

Post-Stimulus Time Histogram

RF:

Receptive Field

RGC:

Retinal Ganglion Cell

SRM:

Spike Response Model

STDP:

Spike Timing-Dependent Plasticity

V1:

primary visual cortex (a.k.a. area 17).

References

  • Albrecht, D. G., Geisler, W. S., Frazor, R. A., & Crane, A. M. (2002). Visual cortex neurons of monkeys and cats: temporal dynamics of the contrast response function. Journal of Neurophysiology, 88(2), 888–913.

    PubMed  Google Scholar 

  • Babadi, B., Casti, A., Xiao, Y., Kaplan, E., & Paninski, L. (2010). A generalized linear model of the impact of direct and indirect inputs to the lateral geniculate nucleus. Journal of Vision, 10(10), 22.

    Article  PubMed  Google Scholar 

  • Bacon-Mace, N., Mace, M. J., Fabre-Thorpe, M., & Thorpe, S. J. (2005). The time course of visual processing: Backward masking and natural scene categorisation. Vision Research, 45(11), 1459–69.

    Article  PubMed  Google Scholar 

  • Barlow, H. (1961). Possible principles underlying the transformation of sensory messages. In Sensory communication (pp. 217–234). Cambridge: MIT. wa rosenblith edition.

    Google Scholar 

  • Bell, A. J., & Sejnowski, T. J. (1997). The “independent components” of natural scenes are edge filters. Vision Research, 37(23), 3327–3338.

    Article  PubMed  CAS  Google Scholar 

  • Berkes, P., & Wiskott, L. (2005). Slow feature analysis yields a rich repertoire of complex cell properties. Journal of Vision, 5(6), 579–602.

    Article  PubMed  Google Scholar 

  • Betsch, B., Einhäuser, W., Körding, K., & König, P. (2004). The world from a cat’s perspective—statistics of natural videos. Biological Cybernetics, 90(1), 41–50.

    Article  PubMed  Google Scholar 

  • Brette, R., & Guigon, E. (2003). Reliability of spike timing is a general property of spiking model neurons. Neural Computation, 15(2), 279–308.

    Article  PubMed  Google Scholar 

  • Butts, D. A., Weng, C., Jin, J., Yeh, C.-I., Lesica, N. A., Alonso, J.-M., et al. (2007). Temporal precision in the neural code and the timescales of natural vision. Nature, 449(7158), 92–95.

    Article  PubMed  CAS  Google Scholar 

  • Cai, D., DeAngelis, G. C., & Freeman, R. D. (1997). Spatiotemporal receptive field organization in the lateral geniculate nucleus of cats and kittens. Journal of Neurophysiology, 78(2), 1045–1061.

    PubMed  CAS  Google Scholar 

  • Caporale, N., & Dan, Y. (2008). Spike timing-dependent plasticity: a hebbian learning rule. Annual Review of Neuroscience, 31, 25–46.

    Article  PubMed  CAS  Google Scholar 

  • Carandini, M., Horton, J. C., & Sincich, L. C. (2007). Thalamic filtering of retinal spike trains by postsynaptic summation. Journal of Vision, 7(14), 20.1–2011.

    Article  Google Scholar 

  • Celebrini, S., Thorpe, S., Trotter, Y., & Imbert, M. (1993). Dynamics of orientation coding in area V1 of the awake primate. Visual Neuroscience, 10(5), 811–825.

    Article  PubMed  CAS  Google Scholar 

  • Chapman, B., Zahs, K. R., & Stryker, M. P. (1991). Relation of cortical cell orientation selectivity to alignment of receptive fields of the geniculocortical afferents that arborize within a single orientation column in ferret visual cortex. Journal of Neuroscience, 11(5), 1347–1358.

    PubMed  CAS  Google Scholar 

  • Chase, S. M., & Young, E. D. (2007). First-spike latency information in single neurons increases when referenced to population onset. Proceedings of the National Academy of Sciences of the United States of America, 104(12), 5175–5180.

    Article  PubMed  CAS  Google Scholar 

  • Chung, S., & Ferster, D. (1998). Strength and orientation tuning of the thalamic input to simple cells revealed by electrically evoked cortical suppression. Neuron, 20(6), 1177–1189.

    Article  PubMed  CAS  Google Scholar 

  • Coppola, D., & Purves, D. (1996). The extraordinarily rapid disappearance of entopic images. Proceedings of the National Academy of Sciences of the United States of America, 93(15), 8001–8004.

    Article  PubMed  CAS  Google Scholar 

  • Crouzet, S. M., Kirchner, H., & Thorpe, S. J. (2010). Fast saccades toward faces: face detection in just 100 ms. Journal of Vision, 10(4), 1–17.

    Article  PubMed  Google Scholar 

  • Delorme, A., Perrinet, L., Thorpe, S., & Samuelides, M. (2001). Networks of integrate-and-fire neurons using rank order coding B: spike timing dependent plasticity and emergence of orientation selectivity. Neurocomputing, 38–40, 539–545.

    Article  Google Scholar 

  • Delorme, A., & Thorpe, S. J. (2001). Face identification using one spike per neuron: resistance to image degradations. Neural Networks, 14(6–7), 795–803.

    Article  PubMed  CAS  Google Scholar 

  • Desbordes, G., Jin, J., Weng, C., Lesica, N. A., Stanley, G. B., & Alonso, J.-M. (2008). Timing precision in population coding of natural scenes in the early visual system. PLoS Biology, 6(12), e324.

    Article  PubMed  Google Scholar 

  • Einhäuser, W., Kayser, C., König, P., & Körding, K. P. (2002). Learning the invariance properties of complex cells from their responses to natural stimuli. European Journal of Neuroscience, 15(3), 475–486.

    Article  PubMed  Google Scholar 

  • Enroth-Cugell, C., Robson, J. G., Schweitzer-Tong, D. E., & Watson, A. B. (1983). Spatio-temporal interactions in cat retinal ganglion cells showing linear spatial summation. The Journal of Physiology, 341, 279–307.

    PubMed  CAS  Google Scholar 

  • Fabre-Thorpe, M., Richard, G., & Thorpe, S. J. (1998). Rapid categorization of natural images by rhesus monkeys. NeuroReport, 9(2), 303–8.

    Article  PubMed  CAS  Google Scholar 

  • Ferster, D., Chung, S., & Wheat, H. (1996). Orientation selectivity of thalamic input to simple cells of cat visual cortex. Nature, 380(6571), 249–252.

    Article  PubMed  CAS  Google Scholar 

  • Földiák, P. (1991). Learning invariance from transformation sequences. Neural Computation, 3, 194–200.

    Article  Google Scholar 

  • Fukushima, K. (1980). Neocognitron: a self organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological Cybernetics, 36(4), 193–202.

    Article  PubMed  CAS  Google Scholar 

  • Gawne, T., Kjaer, T., & Richmond, B. (1996). Latency: another potential code for feature binding in striate cortex. Journal of Neurophysiology, 76(2), 1356–1360.

    PubMed  CAS  Google Scholar 

  • Gerstner, W., Ritz, R., & van Hemmen, J. L. (1993). Why spikes? hebbian learning and retrieval of time-resolved excitation patterns. Biological Cybernetics, 69(5–6), 503–515.

    PubMed  CAS  Google Scholar 

  • Gilson, M., Masquelier, T., & Hugues, E. (2011). STDP allows fast rate-modulated coding with Poisson-like spike trains. PLoS Computational Biology (in press).

  • Girard, P., Jouffrais, C., & Kirchner, C. H. (2008). Ultra-rapid categorisation in non-human primates. Animal Cognition, 11(3), 485–493.

    Article  PubMed  CAS  Google Scholar 

  • Gollisch, T., & Meister, M. (2008). Rapid neural coding in the retina with relative spike latencies. Science, 319(5866), 1108–1111.

    Article  PubMed  CAS  Google Scholar 

  • Guyonneau, R., VanRullen, R., & Thorpe, S. (2005). Neurons tune to the earliest spikes through STDP. Neural Computation, 17(4), 859–879.

    Article  PubMed  Google Scholar 

  • Hubel, D., & Wiesel, T. (1962). Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. Journal of Physiology, 160, 106–154.

    PubMed  CAS  Google Scholar 

  • Hung, C., Kreiman, G., Poggio, T., & DiCarlo, J. (2005). Fast readout of object identity from macaque inferior temporal cortex. Science, 310(5749), 863–866.

    Article  PubMed  CAS  Google Scholar 

  • Hyvärinen, A., & Hoyer, P. O. (2001). A two-layer sparse coding model learns simple and complex cell receptive fields and topography from natural images. Vision Research, 41(18), 2413–2423.

    Article  PubMed  Google Scholar 

  • Johansson, R. S., & Birznieks, I. (2004). First spikes in ensembles of human tactile afferents code complex spatial fingertip events. Nature Neuroscience, 7(2), 170–177.

    Article  PubMed  CAS  Google Scholar 

  • Kara, P., Reinagel, P., & Reid, R. C. (2000). Low response variability in simultaneously recorded retinal, thalamic, and cortical neurons. Neuron, 27(3), 635–646.

    Article  PubMed  CAS  Google Scholar 

  • Keat, J., Reinagel, P., Reid, R. C., & Meister, M. (2001). Predicting every spike: a model for the responses of visual neurons. Neuron, 30(3), 803–817.

    Article  PubMed  CAS  Google Scholar 

  • Kempter, R., Gerstner, W., & van Hemmen, J. L. (1999). Hebbian learning and spiking neurons. Physical Review E, 59(4), 4498–4514.

    Article  CAS  Google Scholar 

  • Kirchner, H., & Thorpe, S. (2006). Ultra-rapid object detection with saccadic eye movements: visual processing speed revisited. Vision Research, 46(11), 1762–1776.

    Article  PubMed  Google Scholar 

  • König, P., Engel, A. K., & Singer, W. (1996). Integrator or coincidence detector? The role of the cortical neuron revisited. Trends in Neurosciences, 19(4), 130–7.

    Article  PubMed  Google Scholar 

  • Körding, K., Kayser, C., Einhäuser, W., & König, P. (2004). How are complex cell properties adapted to the statistics of natural stimuli? Journal of Neurophysiology, 91(1), 206–212.

    Article  PubMed  Google Scholar 

  • LeCun, Y., & Bengio, Y. (1998). Convolutional networks for images, speech, and time series. In M. A. Arbib (Ed.), The handbook of brain theory and neural networks (pp. 255–258). Cambridge: MIT.

    Google Scholar 

  • Lichtsteiner, P., Posch, C., & Delbruck, T. (2007). An 128 × 128 120db 15us-latency temporal contrast vision sensor. IEEE Journal of Solid-State Circuits, 43(2), 566–576.

    Article  Google Scholar 

  • Liu, H., Agam, Y., Madsen, J. R., & Kreiman, G. (2009). Timing, timing, timing: fast decoding of object information from intracranial field potentials in human visual cortex. Neuron, 62(2), 281–290.

    Article  PubMed  CAS  Google Scholar 

  • Martinez-Conde, S., Macknik, S. L., & Hubel, D. H. (2004). The role of fixational eye movements in visual perception. Nature Reviews Neuroscience, 5(3), 229–240.

    Article  PubMed  CAS  Google Scholar 

  • Masquelier, T., Guyonneau, R., & Thorpe, S. J. (2008). Spike timing dependent plasticity finds the start of repeating patterns in continuous spike trains. PloS One, 3(1), e1377.

    Article  PubMed  Google Scholar 

  • Masquelier, T., Guyonneau, R., & Thorpe, S. J. (2009). Competitive STDP-based spike pattern learning. Neural Computation, 21(5), 1259–1276.

    Article  PubMed  Google Scholar 

  • Masquelier, T., Hugues, E., Deco, G., & Thorpe, S. J. (2009). Oscillations, phase-of-firing coding, and spike timing-dependent plasticity: an efficient learning scheme. Journal of Neuroscience, 29(43), 13484–13493.

    Article  PubMed  CAS  Google Scholar 

  • Masquelier, T., Serre, T., Thorpe, S., & Poggio, T. (2007). Learning complex cell invariance from natural videos: a plausibility proof. Massachusetts Institute of Technology, CBCL Paper #269/MIT-CSAIL-TR #2007-060.

  • Masquelier, T., & Thorpe, S. J. (2007). Unsupervised learning of visual features through spike timing dependent plasticity. PLoS Computational Biology, 3(2), e31.

    Article  PubMed  Google Scholar 

  • Miller, K. D., & MacKay, D. J. C. (1994). The role of constraints in hebbian learning. Neural Computation, 6, 100–126.

    Article  Google Scholar 

  • Montemurro, M. A., Rasch, M. J., Murayama, Y., Logothetis, N. K., & Panzeri, S. (2008). Phase-of-firing coding of natural visual stimuli in primary visual cortex. Current Biology, 18(5), 375–380.

    Article  PubMed  CAS  Google Scholar 

  • Olshausen, B. A., & Field, D. J. (1996). Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 381, 607–609.

    Article  PubMed  CAS  Google Scholar 

  • Oram, M., & Perrett, D. (1992). Time course of neural responses discriminating different views of the face and head. Journal of Neurophysiology, 68(1), 70–84.

    PubMed  CAS  Google Scholar 

  • Panzeri, S., & Diamond, M. E. (2010). Information carried by population spike times in the whisker sensory cortex can be decoded without knowledge of stimulus time. Frontiers in Synaptic Neuroscience, 2(17), 1–14.

    Google Scholar 

  • Puchalla, J. L., Schneidman, E., Harris, R. A., & Berry, M. J. (2005). Redundancy in the population code of the retina. Neuron, 46(3), 493–504.

    Article  PubMed  CAS  Google Scholar 

  • Rathbun, D. L., Warland, D. K., & Usrey, W. M. (2010). Spike timing and information transmission at retinogeniculate synapses. Journal of Neuroscience, 30(41), 13558–13566.

    Article  PubMed  CAS  Google Scholar 

  • Rehn, M., & Sommer, F. T. (2007). A network that uses few active neurones to code visual input predicts the diverse shapes of cortical receptive fields. Journal of Computational Neuroscience, 22(2), 135–146.

    Article  PubMed  Google Scholar 

  • Riesenhuber, M., & Poggio, T. (1999). Hierarchical models of object recognition in cortex. Nature Neuroscience, 2(11), 1019–1025.

    Article  PubMed  CAS  Google Scholar 

  • Rolls, E., & Milward, T. (2000). A model of invariant object recognition in the visual system: learning rules, activation functions, lateral inhibition, and information-based performance measures. Neural Computation, 12(11), 2547–2572.

    Article  PubMed  CAS  Google Scholar 

  • Rousselet, G. A., Fabre-Thorpe, M., & Thorpe, S. J. (2002). Parallel processing in high-level categorization of natural images. Nature Neuroscience, 5(7), 629–30.

    PubMed  CAS  Google Scholar 

  • Serre, T., Oliva, A., & Poggio, T. (2007). A feedforward architecture accounts for rapid categorization. Proc. Nat. Acad. Sci. USA, 104(15).

  • Singer, W., Tretter, F., & Cynader, M. (1975). Organization of cat striate cortex: a correlation of receptive-field properties with afferent and efferent connections. Journal of Neurophysiology, 38(5), 1080–1098.

    PubMed  CAS  Google Scholar 

  • Song, S., Miller, K., & Abbott, L. (2000). Competitive hebbian learning through spike-timing-dependent synaptic plasticity. Nature Neuroscience, 3(9), 919–926.

    Article  PubMed  CAS  Google Scholar 

  • Spratling, M. (2005). Learning viewpoint invariant perceptual representations from cluttered images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(5).

  • Stevenson, I. H., & Kording, K. P. (2011). How advances in neural recording affect data analysis. Nature Neuroscience, 14(2), 139–142.

    Article  PubMed  CAS  Google Scholar 

  • Stewart, N., Brown, G. D. A., & Chater, N. (2005). Absolute identification by relative judgment. Psychological Review, 112(4), 881–911.

    Article  PubMed  Google Scholar 

  • Stone, J. (1965). A quantitative analysis of the distribution of ganglion cells in the cat’s retina. The Journal of Comparative Neurology, 124(3), 337–352.

    Article  PubMed  CAS  Google Scholar 

  • Stringer, S., & Rolls, E. (2000). Position invariant recognition in the visual system with cluttered environments. Neural Networks, 13(3), 305–315.

    Article  PubMed  CAS  Google Scholar 

  • Thorpe, S., Fize, D., & Marlot, C. (1996). Speed of processing in the human visual system. Nature, 381(6582), 520–2.

    Article  PubMed  CAS  Google Scholar 

  • Thorpe, S., & Gautrais, J. (1998). Rank order coding. In J. M. Bower (Ed.), Computational neuroscience: Trends in research (pp. 113–118). New York: Plenum.

    Chapter  Google Scholar 

  • Thorpe, S., & Imbert, M. (1989). Biological constraints on connectionist modelling. In R. Pfeifer, Z. Schreter, F. Fogelman-Soulie, & L. Steels (Eds.), Connectionism in perspective (pp. 63–92). Amsterdam: Elsevier.

    Google Scholar 

  • Ullman, S., Vidal-Naquet, M., & Sali, E. (2002). Visual features of intermediate complexity and their use in classification. Nature Neuroscience, 5(7), 682–687.

    PubMed  CAS  Google Scholar 

  • van Hateren, J. H., & Ruderman, D. L. (1998). Independent component analysis of natural image sequences yields spatio-temporal filters similar to simple cells in primary visual cortex. Proceedings. Biological sciences / The Royal Society, 265(1412), 2315–2320.

    Article  PubMed  Google Scholar 

  • van Hateren, J. H., & van der Schaaf, A. (1998). Independent component filters of natural images compared with simple cells in primary visual cortex. Proceedings. Biological sciences / The Royal Society, 265(1394), 359–366.

    Article  PubMed  Google Scholar 

  • van Rossum, M. C., Bi, G. Q., & Turrigiano, G. G. (2000). Stable hebbian learning from spike timing-dependent plasticity. Journal of Neuroscience, 20(23), 8812–8821.

    PubMed  Google Scholar 

  • VanRullen, R., Gautrais, J., Delorme, A., & Thorpe, S. (1998). Face processing using one spike per neurone. Biosystems, 48(1–3), 229–239.

    Article  CAS  Google Scholar 

  • VanRullen, R., & Thorpe, S. (2001). Rate coding versus temporal order coding: what the retinal ganglion cells tell the visual cortex. Neural Computation, 13(6), 1255–1283.

    Article  CAS  Google Scholar 

  • VanRullen, R., & Thorpe, S. (2002). Surfing a spike wave down the ventral stream. Vision Research, 42(23), 2593–2615.

    Article  PubMed  Google Scholar 

  • Vinje, W. E., & Gallant, J. L. (2000). Sparse coding and decorrelation in primary visual cortex during natural vision. Science, 287(5456), 1273–1276.

    Article  PubMed  CAS  Google Scholar 

  • Wallis, G., & Rolls, E. (1997). Invariant face and object recognition in the visual system. Progress in Neurobiology, 51(2), 167–194.

    Article  PubMed  CAS  Google Scholar 

  • Williams, P. E., Mechler, F., Gordon, J., Shapley, R., & Hawken, M. J. (2004). Entrainment to video displays in primary visual cortex of macaque and humans. Journal of Neuroscience, 24(38), 8278–8288.

    Article  PubMed  CAS  Google Scholar 

  • Wilson, J. R., & Sherman, S. M. (1976). Receptive-field characteristics of neurons in cat striate cortex: Changes with visual field eccentricity. Journal of Neurophysiology, 39(3), 512–533.

    PubMed  CAS  Google Scholar 

  • Wiskott, L., & Sejnowski, T. J. (2002). Slow feature analysis: unsupervised learning of invariances. Neural Computation, 14(4), 715–770.

    Article  PubMed  Google Scholar 

  • Wohrer, A. (2008). Model and large-scale simulator of a biological retina, with contrast gain control. PhD thesis, University of Nice-Sophia Antipolis.

  • Wohrer, A., & Kornprobst, P. (2009). Virtual retina: a biological retina model and simulator, with contrast gain control. Journal of Computational Neuroscience, 26(2), 219–249.

    Article  PubMed  Google Scholar 

  • Wörgötter, F., Nelle, E., Li, B., & Funke, K. (1998). The influence of corticofugal feedback on the temporal structure of visual responses of cat thalamic relay cells. The Journal of Physiology, 509(Pt 3), 797–815.

    Article  PubMed  Google Scholar 

  • Zamarreño-Ramos, C., Camuñas-Mesa, L., Perez-Carrasco, J. A., Masquelier, T., Serrano-Gotarredona, T., & Linares-Barranco, B. (2011). On spike-timing-dependent-plasticity, memristive devices, and building a self-learning visual cortex. Front. Neurosc.—Neuromorph. Eng., 5(26).

Download references

Acknowledgements

This research was supported by the Fyssen Foundation and the FP7 European Project Coronet. We thank Adrien Wohrer for having developed the Virtual Retina simulator (Wohrer and Kornprobst 2009), both user-friendly and highly configurable, and for the quality of his support. We also thank Wolfgang Einhäuser for kindly providing the videos used in (Betsch et al. 2004), and Mario Pannunzi for the numerous insightful brainstorms we had.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Timothée Masquelier.

Additional information

Action Editor: Ken Miller

Electronic supplementary material

Below is the link to the electronic supplementary material.

Esm 1

(AVI 23590 kb)

Esm 2

(PDF 5 kb)

Esm 3

(PDF 5 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Masquelier, T. Relative spike time coding and STDP-based orientation selectivity in the early visual system in natural continuous and saccadic vision: a computational model. J Comput Neurosci 32, 425–441 (2012). https://doi.org/10.1007/s10827-011-0361-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10827-011-0361-9

Keywords

Navigation