Skip to main content
SpringerLink
Log in

Improved lower bound for the mutual information between signal and neural spike count

  • Original Article
  • Published:
Biological Cybernetics Aims and scope Submit manuscript

Abstract

The mutual information between a stimulus signal and the spike count of a stochastic neuron is in many cases difficult to determine. Therefore, it is often approximated by a lower bound formula that involves linear correlations between input and output only. Here, we improve the linear lower bound for the mutual information by incorporating nonlinear correlations. For the special case of a Gaussian output variable with nonlinear signal dependencies of mean and variance we also derive an exact integral formula for the full mutual information. In our numerical analysis, we first compare the linear and nonlinear lower bounds and the exact integral formula for two different Gaussian models and show under which conditions the nonlinear lower bound provides a significant improvement to the linear approximation. We then inspect two neuron models, the leaky integrate-and-fire model with white Gaussian noise and the Na–K model with channel noise. We show that for certain firing regimes and for intermediate signal strengths the nonlinear lower bound can provide a substantial improvement compared to the linear lower bound. Our results demonstrate the importance of nonlinear input–output correlations for neural information transmission and provide a simple nonlinear approximation for the mutual information that can be applied to more complicated neuron models as well as to experimental data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. There is a strong similarity of the observed maximum of the spike count vs mean signal to a standard problem in statistical physics: the giant acceleration of diffusion in a tilted periodic potential (Reimann et al. 2001; Lindner and Sokolov 2016). A Brownian particle in an inclined washboard potential attains a pronounced maximum in its positional variance [quantified by the diffusion coefficient, \(D=\lim _{T\rightarrow \infty } \langle (x-\langle x\rangle )^2\rangle /(2T)\)] for a certain intermediate value of the bias force [equivalent to the mean signal in the LIF model]. The maximum is attained for a force (mean signal strength) close to the bifurcation value that determines the transition from a noise-induced transport (or firing) regime to a regime, in which movement (spiking) is possible without noise.

References

  • Aldworth ZN, Dimitrov AG, Cummins GI, Gedeon T, Miller JP (2011) Temporal encoding in a nervous system. PLOS Comput Biol 7(5):e1002041

    Article  CAS  Google Scholar 

  • Bernardi D, Lindner B (2015) A frequency-resolved mutual information rate and its application to neural systems. J Neurophysiol 113(5):1342–1357

    Article  Google Scholar 

  • Bialek W, Rieke F, Vansteveninck RRD, Warland D (1991) Reading a neural code. Science 252:1854

    Article  CAS  Google Scholar 

  • Bialek W, Deweese M, Rieke F, Warland D (1993) Bits and brains—information-flow in the nervous system. Physica A 200:581

    Article  Google Scholar 

  • Borst A, Theunissen F (1999) Information theory and neural coding. Nat Neurosci 2:947

    Article  CAS  Google Scholar 

  • Brunel N, Nadal JP (1998) Mutual information, fisher information, and population coding. Neural Comput 10:1731

    Article  CAS  Google Scholar 

  • Burkitt AN (2006) A review of the integrate-and-fire neuron model: I. Homogeneous synaptic input. Biol Cybern 95:1

    Article  CAS  Google Scholar 

  • Chacron MJ, Doiron B, Maler L, Longtin A, Bastian J (2003) Non-classical receptive field mediates switch in a sensory neuron’s frequency tuning. Nature 423:77

    Article  CAS  Google Scholar 

  • Cover T, Thomas J (1991) Elements of information theory. Wiley, New York

    Book  Google Scholar 

  • Cox DR (1962) Renewal theory. Methuen, London

    Google Scholar 

  • Doose J, Doron G, Brecht M, Lindner B (2016) Noisy juxtacellular stimulation in vivo leads to reliable spiking and reveals high-frequency coding in single neurons. J Neurosci 36(43):11120–11132

    Article  CAS  Google Scholar 

  • Droste F, Schwalger T, Lindner B (2013) Interplay of two signals in a neuron with heterogeneous short-term synaptic plasticity. Front Comput Neurosci 7:1

    Article  Google Scholar 

  • Gabbiani F (1996) Coding of time-varying signals in spike trains of linear and half-wave rectifying neurons. Netw Comput Neural Syst 7:61

    Google Scholar 

  • Gammaitoni L, Hänggi P, Jung P, Marchesoni F (1998) Stochastic resonance. Rev Mod Phys 70:223

    Article  CAS  Google Scholar 

  • Grewe J, Kruscha A, Lindner B, Benda J (2017) Synchronous spikes are necessary but not sufficient for a synchrony code in populations of spiking neurons. PNAS 114(10):1977–1985

    Article  Google Scholar 

  • Izhikevich EM (2007) Dynamical systems in neuroscience: the geometry of excitability and bursting. The MIT Press, Cambridge

    Google Scholar 

  • Juusola M, French AS (1997) The efficiency of sensory information coding by mechanoreceptor neurons. Neuron 18(6):959–968

    Article  CAS  Google Scholar 

  • Kraskov A, Stögbauer H, Grassberger P (2004) Estimating mutual information. Phys Rev E 69(6):066138

    Article  Google Scholar 

  • Lagarias J (2013) Euler’s constant: Euler’s work and modern developments. Bull Am Math Soc 50(4):527–628

    Article  Google Scholar 

  • Lindner B, Sokolov IM (2016) Giant diffusion of underdamped particles in a biased periodic potential. Phys Rev E 93(4):042106

    Article  Google Scholar 

  • Lindner B, Schimansky-Geier L, Longtin A (2002) Maximizing spike train coherence or incoherence in the leaky integrate-and-fire model. Phys Rev E 66:031916

    Article  Google Scholar 

  • Marmarelis PZ, Naka K (1972) White-noise analysis of a neuron chain: an application of the Wiener theory. Science 175(4027):1276–1278

    Article  CAS  Google Scholar 

  • McDonnell MD, Ward LM (2011) The benefits of noise in neural systems: bridging theory and experiment. Nat Rev Neurosci 12:415

    Article  CAS  Google Scholar 

  • Morris C, Lecar H (1981) Voltage oscillations in the barnacle giant muscle fiber. Biophys J 35:193

    Article  CAS  Google Scholar 

  • Neiman AB, Russell DF (2011) Sensory coding in oscillatory electroreceptors of paddlefish. Chaos 21:047505

    Article  Google Scholar 

  • Nemenman I, Lewen GD, Bialek W, de Ruyter van Steveninck RR (2008) Neural coding of natural stimuli: information at sub-millisecond resolution. PLOS Comput Biol 4:e1000025

    Article  Google Scholar 

  • Nikias CL, Petropulu AP (1993) Higher-order spectral analysis. PTR Prentice Hall, Upper Saddle River

    Book  Google Scholar 

  • Panzeri S, Schultz SR (2001) A unified approach to the study of temporal, correlational, and rate coding. Neural Comput 13(6):1311–1349

    Article  CAS  Google Scholar 

  • Panzeri S, Senatore R, Montemurro MA, Petersen RS (2007) Correcting for the sampling bias problem in spike train information measures. J Neurophys 98(3):1064–1072

    Article  Google Scholar 

  • Passaglia CL, Troy JB (2004) Information transmission rates of cat retinal ganglion cells. J Neurophysiol 91(3):1217–1229

    Article  Google Scholar 

  • Reimann P, Van den Broeck C, Linke H, Hänggi P, Rubi M, Perez-Madrid A (2001) Giant acceleration of free diffusion by use of tilted periodic potentials. Phys Rev Lett 87:010602

    Article  CAS  Google Scholar 

  • Ricciardi LM (1977) Diffusion processes and related topics on biology. Springer, Berlin

    Book  Google Scholar 

  • Rieke F, Warland D, Bialek W (1993) Coding efficiency and information rates in sensory neurons. Europhys Lett 22:151

    Article  Google Scholar 

  • Rieke F, Bodnar D, Bialek W (1995) Naturalistic stimuli increase the rate and efficiency of information transmission by primary auditory afferents. Proc Biol Sci 262:259

    Article  CAS  Google Scholar 

  • Rieke F, Warland D, de Ruyter van Steveninck R, Bialek W (1996) Spikes: exploring the neural code. MIT Press, Cambridge

    Google Scholar 

  • Ryzhik IM, Gradshtein IS (1963) Tables of series, products, and integrals. VEB Deutscher Verlag der Wissenschaften, Berlin

    Google Scholar 

  • Sadeghi SG, Chacron MJ, Taylor MC, Cullen KE (2007) Neural variability, detection thresholds, and information transmission in the vestibular system. J Neurosci 27(4):771–781

    Article  CAS  Google Scholar 

  • Shannon R (1948) The mathematical theory of communication. Bell Syst Tech J 27:379

    Article  Google Scholar 

  • Siegert AJF (1951) On the first passage time problem. Phys Rev 81:617

    Article  Google Scholar 

  • Stemmler M, Koch C (1999) How voltage-dependent conductances can adapt to maximize the information encoded by neuronal firing rate. Nat Neurosci 2:521

    Article  CAS  Google Scholar 

  • Strong SP, Koberle R, van Steveninck RRD, Bialek W (1998) Entropy and information in neural spike trains. Phys Rev Lett 80:197

    Article  CAS  Google Scholar 

  • Theunissen F, Miller JP (1995) Temporal encoding in nervous systems: a rigorous definition. J Comput Neurosci 2(2):149–162

    Article  CAS  Google Scholar 

  • Thomas PJ, Lindner B (2014) Asymptotic phase for stochastic oscillators. Phys Rev Lett 113(25):254101–5

    Article  Google Scholar 

  • Victor JD (2002) Binless strategies for estimation of information from neural data. Phys Rev E 66(5):051903

    Article  Google Scholar 

  • Victor JD (2006) Approaches to information-theoretic analysis of neural activity. Biol Theory 1(3):302–316

    Article  Google Scholar 

  • Vilela RD, Lindner B (2009a) Are the input parameters of white-noise-driven integrate and fire neurons uniquely determined by rate and CV? J Theor Biol 257:90

    Article  Google Scholar 

  • Vilela RD, Lindner B (2009b) A comparative study of three different integrate-and-fire neurons: spontaneous activity, dynamical response, and stimulus-induced correlation. Phys Rev E 80:031909

    Article  Google Scholar 

  • Voronenko SO (2018) Nonlinear signal processing by noisy spiking neurons. Ph.D. thesis, Humboldt-Universität zu Berlin, Berlin

  • Voronenko SO, Lindner B (2017) Weakly nonlinear response of noisy neurons. New J Phys 19(3):033–038

    Article  Google Scholar 

  • Voronenko SO, Stannat W, Lindner B (2015) Shifting spike times or adding and deleting spikes—how different types of noise shape signal transmission in neural populations. JMN 5(1):1–35

Download references

Acknowledgements

This work was supported by the BMBF (FKZ: 01GQ1001A) and the DFG (Research Training Group GRK1589/2).

Author information

Authors and Affiliations

  1. Bernstein Center for Computational Neuroscience Berlin, Philippstr. 13, Haus 2, 10115, Berlin, Germany

    Sergej O. Voronenko & Benjamin Lindner

  2. Physics Department, Humboldt University Berlin, Newtonstr. 15, 12489, Berlin, Germany

    Sergej O. Voronenko & Benjamin Lindner

Authors
  1. Sergej O. Voronenko
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Benjamin Lindner
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sergej O. Voronenko.

Additional information

Communicated by Peter J. Thomas.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Voronenko, S.O., Lindner, B. Improved lower bound for the mutual information between signal and neural spike count. Biol Cybern 112, 523–538 (2018). https://doi.org/10.1007/s00422-018-0779-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00422-018-0779-5

Keywords

Search

Navigation