Elsevier

Neurocomputing

Volume 69, Issues 13–15, August 2006, Pages 1561-1569
Neurocomputing

Evolving networks of integrate-and-fire neurons

https://doi.org/10.1016/j.neucom.2005.06.013Get rights and content

Abstract

This paper addresses the following question: “What neural circuits can emulate the monosynaptic correlogram generated by a direct connection between two neurons?” The search for answers to that question has been tackled in two steps: (1) we incorporated into an integrate-and-fire (IAF) neuron model those aspects of neuronal physiology that can influence cross-correlated activity; (2) we evolved networks of biologically realistic neurons towards circuits that are able to generate a monosynaptic correlogram between two neurons. Evolutionary strategies and genetic algorithms were used to explore a computationally intractable search space of physiological parameters and network connectivity. We found that evolutionary strategies perform well in refining good initial solutions, while the simple genetic algorithm achieves worse results even when using a higher computational load. The main obstacles in this challenging study of evolutionary neural networks are exposed and discussed, as well as the results obtained after intensive simulation.

Introduction

The hybridization of neural and evolutionary computation has been extensively used to search neural architectures. Most of these studies were motivated by engineering applications, and the process of evolving neural architectures has sometimes been addressed as a learning strategy. But much less work has been done in the ‘wet’ domain, i.e. inferring plausible patterns of connectivity in real brains with search methods that are droved by physiological data. Here we have applied evolutionary techniques to search for networks of neurons that emulate monosynaptic correlograms measured in electrophysiological experiments.

Artificial neural networks (ANNs) often make use of very simple neuron models based on the representation of the neuron's mean firing rate. Biologically realistic models mimic the dynamics of the neuron's membrane potential, ranging in complexity from the Hodgkin-Huxley model [11], [12], [13], [14] (based on ions flow) to the non-leaky integrate-and-fire (IAF) neuron [29] (based on voltage thresholding). The class of IAF neurons is of special interest, because it reproduces the spiking behaviour of neurons by using the lowest computational load possible. While the electrical functioning of the biological neuron is well understood, and its physiology has been modelled since the beginning of the last century, the precise circuitry of neurons within the nervous system remains obscure. The study of neural circuitry has traditionally involved anatomical and physiological techniques (intra- and extracellular recordings), which are not free of limitations. The anatomical techniques usually provide information about connection patterns but not about the physiological properties of the neurons that are connected. The intracellular recordings combined with anatomical tracers overcome this problem but cannot be used to study a large sample of neurons. Finally, the extracellular techniques allow recording simultaneously from multiple neurons but it is not always possible to know which neurons are monosynaptically connected. Cross-correlation analysis has become a popular tool to identify monosynaptic connections with extracellular recordings in vivo [7], [8], [9], [24], [25]. However, interpreting the different correlogram shapes can be challenging, in part because we do not know precisely how the different correlogram shapes relate to the different connectivity patterns.

In engineering applications of ANNs the topology of the network is determined upon the characteristics of the problem. But this goal-directed approach is hard to set up in biological networks, where we know little about how neurons connect with each other. In extracellular recordings, a strong monosynaptic connection between a presynaptic neuron and a postsynaptic neuron is usually identified based on the shape of the correlogram. The correlogram has a narrow peak displaced from zero indicating that the presynaptic neuron tends to fire before the postsynaptic neuron. Fig. 1 shows (left) an example of a monosynaptic correlogram obtained in a simultaneous recording from a thalamic neuron and a cortical neuron in the cat, and (right) a monosynaptic correlogram obtained with two simulated neurons (see [1], [2], [22], [30] for examples of this kind of connections at different stages in the visual pathway). While this type of correlated activity seems to be a necessary condition to identify monosynaptic connections between neurons, it is not clear to what extent it is also sufficient. The modelling of physiological factors affecting correlated neuronal activity, and the simulation of all possible circuits that can generate a monosynaptic correlogram could help to better define the criteria to measure connectivity with cross-correlation analysis. The difficulty behind this approach is that it requires an exhaustive search of a large space of neural parameters, and the many different topologies in which two neurons could be embedded. Although this approach seems intractable at first, search methods can be adapted to explore a large number of networks and to evolve a randomly connected network towards a specific goal, i.e. the correlogram between two selected neurons (N1, N2) has to be as similar as possible to a specified monosynaptic correlogram. Because the conditions for this network evolution can also be specified, it is possible to impose that N1 and N2 cannot be directly connected.

Evolutionary artificial neural networks (EANNs) offer a general framework for adaptive systems that deal with this type of search (where weights adjustment is a particular case of especial relevance). Although the search for networks with rate-based (continuous output) neurons has been extensively reported in the literature (see [32] for a review), very few works used evolutionary methods to define spiking networks [4], [5], [6]. In this work we apply an evolutionary strategy (ES) and a simple genetic algorithm (SGA) to the problem posed above. The search for circuits that include a pair of neurons with monosynaptic-like cross-correlated activity is formulated as an optimization problem, where a fitness value is computed for each particular combination of network physiology and connectivity. This fitness value measures how well the correlogram between two particular neurons fits a template of a monosynaptic correlogram. According to this framework, populations of biological neural networks (cells parameterized with concrete values and connected in a particular way) compete in fitting the monosynaptic correlogram under random stationary stimulation.

The following sections introduce the neuron model designed to account for most degrees of freedom in shaping the correlogram (e.g. peak height, peak width, baseline height), and the techniques applied to make the networks evolve correctly: methods to generate the initial population, and operators to combine the solutions.

Section snippets

The neuron model

Compartmental models take into account the geometrical and physiological characteristics of the neuron, like axons, dendritic spines, ionic channels or soma. On the other hand, point models characterize the neuron's electrical behaviour, and consider it as a single point in space [10], [15], [16], [23]. These simplified models can introduce the main features of neuronal physiology with a relatively low computational cost.

IAF neurons are a particular case of simplified models that derived from

Assessment of the search space

The first step to start searching for networks with monosynaptic-like correlograms is to estimate the size of the parameter space. For this purpose it is necessary to identify: (1) the set of parameters that determines the shape of the correlogram; (2) the biological ranges of variation for each parameter; and (3) the resolution (bin size) for each parameter. Fig. 3 shows a monosynaptic correlogram generated by two simulated neurons, N1 and N2 (Fig. 3. top, left). The shape of the N1→N2

Results

The methods explained above were implemented to evolve networks of spiking neurons towards a monosynaptic correlogram. Two different approaches were used: an ES and an SGA. The ES and the SGA were run with different network sizes, percentage of inhibitory neurons, mutation rates and ranges of variation for the different genes, random seeds for initial populations and fitness functions. All these simulations run over a heterogeneous Mosix-based cluster of computers. We measured the evolution of

Conclusions

The main limitations in the formulation of this problem as an evolutionary search have been overcome. The correlograms shown in Fig. 6, Fig. 7 are representative of most evolutions run in our experiments. They demonstrate that the ES performs well in refining good initial solutions, while the SGA achieves worse general results, even when dedicating a higher computational load, derived from the application of the crossover operator.

From the results obtained we cannot conclude that polysynaptic

Acknowledgments

This research has been partially funded by NIH-EY 05253, and by the Spanish Ministerio de Ciencia y Tecnología (projects PTR95.0734.OP and OTRI-PAI-03-18).

Francisco J. Veredas was born in Málaga, Spain, in 1970. He received the M.Sc. and Ph.D. degrees in computer science from the University of Málaga, Spain, in 1996, and 2004, respectively. His Ph.D. thesis was about correlated activity in networks of spiking neurons.

He has been working as a teacher in computer programming languages from 1998 at the ‘Department of Languages and Computational Sciences’, at the University of Málaga. He develops his current research work at the group of

References (32)

  • G. Gerstein et al.

    Representation of cooperative firing activity among simultaneously recorded neurons

    J. Neurophysiol.

    (1985)
  • G. Gerstein et al.

    Cooperative firing activity in simultaneously recorded populations of neurons: detection and measurement

    J. Neurosci.

    (1985)
  • S. Haykin

    Neural networks. A Comprehensive Foundation

    (1994)
  • A. Hodking et al.

    The components of membrane conductance in giant axon of Loligo

    J. Physiol.

    (1952)
  • A. Hodking et al.

    Currents carried by sodium and potassium ions through the membrane of the giant axon of Loligo

    J. Physiol.

    (1952)
  • A. Hodking et al.

    The dual effect of membrane potential on sodium conductance in the membrane of the giant axon of Loligo

    J. Physiol.

    (1952)
  • Cited by (3)

    Francisco J. Veredas was born in Málaga, Spain, in 1970. He received the M.Sc. and Ph.D. degrees in computer science from the University of Málaga, Spain, in 1996, and 2004, respectively. His Ph.D. thesis was about correlated activity in networks of spiking neurons.

    He has been working as a teacher in computer programming languages from 1998 at the ‘Department of Languages and Computational Sciences’, at the University of Málaga. He develops his current research work at the group of ‘Computational Intelligence and Image Analysis’ of that university. His research interests relate to computational neuroscience and evolutionary computation.

    Francisco J. Vico was born in Barcelona, Spain, in 1967. He received the M.Sc. and Ph.D. degrees in computer science from the University of Málaga, Spain, in 1991, and 1995, respectively.

    He heads a research group dedicated to Biomimetics, that has developed in recent years an intense activity in technology transfer with industry, specially in effective applications of neural and evolutionary computation. From 2003 to 2004 he spent a postdoctoral year with Ad Aertsen at the Department of Neurobiology and Biophysics, Albert-Ludwigs Universität, Freiburg (Germany). His current research interests relate to general aspects of biological computation.

    Jose-Manuel Alonso was born in Vigo, Spain, in 1964. He received his M.D. from the University of Santiago de Compostela and Ph.D. from the Universidad Autonoma de Madrid. After finishing his Ph.D., he won a Fulbright fellowship to join the laboratory of Dr. Wiesel at Rockefeller University as a postdoctoral student. In 1999 he became an Assistant Professor at the University of Connecticut and in 2002 he moved to New York (SUNY Optometry) where he is currently an Associate Professor and heads a laboratory in Visual Neuroscience.

    View full text