Skip to main content

Advertisement

SpringerLink
Log in

Cooperation in self-organizing map networks enhances information transmission in the presence of input background activity

  • Original Paper
  • Published:
Biological Cybernetics Aims and scope Submit manuscript

Abstract

The self-organizing map (SOM) algorithm produces artificial neural maps by simulating competition and cooperation among neurons. We study the consequences of input background activity on simulated self-organization, using the SOM, of the retinotopic map in the superior colliculus. The colliculus not only represents its inputs but also uses them to localize saccadic targets. Using the colliculus as a test-bed enables us to quantify the results of self- organization both descriptively, in terms of input-output mutual information, and functionally, in terms of the probability of error (expected distortion) in localizing targets. We find that mutual information is low, and distortion is high, when the SOM operates in the presence of input background activity but without the cooperative component (no neighbor training). Cooperation (training neighbors) greatly increases mutual information and greatly decreases expected distortion. Our simulation results extend theoretical work suggesting that cooperative mechanisms are needed to increase the information content of neural representations. They also identify input background activity as a factor affecting the self-organization of information-transmitting channels in the nervous system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Aitkin LM and Webster WR (1972). Medial geniculate body of the cat: organization and responses to tonal stimuli of neurons in ventral division. J Neurophysiol 35: 365–80

    CAS  PubMed  Google Scholar 

  • Anastasio TJ and Patton PE (2003). A two-stage unsupervised learning algorithm reproduces multisensory enhancement in a neural network of the corticotectal system. J Neurosci 23: 6713–727

    CAS  PubMed  Google Scholar 

  • Arimoto S (1972). An algorithm for calculating the capacity of an arbitrary discrete memoryless channel. IEEE Trans Info Theory IT-18: 14–0

    Article  Google Scholar 

  • Baddeley R, Hancock P, Földiák P (eds) (2000). Information theory and the brain. Cambridge University Press, Cambridge

    Google Scholar 

  • Berger T (1971). Rate distortion theory: a mathematical basis for data compression. Prentice Hall, Englewood Cliffs

    Google Scholar 

  • Berger T (2003) Living information theory. In: IEEE Info Theory Soc Newsletter, March 2003: http://www.itsoc.org/publications/newsletters.html

  • Berger T and Gibson JA (1998). Lossy source coding. IEEE Trans Info Theory 44: 2693–723

    Article  Google Scholar 

  • Bishop PO, Kozak W, Levick WR and Vakkur GJ (1962). The determination of the projection of the visual field on to the lateral geniculate nucleus in the cat. J Physiol Lond 163: 503–39

    CAS  PubMed  Google Scholar 

  • Blahut RE (1972). Computation of channel capacity and rate-distortion functions. IEEE Trans Info Theory IT-18: 460–73

    Article  Google Scholar 

  • Bock GR, Webster WR and Aitkin LM (1971). Discharge patterns of single units in inferior colliculus of the alert cat. J Neurophysiol 35: 265–77

    Google Scholar 

  • Bourk TR, Mielcarz JP and Norris BE (1981). Tonotopic organization of the anteroventral cochlear nucleus of the cat. Hear Res 4: 215–41

    Article  CAS  PubMed  Google Scholar 

  • Brownell WE (1975). Organization of the cat trapezoid body and the discharge characteristics of its fibers. Brain Res 93: 413–33

    Article  Google Scholar 

  • Cleland BG, Dubin MW and Levick WR (1971). Sustained and transient neurons in the cats retina and lateral geniculate nucleus. J Physiol Lond 217: 473–96

    CAS  PubMed  Google Scholar 

  • Cline HT (1991). Activity-dependent plasticity in the visual systems of frogs and fish. Trends Neurosci 14: 104–11

    Article  CAS  PubMed  Google Scholar 

  • Cline HT (1998). Topographic maps: developing roles for synaptic plasticity. Curr Biol 8: R836–39

    Article  CAS  PubMed  Google Scholar 

  • Constantine-Paton M, Cline HT and Debski E (1990). Patterned activity, synaptic convergence, and the NMDA receptor in developing visual pathways. Annu Rev Neurosci 13: 129–54

    Article  CAS  PubMed  Google Scholar 

  • Contestabile A (2000). Roles of NMDA receptor activity and nitric oxide production in brain development. Brain Res Revs 32: 476–09

    Article  CAS  Google Scholar 

  • Cover TM and Thomas JA (1991). Elements of information theory. Wiley, New York

    Book  Google Scholar 

  • Daniel PM and Whitteridge D (1961). The representation of the visual field on the cerebral cortex in monkeys. J Physiol Lond 159: 203–21

    CAS  PubMed  Google Scholar 

  • Dayan P and Abbott CF (2001). Theoretical neuroscience: computational and mathematical modeling of neural systems. MIT Press, Cambrdige

    Google Scholar 

  • Dersch DR and Tavan P (1995). Asymptotic level density in topological feature maps. IEEE Trans Neural Netw 6: 230–36

    Article  CAS  PubMed  Google Scholar 

  • Edwards SB, Ginsburgh CL, Henkel CK and Stein BE (1979). Sources of subcortical projections to the superior colliculus in the cat. J Comp Neurol 184: 309–29

    Article  CAS  PubMed  Google Scholar 

  • Ferrell C (1996) Orientation behavior using registered topographic maps. In: Conference on the Simulation of Adaptive Behavior. Cape Cod, MA: http://www.ai.mit.edu/projects/cog/publications.html

  • Gastpar M, Rimoldi B and Vetterli M (2003). To code, or not to code: lossy source-channel communication revisited. IEEE Trans. Info Theory 49: 1147–158

    Article  Google Scholar 

  • Gelfand JJ, Pearson JC, Spence CD, Sullivain WE (1988) Multisensor integration in biological systems. In: IEEE international symposium on intelligent control. IEEE Computer Society Press, Arlington, pp 147–53

  • Gersho A and Gray RM (1992). Vector quantization and signal compression. Kluwer, Boston

    Google Scholar 

  • Graf S and Luschgy H (2000). Foundations of quantization for probability distributions. Springer-Verlag, Berlin

    Google Scholar 

  • Guinan JJ, Guinan SS and Norris BE (1972). Single auditory units in the superior olivary complex. I. Responses to sounds and classification based on physiological properties. Int J Neurosci 4: 101–20

    Article  Google Scholar 

  • Grossberg S (1976). Adaptive pattern classification and universal recoding. I. Parallel development and coding of neural feature detectors. Biol Cybern 23: 121–34

    Article  CAS  PubMed  Google Scholar 

  • Haykin S (1999). Neural networks: a comprehensive foundation, 2nd edn. Prentice Hall, Upper Saddle River

    Google Scholar 

  • Hayward JN (1975). Response of ventrobasal thalamic cells to hair displacement on the face of the waking monkey. J Physiol Lond 250: 385–07

    CAS  PubMed  Google Scholar 

  • Hepp K, Van Opstal AJ, Straumann D, Hess BMJ and Henn V (1993). Monkey superior colliculus represents rapid eye movements in a two-dimensional motor map. J Neurophysiol 69: 965–79

    CAS  PubMed  Google Scholar 

  • Hubel DA and Wiesel TN (1960). Receptive fields of optic nerve fibers in the spider monkey. J Physiol Lond 154: 572–80

    CAS  PubMed  Google Scholar 

  • Katsuki Y, Suga N and Kanno Y (1962). Neural mechanism of the peripheral and central auditory systems in monkeys. J Acoust Soc Am 34: 1396–410

    Article  Google Scholar 

  • Keller EL and McPeek RM (2002). Neural discharge in the superior colliculus during target search paradigms. Ann NY Acad Sci 956: 130–42

    Article  PubMed  Google Scholar 

  • Kiang NYS (1965). Stimulus coding in auditory nerve and cochlear nucleus. Acta OtoLaryngol 59: 186–00

    Article  Google Scholar 

  • Kohonen T (1982). Self-organized formation of topologically correct feature maps. Biol Cybern 44: 59–9

    Article  Google Scholar 

  • Kohonen T (1988). Self organization and associative memory, 2nd edn. Springer, Berlin

    Google Scholar 

  • Kuffler SW (1953). Discharge patterns and functional organization of mammalian retina. J Neurophysiol 16: 37–8

    CAS  PubMed  Google Scholar 

  • Linsker R (1989). How to generate ordered maps by maximizing the mutual information between input and output signals. Neural Comput 1: 402–11

    Article  Google Scholar 

  • Luttrell SP (1989) Self-organization: a derivation from first principles of a class of learning algorithms. IEEE conference on neural networks, Washington DC, pp 495–98

  • Luttrell SP (1994). A Bayesian analysis of self-organizing maps. Neural Comput 6: 767–94

    Article  Google Scholar 

  • MacQueen J (1967) Some methods for classification and analysis of multivariate observations. In: Proceedings of 5th Berkeley symposium on mathematical statistics and probability, vol 1, pp 281–96

  • Malpeli JG and Baker FH (1975). The representation of the visual field in the lateral geniculate nucleus of Macaca mulatta. J Comp Neurol 161: 569–94

    Article  CAS  PubMed  Google Scholar 

  • McPeek RM and Keller EL (2002). Superior colliculus activity related to concurrent processing of saccade goals in a visual search task. J Neurophysiol 87(4): 1805–815

    PubMed  Google Scholar 

  • Meredith MA and Stein BE (1990). The visuotopic component of the multisensory map in the deep laminae of the cat superior colliculus. J Neurosci 10: 3727–742

    CAS  PubMed  Google Scholar 

  • Meredith MA, Clemo HR and Stein BE (1991). Somatotopic component of the multisensory map in the deep laminae of the cat superior colliculus. J Comp Neurol 312: 353–70

    Article  CAS  PubMed  Google Scholar 

  • Merényi E, Jain A and Villmann T (2007). Explicit magnification control of self-organizing maps for ‘forbidden’data. IEEE Trans Neural Netw 18: 786–97

    Article  PubMed  Google Scholar 

  • Merzenich MM, Knight PL and Roth GL (1975). Representation of cochlea within primary auditory cortex in the cat. J Neurophysiol 38: 231–49

    CAS  PubMed  Google Scholar 

  • Middlebrooks JC and Knudsen EI (1984). A neural code for auditory space in the cat’s superior colliculus. J Neurosci 4: 2621–634

    CAS  PubMed  Google Scholar 

  • Mountcastle VB, Pogio GF and Werner G (1963). The relation of thalamic cell response to peripheral stimuli carried over an intensive continuum. J Neurophysiol 26: 807–34

    CAS  PubMed  Google Scholar 

  • Obermayer K, Ritter H and Schulten K (1990). A principle for the formation of the spatial structure of cortical feature maps. Proc Natl Acad Sci USA 87: 8345–349

    Article  CAS  PubMed  Google Scholar 

  • Obermayer K, Blasdel GG and Schulten K (1992). Statistical-mechanical analysis of self-organization and pattern formation during development of visual maps. Phys Rev A 45: 7568–589

    Article  PubMed  Google Scholar 

  • O’Leary DDM, Yates PA and McLaughlin T (1999). Molecular development of sensory maps: Representing sights and smells in the brain. Cell 96: 255–69

    Article  PubMed  Google Scholar 

  • Ritter H (1991). Asymptotic level density for a class of vector quantization processes. IEEE Trans Neural Netw 2: 173–75

    Article  CAS  PubMed  Google Scholar 

  • Ritter H and Schulten K (1986). On the stationary state of Kohonen’s self-organizing sensory mapping. Biol Cybern 54: 99–06

    Article  Google Scholar 

  • Ritter H, Martinetz T and Schulten K (1992). Neural computation and self-organizing maps: an introduction. Addison-Wesley, Reading

    Google Scholar 

  • Robinson DA (1972). Eye movements evoked by collicular stimulation in the alert monkey. Vis Res 12: 179–808

    Google Scholar 

  • Rumelhart DE and Zipser D (1985). Feature discovery by competitive learning. Cogn Sci 9: 75–12

    Article  Google Scholar 

  • Schmidt JT (1985). Formation of retinotopic connections: Selective stabilization by an activity dependent mechanism. Cell Mol Neurobiol 5: 65–4

    Article  CAS  PubMed  Google Scholar 

  • Schmidt M (1996). Neurons in the cat pretectum that project to the dorsal lateral geniculate nucleus are activated during saccades. J Neurophysiol 76: 2907–918

    CAS  PubMed  Google Scholar 

  • Tessier-Lavigne M (1995). Eph receptor tyrosine kinases, axon repulsion and the development of topographic maps. Cell 82: 345–48

    Article  CAS  PubMed  Google Scholar 

  • Tessier-Lavigne M and Goodman CS (1996). The molecular biology of axon guidance. Science 274: 1123–133

    Article  CAS  PubMed  Google Scholar 

  • Tsumato T and Nakamura S (1974). Inhibitory organization of the thalamic ventrobasal neurons with different peripheral representations. Exp Brain Res 21: 195–10

    Google Scholar 

  • Tusa RJ, Palmer LA and Rosenquist AC (1978). The retinotopic organization of area 17 (striate cortex) in the cat. J Comp Neurol 177: 213–36

    Article  CAS  PubMed  Google Scholar 

  • Van Hulle MM (1996). Topographic map formation by maximizing unconditional entropy: A plausible strategy for on-line unsupervised competitive learning and nonparametric density estimation. IEEE Trans Neural Netw 7: 1299–305

    Article  CAS  PubMed  Google Scholar 

  • Van Hulle MM (1997). Nonparametric density estimation and regression achieved with topographic maps maximizing the information-theoretic entropy of their outputs. Biol Cybern 77: 49–1

    Article  Google Scholar 

  • Vanegas H, (ed) (1984). Comparative neurology of the optic tectum. Plenum Press, New York

    Google Scholar 

  • Villmann T and Claussen JC (2006). Magnification control in self- organizing maps and neural gas. Neural Comput 18: 446–69

    Article  PubMed  Google Scholar 

  • Wallace MT, Wilkinson LK and Stein BE (1996). Representation and integration of multiple sensory inputs in primate superior colliculus. J Neurophysiol 76: 1246–266

    CAS  PubMed  Google Scholar 

  • Willshaw DJ and von der Malsburg C (1976). How patterned neural connections can be set up by self-organization. Proc R Soc Lond B 194: 431–45

    Article  CAS  PubMed  Google Scholar 

  • Woolsey CN (1981). Cortical sensory organization: multiple somatic areas. Humana Press, Clifton

    Google Scholar 

  • Zador PL (1982). Asymptotic quantization error of continuous signals and the quantization dimension. IEEE Trans Info Theory IT-28: 139–49

    Article  Google Scholar 

  • Zhang LL, Tao HW, Holt CE, Harris WA and Poo M-M (1998). A critical window for cooperation and competition among developing retinotectal synapses. Nature 395: 37–4

    Article  CAS  PubMed  Google Scholar 

Download references

Author information

Author notes

  1. Maxim Raginsky

    Present address: Department of Electrical and Computer Engineering, Duke University, Durham, NC, 27708, USA

Authors and Affiliations

  1. Beckman Institute for Advanced Science and Technology, University of Illinois, 405 N Mathews Ave, Urbana, IL, 61801, USA

    Maxim Raginsky

  2. Beckman Institute for Advanced Science and Technology, Department of Molecular and Integrative Physiology, University of Illinois, Urbana, IL, 61801, USA

    Thomas J. Anastasio

Authors
  1. Maxim Raginsky
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thomas J. Anastasio
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Maxim Raginsky.

Additional information

This work was funded by Beckman Institute Fellowship to MR, and by Office of Naval Research Grant N00014-01-1-0249 to TJA.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Raginsky, M., Anastasio, T.J. Cooperation in self-organizing map networks enhances information transmission in the presence of input background activity. Biol Cybern 98, 195–211 (2008). https://doi.org/10.1007/s00422-007-0203-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00422-007-0203-z

Keywords

Search

Navigation