Skip to main content
SpringerLink
Log in

On the Natural Hierarchical Composition of Cliques in Cell Assemblies

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Hebbian cell assemblies can be formalised as sets of tightly connected cells in auto- and hetero-associative memories. Direct evidence for such “cliques” has recently been obtained in multiple-unit recordings from rat hippocampal neurons. These experiments suggest a hierarchical organisation where cliques are embedded in each other such that larger cliques represent less specific stimulus conditions. We here suggest an interpretation stating that the firing patterns may not just reflect nested categories but a lattice of concepts about stimulus–response mappings in the sense of formal concept analysis, an applied branch of set theory. We present an implementation of formal concept lattices in bidirectional associative memories that in contrast to previous work satisfies Dale’s principle and uses balanced excitation and inhibition. Inhibitory cells have fixed, non-plastic synapses even if the model learns new concepts. As an extreme case a single global inhibitory cell is enough that controls the total level of activation. The excitatory cells can further learn incrementally using a Hebbian coincidence learning rule. Implications of the model for retrieval in auto-associative memories are further outlined. Overall the model is well suited for representing hierarchical compositional relationships between entities in the form of correlated patterns in technical cognitive systems and potentially the brain.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. We mainly follow Bělohlávek’s notation and definitions in [5] in the present work.

  2. These entities may be features or objects or just anything, we do not make such distinctions in the auto-associative setup.

References

  1. Abbott LF, Nelson SB. Synaptic plasticity: taming the beast. Nat Neurosci. 2000;3:1178–83.

    Article  PubMed  CAS  Google Scholar 

  2. Amit DJ. Modeling brain function. Cambridge: Cambridge University Press; 1988.

  3. Amit DJ. The Hebbian paradigm reintegrated: local reverberations as internal representations. Behav Brain Sci. 1995;18:617–57.

    Article  Google Scholar 

  4. Bi G-q, Poo M-m. Synaptic modification by correlated activity: Hebb’s postulate revisited. Annu Rev Neurosci. 2001;24:139–66.

    Article  PubMed  CAS  Google Scholar 

  5. Bělohlávek R. Representation of concept lattices by bidirectional associative memories. Neural Comput. 2000;12:2279–90.

    Google Scholar 

  6. Braitenberg V. Cell assemblies in the cerebral cortex. In: Heim R, Palm G, editors. Theoretical approaches to complex systems. Berlin: Springer; 1978. p. 171–88.

  7. Braitenberg V, Schüz A. Anatomy of the cortex. Berlin: Springer-Verlag; 1991.

    Google Scholar 

  8. Dayan P. Images, frames, and connectionist hierarchies. Neural Comput. 2006;18:2293–319.

    Article  PubMed  Google Scholar 

  9. Eccles JC, Fatt P, Koketsu K. Cholinergic and inhibitory synapses in a pathway from motor-axon collaterals to motoneurones. J Physiol (Lond). 1954;126:52462.

    Google Scholar 

  10. Feldman J. Minimization of boolean complexity in human concept learning. Nature. 2000;407:630–3.

    Article  PubMed  CAS  Google Scholar 

  11. Freedman DJ, Riesenhuber M, Poggio T, Miller EK. A comparison of primate prefrontal and inferior temporal cortices during visual categorization. J Neurosci. 2003;23:5235–46.

    PubMed  CAS  Google Scholar 

  12. Friston K. Hierarchical models in the brain. PLoS Comput Biol. 2008;4:e1000211.

    Article  PubMed  CAS  Google Scholar 

  13. Fukushima K. Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol Cybern. 1980;36:193–202.

    Article  PubMed  CAS  Google Scholar 

  14. Fukushima K. Restoring partly occluded patterns: a neural network model. Neural Netw. 2005;18:33–43.

    Article  PubMed  Google Scholar 

  15. Fukushima K, Myaki S. Neocognitron: a new algorithm for pattern recognition tolerant of deformations and shifts in position. Pattern Recognit 1982;15:455–69.

    Article  Google Scholar 

  16. Fusi S, Abbott LF. Limits on the memory storage capacity of bounded synapses. Nat Neurosci. 2007;10:485–93.

    PubMed  CAS  Google Scholar 

  17. Fusi S, Annunziato M, Badoni D, Salamon A, Amit DJ. Spike-driven synaptic plasticity: theory, simulation, VLSI implementation. Neural Comput. 2000;12:2227–58.

    Article  PubMed  CAS  Google Scholar 

  18. Fusi S, Drew PJ, Abbott LF. Cascade models of synaptically stored memories. Neuron. 2005;45:599–611.

    Article  PubMed  CAS  Google Scholar 

  19. Fuster JM. Memory in the cerebral cortex. An empirical approach to neural networks in the human and nonhuman primate. Cambridge, MA:MIT Press; 1994.

    Google Scholar 

  20. Ganter B, Wille R. Formal concept analysis: mathematical foundations. Berlin: Springer-Verlag; 1999.

    Google Scholar 

  21. Harris KD, Csicsvari J, Hirase H, Dragoi G, Buzsaki G. Organization of cell assemblies in the hippocampus. Nature. 2003;424:552–5.

    Article  PubMed  CAS  Google Scholar 

  22. Hebb D. The organization of behavior. New York: Wiley; 1949.

    Google Scholar 

  23. Hertz J, Krogh A, Palmer RG. Introduction to the theory of neural computation. Redwood city: Addison Wesley; 1991.

  24. Hopfield JJ. Neural networks and physical systems with emergent collective computational abilities. PNAS. 1982;79:2554–8.

    Article  PubMed  CAS  Google Scholar 

  25. Indiveri G. Synaptic plasticity and spike-based computation in VLSI networks of integrate-and-fire neurons. Neural Inf Process—Lett Rev. 2007;11:135–46.

    Google Scholar 

  26. Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 20:1254–1259

    Article  Google Scholar 

  27. Kiani R, Esteky H, Mirpour K, Tanaka K. Object category structure in response patterns of neuronal population in monkey inferior temporal cortex. J Neurophysiol. 2007;97(6):4296–309.

    Google Scholar 

  28. Kirk U. The neural basis of object-context relationships on aesthetic judgment. PLoS ONE. 2008;3:e3754.

    Article  PubMed  CAS  Google Scholar 

  29. Knoblauch A, Markert H, Palm GG. An associative cortical model of language understanding and action planning. In: Mira J, Alvarez JR, editors. Proceedings of IWINAC 2005, First International Work-Conference on the Interplay between natural and artificial computation, Las Palmas de Gran Canaria, Spain. vol. 3562, Lecture notes in computer science. Berlin, New York: Springer; 2005. p. 405–14.

  30. Knoblauch A, Kupper R, Gewaltig M-O, Körner U, Körner E. A cell assembly based model for the cortical microcircuitry. Neurocomputing. 2007;70:1838–42.

    Article  Google Scholar 

  31. Kosko B. Bidirectional associative memory. IEEE Trans Syst Man Cybern. 1988;18:49–60.

    Article  Google Scholar 

  32. Lin L, Osan R, Shoham S, Jin W, Zuo W, Tsien JZ. Identification of network-level coding units for real-time representation of episodic experiences in the hippocampus. Proc Natl Acad Sci USA. 2005;102:6125–30.

    Article  PubMed  CAS  Google Scholar 

  33. Lin L, Osan R, Tsien JZ. Organizing principles of real-time memory encoding: neural clique assemblies and universal neural codes. Trends Neurosci. 2006;29:48–57.

    Article  PubMed  CAS  Google Scholar 

  34. Logothetis NK, Sheinberg DL. Visual object recognition. Annu Rev Neurosci. 1996;19:577–621.

    Article  PubMed  CAS  Google Scholar 

  35. Martinovic J, Gruber T, Müller MM. Coding of visual object features and feature conjunctions in the human brain. PLoS One. 2008;3:3781.

    Article  CAS  Google Scholar 

  36. Miyashita Y, Chang HS. Neural correlate of pictorial short term memory. Nature. 1988;331:68–70.

    Article  PubMed  CAS  Google Scholar 

  37. Nosofsky RM, Gluck MA, Palmeri TJ, McKinley SC, Clautier P. Comparing models of rule-based classification learning: a replication and extension of Shepard, Hovland and Jenkins (1961). Mem Cognit. 1994;22:352–69.

    PubMed  CAS  Google Scholar 

  38. Palm G. Neural assemblies: an alternative approach to artificial intelligence. Berlin: Springer-Verlag; 1982.

    Google Scholar 

  39. Palm G. Memory capacities of local rules for synaptic modification. Concepts Neurosci. 1991;2:97–128.

    Google Scholar 

  40. Palm G, Sommer F. Information capacity in recurrent McCulloch-Pitts networks with sparsely coded memory states. Network. 1992;3:177ff.

    Article  Google Scholar 

  41. Pulvermüller F. Constituents of a neurological theory of language. Concepts Neurosci. 1992;3:157–200.

    Google Scholar 

  42. Quiroga RQ, Reddy L, Kreiman G, Koch C, Fried I. Invariant visual representation by single neurons in the human brain. Nature. 2005;435:1102–7.

    Article  PubMed  CAS  Google Scholar 

  43. Quiroga RQ, Kreimann G, Koch C, Fried I. Sparse but not ’grandmother-cell’ coding in the medial temporal lobe. Trends Cogn Sci. 2008(a);12:87–91.

    Article  PubMed  Google Scholar 

  44. Quiroga RQ, Mukamel R, Malach EA, Fried I. Human single-neuron responses at the threshold of conscious recognition. Proc Natl Acad Sci USA. 2008(b);105:3599–604.

    Article  PubMed  CAS  Google Scholar 

  45. Rajapakse R, Denham M. Fast access to concepts in concept lattices via bidirectional associative memories. Neural Comput. 2005;17:2291–300.

    Article  PubMed  Google Scholar 

  46. Rao RPN, Ballard DH. Dynamic model of visual recognition predicts neural response properties in the visual cortex. Neural Comput. 1997;9:721–63.

    Article  PubMed  CAS  Google Scholar 

  47. Raz A, Buhle J. Typologies of attentional networks. Nat Rev Neurosci. 2006;7:367–79.

    Article  PubMed  CAS  Google Scholar 

  48. Riesenhuber M, Poggio T. Hierarchical models of object recognition in cortex. Nat Neurosci. 1999;2(11):1019–25.

    Article  PubMed  CAS  Google Scholar 

  49. Saalmann YB, Pigarev IN, Vidyasagar TR. Neural mechanisms of visual attention: how top-down feedback highlights relevant locations. Science. 2007;316:1612–5.

    Article  PubMed  CAS  Google Scholar 

  50. Schemmel J, Meier K, Mueller E. A new VLSI model of neural microcircuits including spike time dependent plasticity. In: Proceedings IJCNN. IEEE Press; 2007. p. 1711–6.

  51. Shepard R, Hovland CL, Jenkins HM. Learning and memorization of classifications. Psychol Monogr Gen Appl. 1961;75:1–42.

    Google Scholar 

  52. Sloman SA, Rips LJ. Similarity and symbols in human thinking. Cambridge, MA: MIT Press; 1998.

    Google Scholar 

  53. Tanaka K. Neuronal mechanisms of object recognition. Science. 1993;262:685–8.

    Article  PubMed  CAS  Google Scholar 

  54. Tsunoda K, Yamane Y, Nishizaki M, Tanifuji M. Complex objects are represented in macaque inferotemporal cortex by the combination of feature columns. Nat Neurosci. 2001;4:832–8.

    Article  PubMed  CAS  Google Scholar 

  55. Wennekers T. Operational cell assemblies as a paradigm for brain-inspired future computing architectures. Neural Inf Process—Lett Rev. 2006;10:135–45.

    Google Scholar 

  56. Wennekers T, Palm G. Cell assemblies, associative memory, and temporal structure in brain signals. In: Miller R, editors. Time and the brain. vol. 3, CABR—conceptual advances in brain research. Harwood Academic Publishers; 2000. p. 251–73.

  57. Wennekers T, Palm G. Modelling generic cognitive functions with operational Hebbian cell assemblies. In: Weiss ML, editors. Neural network research horizons. Nova Science Publishers; 2007. p. 225–94.

  58. Wennekers T, Garagnani M, Pulvermüller F. Language models based on Hebbian cell assemblies. J Neurosci (Paris). 2006;100:16–30.

    Article  Google Scholar 

  59. Wickelgren WA. Context-sensitive coding, associative memory, and serial order in (speech) behavior. Psychol Rev. 1969;76:1–15.

    Article  Google Scholar 

  60. Wickelgren WA. Webs, cell assemblies, and chunking in neural nets. Concepts Neurosci. 1992;3:1–53.

    Google Scholar 

  61. Wijekoon JHB, Dudek P. Compact silicon neuron circuit with spiking and bursting behaviour. Neural Netw. 2008;21:524–34.

    PubMed  Google Scholar 

  62. Wille R. Restructuring lattice theory: an approach based on hierarchies of concepts. In: Rival I, editor. Ordered sets. Dordrecht: Reidel; 1982. p. 445–70.

    Google Scholar 

  63. Willshaw DJ, Buneman OP, Longuet-Higgins HC. Non-holographic associative memory. Nature. 1969;222:960–2.

    Article  PubMed  CAS  Google Scholar 

  64. Wolff KE. A first course in formal concept analysis. In: Faulbaum F, editors. StatSoft ’93 Advances in statistical software. Gustav Fischer Verlag; 1993. p. 429–38.

  65. Yamashita Y, Tani J. Emergence of functional hierarchy in a multiple timescale neural network model: a humanoid robot experiment. PLoS Comput Biol 2008;4:e1000220

    Article  PubMed  CAS  Google Scholar 

Download references

Acknowledgement

The author acknowledges support by the Engineering and Physical Sciences Research Council (grant EP/C010841/1, COLAMN—A Novel Computing Architecture for Cognitive Systems based on the Laminar Microcircuitry of the Neocortex) and the European Community (FACETS—Fast Analog Computing with Emergent Transient States). Comments by the reviewer are also greatly acknowledged.

Author information

Authors and Affiliations

  1. Centre for Theoretical and Computational Neuroscience, School of Psychology, University of Plymouth, Portland Square Building, Room A218, PL4 8AA, Plymouth, Devon, UK

    Thomas Wennekers

Authors
  1. Thomas Wennekers
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thomas Wennekers.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wennekers, T. On the Natural Hierarchical Composition of Cliques in Cell Assemblies. Cogn Comput 1, 128–138 (2009). https://doi.org/10.1007/s12559-008-9004-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-008-9004-5

Keywords

Search

Navigation