Original contribution
Correlations in high dimensional or asymmetric data sets: Hebbian neuronal processing

https://doi.org/10.1016/0893-6080(91)90070-LGet rights and content

Abstract

The Hebbian neural learning algorithm that implements Principal Component Analysis (PCA) can be extended for the analysis of more realistic forms of neural data by including higher than two-channel correlations and non-Euclidean 1p metrics. Maximizing a dth rank tensor form which correlates d channels is equivalent to raising the exponential order of variance correlation from 2 to d in the algorithm that implements PCA. Simulations suggest that a generalized version of Oja's PCA neuron can detect such a dth order principal component. Arguments from biology and pattern recognition suggest that neural data in general is not symmetric about its mean; performing PCA with an implicit 1l metric rather than the Euclidean metric weights exponentially distributed vectors according to their probability, as does a highly nonlinear Hebb rule. The correlation order d and the 1p metric exponent p were each roughly constant for each of several Hebb rules simulated. High-order correlation analysis may prove increasingly useful as data from large networks of cells engaged in information processing becomes available.

References (36)

  • P. Baldi et al.

    Neural networks and principal component analysis: learning from examples withour local minima

    Neural Networks

    (1989)
  • Y.C. Lee et al.

    Machine learning using a higher order correlation network

    Physics

    (1986)
  • J. Atick et al.

    Towards a theory of early visual processing

    Neural Computation

    (1990)
  • H.B. Barlow

    Possible principles underlying the transformation of sensory messages

  • H.B. Barlow

    Critical limiting factors in the design of the eye and visual cortex

  • E. Bienenstock et al.

    Theory for the development of neuron selectivity, orientation specificity and binocular interaction in visual cortex

    N. Neurosci.

    (1982)
  • R. Duda et al.

    Pattern classification & scene analysis

    (1973)
  • D. Gabor

    Theory of communication

    J. Inst. Electr. Eng.

    (1946)
  • S. Grossberg

    On the development of feature detectors in the visual cortex with applications to learning and reaction-diffusion systems

    Biol. Cybernetics

    (1976)
  • R.L.V. Hartley

    Transmission of information

    Bell Systems Technical Journal

    (1928)
  • D.O. Hebb

    The Organization of Behavior

    (1949)
  • B. Julesz et al.

    Visual discrimination of textures with identical third-order statistics

    Biol. Cybernetics

    (1978)
  • D.M. Kammen

    Self-organization in neural networks

  • D.M. Kammen et al.

    Spontaneous symmetry-breaking energy functions and the emergence of orientation selective cortical cells

    Biol. Cybern.

    (1988)
  • D.M. Kammen et al.

    Self organizing neural networks: Hebbian learning in development and biological computing

  • J.J. Koenderink

    The structure of images

    Biol. Cybernetics

    (1984)
  • T. Kohonen

    Self-organization and associative memory

  • A. Krough et al.

    Hebbian learning of principal components

  • Cited by (46)

    • Adaptive Machine Learning Algorithms with Python: Solve Data Analytics and Machine Learning Problems on Edge Devices

      2022, Adaptive Machine Learning Algorithms with Python: Solve Data Analytics and Machine Learning Problems on Edge Devices
    • Emerging capabilities and applications of artificial higher order neural networks

      2021, Emerging Capabilities and Applications of Artificial Higher Order Neural Networks
    • Stability and Synchronization of a Fractional Neutral Higher-Order Neural Network System

      2020, International Journal of Nonlinear Sciences and Numerical Simulation
    View all citing articles on Scopus

    This work was supported by grants to C. Koch from James S. McDonnell Foundation, the Air Force Office of Scientific Research, and a NSF Presidential Young Investigator Award.

    1

    D. M. K. is supported by a Weizmann Postdoctoral Fellowship from the Division of Biological Sciences.

    View full text