Original Contribution
Hamiltonian dynamics of neural networks

https://doi.org/10.1016/S0893-6080(05)80058-9Get rights and content

Abstract

The activation and weight dynamics of Artificial Neural Networks are derived from a partial differential equation (PDE) that may incorporate weights either as parameters or variables. It is shown that a single first-order Hamilton-Jacobi “parametrical” PDE suffices to derive the various neurodynamical paradigms used today. In the case that weights are taken as variables, a new type of neurodynamics is discovered: A Hamilton function is derived so that the weights obey a second-order ordinary differential equation (ODE). As this ODE models the forces, experienced by the weights in the presence of some generalized error potential, it is called a learning law. Results obtained for the association of time-varying patterns, using parametrical as well as dynamical weights, show that learning rules can be replaced by learning laws at equal performance.

References (31)

  • FaybusovichL.

    Hamiltonian structure of dynamical systems which solve linear programming problems

    Physica D

    (1991)
  • WerbosP.J.

    Generalization of backpropagation with application to a recurrent gas market model

    Neural Networks

    (1988)
  • AlmeidaL.B.

    Backpropagation in non-feedforward networks

  • AmitD. et al.

    Information storage in neural networks with low levels of activity

    Physical Review A

    (1987)
  • BellmanR.

    Mathematical optimization techniques

    (1960)
  • ChuaL.O. et al.

    Cellular neural networks: Theory and applications

    IEEE Transactions on Circuits and Systems

    (1988)
  • FarotimiO. et al.

    A general weight matrix formulation using optimal control

    IEEE Transactions on Neural Networks

    (1991)
  • GardnerE.

    The space of interactions in neural network models

    Journal of Physics A

    (1988)
  • GardnerE. et al.

    Optimal storage properties of neural network models

    Journal of Physics A

    (1988)
  • GawronskaD.A. et al.

    Recurrent neural networks for function association

  • van HemmenJ.L. et al.

    Collective phenomena in neural networks

  • HertzJ.A.

    Statistical dynamics of learning

    (1990)
  • HertzJ. et al.

    Introduction to the theory of neural computation

    (1991)
  • HirschM.W.

    Network dynamics: Principles and problems

  • HopfieldJ.J.

    Neural networks and physical systems with emergent collective computational abilities

  • Cited by (8)

    • Quantum neural network

      1997, International Journal of Theoretical Physics
    View all citing articles on Scopus
    View full text