Abstract
We study asymmetric stochastic networks from two points of view: combinatorial optimization and learning algorithms based on relative entropy minimization. We show that there are non trivial classes of asymmetric networks which admit a Lyapunov function ℒ under deterministic parallel evolution and prove that the stochastic augmentation of such networks amounts to a stochastic search for global minima of ℒ. The problem of minimizing ℒ for a totally antisymmetric parallel network is shown to be associated to an NP-complete decision problem. The study of entropic learning for general asymmetric networks, performed in the non equilibrium, time dependent formalism, leads to a Hebbian rule based on time averages over the past history of the system. The general algorithm for asymmetric networks is tested on a feed-forward architecture.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ackely DH, Hinton GE, Sejnowski TJ (1985) A learning algorithm for Boltzmann machines. Cogn Sci 9:147–169
Amit DJ (1989) Modeling brain function. Cambridge University Press, Cambridge
Apolloni B, de Falco D (1989) Learning by parallel Boltzmann machines. Milano preprint (1989) IEEE Trans Inf Theor: (to be published)
Apolloni B, de Falco D (1991) Learning by feed-forward Boltzmann machines. Proceedings Neuronet-90. World Scientific, Singapore: (to be published)
Baldissera F, Hultborn H, Illert M (1981) Integration in spinal neuronal systems. In: Brooks VB (ed) Handbook of physiology, vol II. Am Phys Soc
Barahona F (1982) On the computational complexity of Ising spin glass models. J Phys A: Math Gen 15:3241–3253
Bruck J, Goodman JW (1988) A generalized convergence theorem for neural networks. IEEE Trans Inf Theor 34:1089–1092
Cellini B (1562) Vita di Benvenuto Cellini scritta da lui medesimo. The pedagogical motivations of the name of the algorithm can be found on page 8 of the 1925 Hoepli; Milan edn
Fogelman F, Goles E, Pellegrin D (1986) Decreasing energy functions as a tool for studying threshold networks. Disc Appl Math 12:261–277
Garey MR, Johnson DS (1979) Computers and intractability. Freeman, San Francisco
Goles E (1986) Positive automata networks. In: Bienestock E, Fogelman F, Weisbuch G (eds) Disordered system and biological organization. Springer, Berlin Heidelberg New York
Hopfield JJ (1982) Neural networks and physical systems with emergent collective computational abilities. Proc Natl Acad Sci (USA) 79:2554–2558
Kullback S (1959) Information theory and statistics. Wiley, New York
Little WA (1974) The existence of persistent states in the brain. Math Biosci 19:101–120
Parisi G (1986) Asymmetric neural networks and the process of learning J Phys A19:L675–680
Renshaw B (1946) Central effects of centripetal impulses in axons of spinal ventral roots J Neurobiophysiol 9:167–183
Rumelhart DE, Hinton GE, Williams RJ (1986) Learning internal representation by error backpropagation. In: Parallel distributed processing: exploration in microstructures of cognition, vol I. MIT Press, Cambridge
Shaw GL, Vasudevan R (1974) Persistent state of neural networks and the random nature of synaptic transmission. Math Biosci 21:207–218
Author information
Authors and Affiliations
Additional information
This research was supported in part by C.N.R. under grants 88.03556.12 and 89.05261.CT12
Rights and permissions
About this article
Cite this article
Apolloni, B., Bertoni, A., Campadelli, P. et al. Asymmetric Boltzmann machines. Biol. Cybern. 66, 61–70 (1991). https://doi.org/10.1007/BF00196453
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/BF00196453