Abstract
We investigate the convergence properties of the self-organizing feature map algorithm for a simple, but very instructive case: the formation of a topographic representation of the unit interval [0,1] by a linear chain of neurons. We extend the proofs of convergence of Kohonen and of Cottrell and Fort to hold in any case where the neighborhood function, which is used to scale the change in the weight values at each neuron, is a monotonically decreasing function of distance from the winner neuron. We prove that the learning dynamics cannot be described by a gradient descent on a single energy function, but may be described using a set of potential functions, one for each neuron, which are independently minimized following a stochastic gradient descent. We derive the correct potential functions for the oneand multi-dimensional case, and show that the energy functions given by Tolat (1990) are an approximation which is no longer valid in the case of highly disordered maps or steep neighborhood functions.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Cottrell M, Fort JC (1987) Etude d'un processus d'auto-organization. Ann Inst Henri Poincaré 23:1–20
Durbin R, Mitchison G (1990) A dimension reduction framework for understanding cortical maps. Nature 343:644–647
Durbin R, Willshaw D (1987) An analogue approach to the travelling salesman problem using an elastic net method. Nature 326:689–691
Erwin E, Obermayer K, Schulten K (1992) Self-organizing maps: Stationary states, metastability and convergence rate. Biol Cybern (this issue)
Favata F, Walker R (1991) A study of the application of Kohonentype neural networks to the travelling salesman problem. Biol Cybern 64:463–468
Geszti T (1990) Physical models of neural networks. World Scientific, Singapore
Hertz J, Krogh A, Palmer RG (1991) Introduction to the theory of neural computation. Addison-Wesley, Redwood City, California
Kohonen T (1982a) Analysis of a simple self-organizing process. Biol Cybern 44:135–140
Kohonen T (1982b) Self-organized formation of topologically correct feature maps. Biol Cybern 43:59–69
Kohonen T (1988) Self-organization and associative memory. Springer, New York Berlin Heidelberg
Kohonen T (1989) Speech recognition based on topology preserving neural maps. In: Aleksander I (ed) Neural computation. London, Kogan Page
Kohonen T (1991) Self-organizing maps: optimization approaches. In: Kohonen T et al. (ed) Artificial neural networks, vol II. North Holland, Amsterdam, pp 981–990
Lo ZP, Bavarian B (1991) On the rate of convergence in topology preserving neural networks. Biol Cybern 65:55–63
Luttrell SP (1989) Self-organization: a derivation from first principles of a class of learning algorithms. In: Proc. 3rd IEEE Int. Joint Conf. on Neural Networks, vol II. Washington, pp 495–498, IEEE Neural Networks Council
Moon P, Spencer DE (1969) Partial differential equations. Heath, Lexington, Mass
Obermayer K, Ritter H, Schulten K (1990a) Large-scale simulations of self-organizing neural networks on parallel computers: Applications to biological modelling. Parallel Computer 14: 381–404
Obermayer K, Ritter H, Schulten K (1990b) A principle for the formation of the spatial structure of cortical feature maps. Proc Natl Acad Sci USA 87:8345–8349
Obermayer K, Blasdel GG, Schulten K (1991) A neural network model for the formation of the spatial structure of retinotopic maps, orientationand ocular dominance columns. In: Kohonen T et al. (ed) Artificial neural networks, vol I. North Holland, Amsterdam, pp 505–511
Ritter H (1988) Selbstorganisierende neuronale Karten. PhD thesis. Technische Universität München
Ritter H, Kohonen T (1989) Self-organizing semantic maps. Biol Cybern 61:241–254
Ritter H, Schulten K (1986) On the stationary state of Kohonen's self-organizing sensory mapping. Biol Cybern 54:99–106
Ritter H, Schulten K (1988) Convergence properties of Kohonen's topology conserving mappings: Fluctuatuions, stability and dimension selection. Biol Cybern 60:59–71
Ritter H, Martinetz T, Schulten K (1989) Topology conserving maps for learning visuomotor-coordination. Neural Networks 2:159–168
Robbins H, Munro S (1951) A stochastic approximation method. Ann Math Statist 22:400–407
Schottes JC (1991) Unsupervised context learning in natural language processing. In: Proc 5th IEEE Int. Joint Conf. on Neural Networks, vol I. Washington, pp 107–112, IEEE Neural Networks Council
Simic PD (1989) Statistical mechanics as the underlying theory of ‘elastic’ and ‘neural’ optimizations. Network 1:89–103
Tolat VV (1990) An analysis of Kohonen's self-organizing maps using a system of energy functions. Biol Cybern 64:155–164
van Kämpen NG (1981) Stochastic process in physics and chemistry. North-Holland, Amsterdam
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Erwin, E., Obermayer, K. & Schulten, K. Self-organizing maps: ordering, convergence properties and energy functions. Biol. Cybern. 67, 47–55 (1992). https://doi.org/10.1007/BF00201801
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/BF00201801