Abstract
Kwok and Smith [1] recently proposed a new kind of optimization dynamics using self-organizing neural networks (SONN) driven by softmax weight renormalization. Such dynamics is capable of powerful intermittent search for high-quality solutions in difficult assignment optimization problems. However, the search is sensitive to temperature setting in the softmax renormalization step of the SONN algorithm. It has been hypothesized that the optimal temperature setting corresponds to symmetry breaking bifurcation of equilibria of the renormalization step, when viewed as an autonomous dynamical system called iterative softmax (ISM). We rigorously analyze equilibria of ISM by determining their number, position and stability types. Moreover, we offer analytical approximations to the critical symmetry breaking bifurcation temperatures that are in good agreement with those found by numerical investigations. So far the critical temperatures have been determined only via numerical simulations. On a set of N-queens problems for a wide range of problem sizes N, the analytically determined critical temperatures predict the optimal working temperatures for SONN intermittent search very well.
Keywords
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Kwok, T., Smith, K.: Optimization via intermittency with a self-organizing neural network. Neural Computation 17, 2454–2481 (2005)
Hopfield, J.: Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Science, USA, 79, 2554–2558 (1982)
Smith, K.: Neural networks for combinatorial optimization: a review of more than a decade of research. INFORMS Journal on Computing 11, 15–34 (1999)
Smith, K.: Solving the generalized quadratic assignment problem using a self-organizing process. In: Proceedings of the IEEE Int. Conf. on Neural Networks, vol. 4, pp. 1876–1879 (1995)
Smith, K., Palaniswami, M., Krishnamoorthy, M.: Neural techniques for combinatorial optimization with applications. IEEE Transactions on Neural Networks 9, 1301–1318 (1998)
Kwok, T., Smith, K.: Performance-enhancing bifurcations in a self-organising neural network. In: Mira, J., Álvarez, J.R. (eds.) IWANN 2003. LNCS, vol. 2686, pp. 390–397. Springer, Heidelberg (2003)
Langton, C.: Computation at the edge of chaos: Phase transitions and emergent computation. Physica D 42, 12–37 (1990)
Tiňo, P.: Equilibria of iterative softmax and critical temperatures for intermittent search in self-organizing neural networks. Technical Report CSRP-06-07, University of Birmingham, School of Computer Science (2006), http://www.cs.bham.ac.uk/~pxt/PAPERS/tino.tr06.pdf
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Tiňo, P. (2006). Critical Temperatures for Intermittent Search in Self-Organizing Neural Networks. In: Runarsson, T.P., Beyer, HG., Burke, E., Merelo-Guervós, J.J., Whitley, L.D., Yao, X. (eds) Parallel Problem Solving from Nature - PPSN IX. PPSN 2006. Lecture Notes in Computer Science, vol 4193. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11844297_64
Download citation
DOI: https://doi.org/10.1007/11844297_64
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-38990-3
Online ISBN: 978-3-540-38991-0
eBook Packages: Computer ScienceComputer Science (R0)