Potts models with two sets of interactive dynamics
Introduction
Neural networks for many combinatorial optimizations have been modeled by systems of Ising spins with energy functions [19], [20]. Mean field annealing techniques have been derived for neural networks [17], [22] from simulated annealing [12] and naive mean field [21]. The dynamics developed by the mean field annealing helps find deep minima of the energy function, which are well designed to encode satisfactory solutions of combinatorial optimizations. The mean field annealing significantly improves the performance of the feedback neural network for solving problems and maintains the parallel and distributed architecture for neural computing. Further, Potts neural units [17] were introduced for proper internal representation of a neural network. Without the limitation of being two states as an Ising spin, Potts models highly reduce the complexity of interconnections and the searching space of a neural network.
Given an energy function, the derivation of the mean field annealing for the neural network can be systematized as two steps. At first, the mean energy and the entropy of a neural network are defined as the analogous macroscopic quantities of a physical system for the temperature. Then the mean field equations of the neural network are derived by minimizing the mean energy and maximizing the entropy simultaneously, which is just the way to derive the stationary point of the free energy for a physical system at thermal equilibrium [23]. It is well known that the temperature is scheduled from a high value to a small value carefully to guarantee a high quality of network relaxation.
In this paper, we will first devise a new energy function that models the elastic ring for the traveling salesman problem (TSP). The energy function contains two kinds of variables. By applying mean field annealing to these two kinds of variables separately, we obtain two sets of dynamics that describe a new neural network. This new neural network belongs to the hairy model proposed by Hzu [10]. We call it HAPER (hairy potts elastic ring) network. The elastic ring method proposed by Durbin and Willshaw [6], the Potts modeling of Peterson and Söderberg [17] and the Hopfield's TSP modeling [9] are shown to be special forms of the HAPER network. And several other energy functions for the HAPER networks are further explored.
Section snippets
HAPER network for elastic ring
The technique of elastic ring is known to have wide applications, including the traveling salesman problem (TSP) [6], the formation of ocular dominance stripes [7], and line extraction [4], etc. We will focus on the elastic ring for the TSP problem in our work to simplify and clear the contexts. The detailed relation between the elastic ring and the Hopfield neural network has been shown by Simic [18] through the underlying theory of statistical mechanism and by Yuille [24] through works on
Examining the elastic ring method proposed by Durbin and Willshaw
In this section, we will show that the elastic ring method proposed by Durbin and Willshaw [6] is a special form of the HAPER network. In the HAPER network, the mean field annealing is employed to determine the dynamics of the combinatorial variables and the geometrical variables. If we do not apply the mean field annealing to the geometrical variables and simply set the time ratio of the mean of each geometrical variable in proportion to the negative derivation of the free energy ψσe in Eq.
Other HAPER networks
The notations of and are followed. One can solve a N-city TSP bywhere is an unit vector (by (b)) which is used to indicate the city located by the node α (by (f)). The model (35) constitutes a mixed integer linear programming for TSP, which contains two kinds of variables, one for Potts neurons, playing the
Simulations and discussions
In this section, we will first compare the performance of the HAPER network modeled by the energy function (36) with those of the related algorithms, including simulated annealing [12], elastic ring of Durbin and Willshaw [6], self-organization-based algorithm [3] and original Potts neural encoding [17], and then summarize our work. We use a 100-city TSP to test these algorithms. The positions of the cities of the 100-city TSP are uniformly generated within a 1×1 unit square. The problem is
Acknowledgements
We thank the reviewers for fruitful comments.
Jiann-Ming Wu was born in Taiwan, on November 22, 1966. He received the B.S. degree in computer engineering in 1988, from National Chiao Tung University, the MS. degree 1990 and the Ph.D. degree in 1994 in computer science and information engineering from National Taiwan University. In 1996, he joined the Faculty as an assistant professor at the Department of Applied Mathematics in National DongHwa University, where he is currently an associate professor. His current research interests include
References (25)
- et al.
Optimization of a 532-city symmetric traveling salesman problem by branch and cut
Oper. Res. Lett.
(1987) - S.V.B. Aiyer, Training hidden Markov models using mean field annealing network, IJCNN’92-Beijing, 1992, pp....
- et al.
A theoretical investigation into the performance of the Hopfield model
IEEE Trans. Neural Networks
(1990) - et al.
Self-organizing feature maps and the travelling salesman problem
Neural Network
(1988) Elastic matching of line drawings
IEEE Trans. Pattern Anal. Machine Intell. PAMI-3
(1980)- et al.
Dynamic properties of neural networks with adapting synapses
Network
(1992) - et al.
An analogue approach to the travelling salesman problem using an elastic net method
Nature
(1987) - et al.
Application of the elastic net algorithm to the formation of ocular dominance stripes
Network
(1990) - et al.
Computer and Intractability: A Guide to the Theory of NP Completeness
(1990) - J.J. Hopfield, D.W. Tank, Neural computation of decisions in optimization problems. Biol. Cybernet. 52 (1985)...
Potts-glass models of neural networks
Phys. Rev. A
Cited by (4)
Annealed Kullback-Leibler divergence minimization for generalized TSP, spot identification and gene sorting
2011, NeurocomputingCitation Excerpt :By numerical simulations, it has been shown that physical-like deterministic annealing is crucial for improving KL minimization of learning an LCGM model. The advantage of overcoming the local minimum problem by physical-like deterministic annealing has been pioneered in [22,31,32] and extensively verified for solving various fundamental tasks in previous works [3,5,20,21]. Iterative execution of interactive dynamics under a fixed temperature constitutes an operation of seeking the saddle point of the KL divergence in the physical-like annealing process (13), where the temperature-like parameter is carefully scheduled from sufficiently high to low values.
Multilayer Potts perceptrons with Levenberg-Marquardt learning
2008, IEEE Transactions on Neural NetworksAnnealing by two sets of interactive dynamics
2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
Jiann-Ming Wu was born in Taiwan, on November 22, 1966. He received the B.S. degree in computer engineering in 1988, from National Chiao Tung University, the MS. degree 1990 and the Ph.D. degree in 1994 in computer science and information engineering from National Taiwan University. In 1996, he joined the Faculty as an assistant professor at the Department of Applied Mathematics in National DongHwa University, where he is currently an associate professor. His current research interests include neural networks and signal processing.