Elsevier

Neurocomputing

Volume 34, Issues 1–4, September 2000, Pages 55-77
Neurocomputing

Potts models with two sets of interactive dynamics

https://doi.org/10.1016/S0925-2312(00)00303-9Get rights and content

Abstract

In this work, we develope Potts models with two sets of interactive dynamics. We derive the mean field annealing for a new energy function which contains two sets of neural variables, one for combinatorial constraints and the other for internal geometrical representations. Two sets of interactive dynamics are further developed for both sets of neural variables. The obtained Potts model thus possesses two sets of dynamics and exactly fits the requirement of the hairy model proposed by Hzu. We call the new Potts model as hairy Potts neural network. The elastic ring method proposed by Durbin and Willshaw and the Potts model of Peterson and Söderberg, and the Hopfield's TSP modeling are shown to be special forms of the hairy Potts neural network. We explore various energy functions for the hairy Potts neural network and test the new network with simulations. The results are encouraging.

Introduction

Neural networks for many combinatorial optimizations have been modeled by systems of Ising spins with energy functions [19], [20]. Mean field annealing techniques have been derived for neural networks [17], [22] from simulated annealing [12] and naive mean field [21]. The dynamics developed by the mean field annealing helps find deep minima of the energy function, which are well designed to encode satisfactory solutions of combinatorial optimizations. The mean field annealing significantly improves the performance of the feedback neural network for solving problems and maintains the parallel and distributed architecture for neural computing. Further, Potts neural units [17] were introduced for proper internal representation of a neural network. Without the limitation of being two states as an Ising spin, Potts models highly reduce the complexity of interconnections and the searching space of a neural network.

Given an energy function, the derivation of the mean field annealing for the neural network can be systematized as two steps. At first, the mean energy and the entropy of a neural network are defined as the analogous macroscopic quantities of a physical system for the temperature. Then the mean field equations of the neural network are derived by minimizing the mean energy and maximizing the entropy simultaneously, which is just the way to derive the stationary point of the free energy for a physical system at thermal equilibrium [23]. It is well known that the temperature is scheduled from a high value to a small value carefully to guarantee a high quality of network relaxation.

In this paper, we will first devise a new energy function that models the elastic ring for the traveling salesman problem (TSP). The energy function contains two kinds of variables. By applying mean field annealing to these two kinds of variables separately, we obtain two sets of dynamics that describe a new neural network. This new neural network belongs to the hairy model proposed by Hzu [10]. We call it HAPER (hairy potts elastic ring) network. The elastic ring method proposed by Durbin and Willshaw [6], the Potts modeling of Peterson and Söderberg [17] and the Hopfield's TSP modeling [9] are shown to be special forms of the HAPER network. And several other energy functions for the HAPER networks are further explored.

Section snippets

HAPER network for elastic ring

The technique of elastic ring is known to have wide applications, including the traveling salesman problem (TSP) [6], the formation of ocular dominance stripes [7], and line extraction [4], etc. We will focus on the elastic ring for the TSP problem in our work to simplify and clear the contexts. The detailed relation between the elastic ring and the Hopfield neural network has been shown by Simic [18] through the underlying theory of statistical mechanism and by Yuille [24] through works on

Examining the elastic ring method proposed by Durbin and Willshaw

In this section, we will show that the elastic ring method proposed by Durbin and Willshaw [6] is a special form of the HAPER network. In the HAPER network, the mean field annealing is employed to determine the dynamics of the combinatorial variables and the geometrical variables. If we do not apply the mean field annealing to the geometrical variables and simply set the time ratio of the mean of each geometrical variable yα in proportion to the negative derivation of the free energy ψσe in Eq.

Other HAPER networks

The notations of σi,xi and yα are followed. One can solve a N-city TSP byminimizingL=12iασ(|xiyα−1|+|xiyα+1|)subjectedto(a)σi∈{e1,…,eN},1≤i≤N,(b)qα∈{e1,…,eN},1≤α≤N,(c)iσ=1,1≤α≤N,(d)αqαi=1,1≤i≤N,(e)iσ|xiyα|=0,1≤α≤N,(f)yα=iqαixi,1≤α≤N.where qα is an unit vector (by (b)) which is used to indicate the city located by the node α (by (f)). The model (35) constitutes a mixed integer linear programming for TSP, which contains two kinds of variables, one for Potts neurons, playing the

Simulations and discussions

In this section, we will first compare the performance of the HAPER network modeled by the energy function H (36) with those of the related algorithms, including simulated annealing [12], elastic ring of Durbin and Willshaw [6], self-organization-based algorithm [3] and original Potts neural encoding [17], and then summarize our work. We use a 100-city TSP to test these algorithms. The positions of the cities of the 100-city TSP are uniformly generated within a 1×1 unit square. The problem is

Acknowledgements

We thank the reviewers for fruitful comments.

Jiann-Ming Wu was born in Taiwan, on November 22, 1966. He received the B.S. degree in computer engineering in 1988, from National Chiao Tung University, the MS. degree 1990 and the Ph.D. degree in 1994 in computer science and information engineering from National Taiwan University. In 1996, he joined the Faculty as an assistant professor at the Department of Applied Mathematics in National DongHwa University, where he is currently an associate professor. His current research interests include

References (25)

  • M. Padberg et al.

    Optimization of a 532-city symmetric traveling salesman problem by branch and cut

    Oper. Res. Lett.

    (1987)
  • S.V.B. Aiyer, Training hidden Markov models using mean field annealing network, IJCNN’92-Beijing, 1992, pp....
  • S.V.B. Aiyer et al.

    A theoretical investigation into the performance of the Hopfield model

    IEEE Trans. Neural Networks

    (1990)
  • B. Angeniol et al.

    Self-organizing feature maps and the travelling salesman problem

    Neural Network

    (1988)
  • D.J. Burr

    Elastic matching of line drawings

    IEEE Trans. Pattern Anal. Machine Intell. PAMI-3

    (1980)
  • D.W. Dong et al.

    Dynamic properties of neural networks with adapting synapses

    Network

    (1992)
  • R. Durbin et al.

    An analogue approach to the travelling salesman problem using an elastic net method

    Nature

    (1987)
  • G.J. Goodhill et al.

    Application of the elastic net algorithm to the formation of ocular dominance stripes

    Network

    (1990)
  • M.R. Gravey et al.

    Computer and Intractability: A Guide to the Theory of NP Completeness

    (1990)
  • J.J. Hopfield, D.W. Tank, Neural computation of decisions in optimization problems. Biol. Cybernet. 52 (1985)...
  • Hzu, H. Harold, Reconfigurable neural nets by energy convergence learning principle based on extended McCulloch–Pitts...
  • I. Kanter

    Potts-glass models of neural networks

    Phys. Rev. A

    (1988)
  • Cited by (4)

    • Annealed Kullback-Leibler divergence minimization for generalized TSP, spot identification and gene sorting

      2011, Neurocomputing
      Citation Excerpt :

      By numerical simulations, it has been shown that physical-like deterministic annealing is crucial for improving KL minimization of learning an LCGM model. The advantage of overcoming the local minimum problem by physical-like deterministic annealing has been pioneered in [22,31,32] and extensively verified for solving various fundamental tasks in previous works [3,5,20,21]. Iterative execution of interactive dynamics under a fixed temperature constitutes an operation of seeking the saddle point of the KL divergence in the physical-like annealing process (13), where the temperature-like parameter is carefully scheduled from sufficiently high to low values.

    • Multilayer Potts perceptrons with Levenberg-Marquardt learning

      2008, IEEE Transactions on Neural Networks
    • Annealing by two sets of interactive dynamics

      2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

    Jiann-Ming Wu was born in Taiwan, on November 22, 1966. He received the B.S. degree in computer engineering in 1988, from National Chiao Tung University, the MS. degree 1990 and the Ph.D. degree in 1994 in computer science and information engineering from National Taiwan University. In 1996, he joined the Faculty as an assistant professor at the Department of Applied Mathematics in National DongHwa University, where he is currently an associate professor. His current research interests include neural networks and signal processing.

    View full text