The impact of energy function structure on solving generalized assignment problem using Hopfield neural network

Abstract

In the last 20 years, neural networks researchers have exploited different penalty based energy functions structures for solving combinatorial optimization problems (COPs) and have established solutions that are stable and convergent. These solutions, however, have in general suffered from lack of feasibility and integrality. On the other hand, operational researchers have exploited different methods for converting a constrained optimization problem into an unconstrained optimization problem. In this paper we have investigated these methods for solving generalized assignment problems (GAPs). Our results concretely establishes that the augmented Lagrangean method can produce superior results with respect to feasibility and integrality, which are currently the main concerns in solving neural based COPs.

Introduction

It has been about two decades since neural networks were first applied to solve combinatorial optimization problems (COPs). It was expected that the inherent parallel processing and analog nature of the interconnected neurons or neural net could create a rapid and powerful solution technique.

In 1985, Hopfield and Tank [7] proposed a model to solve TSP, a famous COP. The network, as a dynamical system, was represented by an energy function that should be made equivalent to the objective function of the combinatorial problem that needs to be minimized, while the constraints of the problem are included in the energy function as penalty terms. They simulated a network of 10 cities (100 neurons), chosen at random on the interior of a two-dimensional unit square. For 20 random starts, 16 converged to valid tours. About 50% of the trials produced one of the two known shortest tours.

In 1988 Wilson and Pawley [13] raised serious doubts as to the reliability and validity of the H–T approach to solving TSPs. They tried to reproduce the H–T solutions for the 10-city problem. Based on 100 random starts, only 15 converged to valid tours, while 45 froze into local minima corresponding to invalid tours, and the remaining 40 did not converge within 1000 iterations. Moreover, for the 15 valid tours they were only slightly better than randomly chosen tours. These results were dramatically different from those of Hopfield and Tank on the identical problem. They felt that it is the structure of energy function, and not the initialization scheme, which is the root of the problems.

In 1992, Abe et al. [1] extended the Hopfield neural networks model to solve inequality constrained combinatorial optimization problems where linear combinations of variables are lower- or upper-bounded. They proposed lower bounds on the energy function penalty terms that could guarantee feasibility. They tested their method on a small instance of knapsack problem (with 6 items) and a small instance of transportation problem (with 6 nodes). However, solutions thus obtained were feasible only if integrality constraints were relaxed, in the contrary to what is expected in COPs.

In 1996, Watta and Hassoun [12] developed a neural net for mixed integer programming. In this network continuous neurons were used to represent both continuous and integer variables and integrality constraints were relaxed. When the network converged, the outputs of these continuous neurons were threshold to yield integer solutions. This is similar to solving a mixed integer program using LP relaxation and rounding up and down the results to obtain integral values. From OR point of view, it is obvious that the solutions so obtained is infeasible in most cases.

In 1999, Walsh et al. [11] showed that between 20% and 45% of experiments on solving a 10 generators scheduling problem using model of Watta and Hassoun [12] led to infeasible solutions. They considered that the difficulty is due to the fact that continuous neurons were used to represent integer variables. They modified the previous model by incorporating discrete neurons to represent integer variables. They tested their model on a 17 generator scheduling problem in which convergence into infeasible solutions were seen.

In 1999, Gall and Zissimopoulos [5] improved the work of Abe et al. [1] and proposed a modification into the penalty function structure of Abe et al. called competitive activation mechanism (CAM), by which more integral values could be produced. They tested their method on a group of medium size set covering problem instances with number of sets ranging between 60 and 150 and number of elements ranging between 30 and 50. Still they reported that on average 20% of variables reach their stable states while having intermediate values, between 0 and 1, not maintaining integrality constraints.

Upon the analysis provided above we can conclude that in spite of the two decades of research [3], [4], [7], [9], [10], neural based combinatorial solutions have suffered from integrality and feasibility properties though stability and convergence have been maintained. To improve this situation, we have investigated four different methods proposed in OR literature, for building energy function structure to solve general assignment problems (GAPs).

While there existed a number of reliable and efficient exact and approximate algorithms for solving GAPs [14], neural network modeling is important for their promise of rapid execution through hardware implementation. Many real life applications demand this feature including crew scheduling and weapon target assignments [10].

In Section 2 the GAP is defined and a numerical example is provided. In Section 3 the problem is solved using four different methods and the results are analyzed. It is shown that the augmented Lagrangean method can improve the solutions with respect to feasibility and integrality properties, while maintaining stability and convergence properties. Conclusions are drawn in Section 4.