A hybrid neural approach to combinatorial optimization

doi:10.1016/0305-0548(95)00064-X

Computers & Operations Research

Volume 23, Issue 6, June 1996, Pages 597-610

https://doi.org/10.1016/0305-0548(95)00064-X Get rights and content

Abstract

Both the Hopfield neural network and Kohonen's principles of self-organization have been used to solve difficult optimization problems, with varying degrees of success. In this paper, a hybrid neural network is presented which combines, for the first time, a new self-organizing approach to optimization with a Hopfield network. It is demonstrated that many of the traditional problems associated with each of these approaches can be resolved when they are combined into a hybrid model. After presenting the broad class of 0–1 optimization problems for which this hybrid neural approach is suited, details of the algorithm are presented and convergence properties are discussed. This hybrid neural approach is applied to solve a practical sequencing problem from the car manufacturing industry. Performance results are compared with classical as well as other neural techniques, and conclusions are drawn.

References (28)

B. Angéniol et al.
Self organizing feature maps and the Travelling Salesman Problem
Neural Networks
(1988)
J.J. Hopfield et al.
“Neural” computation of decisions in optimization problems
Biological Cybernet.
(1985)
D.W. Tank et al.
Simple neural optimization networks: an A/D converter, signal decision circuit and a linear programming circuit
IEEE Trans. Circuit Syst.
(1986)
G. Tagliarini et al.
Solving Constraint Satisfaction Problems with neural networks
G.V. Reklaitis et al.
Generalised Hopfield Networks and nonlinear optimization
G.V. Wilson et al.
On the stability of the TSP algorithm of Hopfield and Tank
Biological Cybernet.
(1988)
S. Hedge et al.
Determination of parameters in a Hopfield/Tank computational network
B. Kamgar-Parsi et al.
Dynamical stability and parameter selection in neural optimization
W.K. Lai et al.
Genetic breeding of control parameters for the Hopfield.Tank neural net
R.D. Brandt et al.
Alternative networks for solving the Travelling Salesman Problem and the List-Matching Problem

D.E. Van den Bout et al.

A Travelling Salesman objective function that works

A.H. Gee et al.

Limitations of neural networks for solving Traveling Salesman Problems

IEEE Trans. Neural Networks

(1995)

T. Kohonen

Self-organized formation of topologically correct feature maps

Biological Cybernet.

(1982)

L. Fang et al.

Design of competition-based neural networks for combinatorial optimization

Int. J. Neural Syst.

(1990)

Cited by (42)

How can artificial intelligence enhance car manufacturing? A Delphi study-based identification and assessment of general use cases
2021, International Journal of Information Management
The latest boom of artificial intelligence (AI) has left the information management community in strong need of structure-providing, high-level overview works. Such works are supposed to allow both researchers and practitioners to keep track of that steep development across the technology's numerous possible application domains. So it is among other things that AI is said to incorporate enormous potential for reducing the operational costs of car manufacturers all over the globe. Nevertheless, many of them are still struggling with adopting it at large scale just because of a lack of knowledge on if and where to apply it. This study is therefore designed to find out which general use cases exist for AI within the context of car manufacturing and which ones might be the most promising ones to pursue at this early stage. We conducted a Delphi study with 39 experts in 25 different globally scattered organizations over one and a half years. As a result, we were able to identify 20 different high-level use cases for AI along the entire car manufacturing process. Our panelists have completely ranked and assessed those 20 use cases within two different dimensions, i.e., their estimated business value and their realizability. Besides being the first study to provide such an overview at one glance and to give such quantitative insights on that steeply emerging topic, four use cases from that list have never been discussed in connection with car manufacturing within the scientific literature until now and can therefore be considered as completely new in that regard.
A new hybrid quadratic regression and cascade forward backpropagation neural network
2016, Neurocomputing
Citation Excerpt :
The entirety of these investigations advocated that hybrid methods had improved forecasting performance equated to individual forecasting methods. Wolport [8], Smith et al., [9] as well as Zhang and Beradi [10] were some of the earliest authors to propose hybrid methods by merging different neural network architectures and showed that hybrid methods involving different neural network architecture produced better forecasting performance compared to individual neural network methods. In our work, new trends in hybrid methods involving the combination of classical statistical methods such as quadratic regression methods with different neural network architectures such as the cascade forward back propagation neural network are proposed.
In this study, a quadratic regression model (QRM) and a cascade forward backpropagation neural network (CFBN) are jointly integrated together to form a hybrid model called the new hybrid quadratic regression method and cascade forward backpropagation neural (QRM-CFBN) network method. The new hybrid method was tested on a daily time series data obtained from the UCI repository data link and the data set was collected from a combined cycle power plant. The joint integration was made possible by the Bayesian model averaging technique, which was used to obtain a combined forecast from the two separate methods. The model resulting from the joint integration was applied on the log difference series of the original time series data. The results obtained from the new hybrid QRM-CFBN were compared with the results obtained from the hybrid ARIMA-RNN, standalone cascade forward backpropagation neural (CFBN) network and layered recurrent neural network (LRNN) after being tested on the same sample time series data respectively. The comparison indicates that the results emerging from the new hybrid QRM-CFBN method on the average, generally results in better performance when compared with the hybrid ARIMA-RNN, the standalone CFBN and the standalone LRNN for 1 day, 3 days as well as 5 days prediction mean absolute error (MAE) and root mean square error (RMSE) for varying data samples of 50, 100, 200, 400 and 800 days respectively. The RMSEs and the MAEs were applied to ascertain the assertion that the new jointly integrated forecast has better forecasting performance greater than the standalone CFBN and LRNN forecasts as well as ARIMA-RNN forecast. The analysis for this study was simulated using MATLAB software, version 8.03.
A neural model for the p-median problem
2008, Computers and Operations Research
There exist several neural techniques for solving NP-hard combinatorial optimization problems. At the beginning of the 1980s, recurrent neural networks were shown to be able to solve optimization problems. Criticism of this approach includes the tendency of recurrent neural networks to produce infeasible solutions and poor local minima. This paper proposes a new technique which always provides feasible solutions and removes the tuning phase since the constraints are incorporated in the neural architecture instead of the energy function, therefore the tuning parameters are unnecessary.
One of the most popular and well-known facility location problems is the p-median problem, which concerns the location of p facilities (or medians) in order to minimize the total weighted distance between the demand points and the facilities. There exist several heuristics to find optimal solutions of the problem based on the traditional formulation.
In this paper a new formulation for the p-median problem based on two types of decision variables and with only $n + p$ linear equality constraints is presented, where n is the number of demand points or customers and p is the number of facilities (medians). Also, a competitive recurrent neural network is proposed for this problem. The neural network consists of two layers (allocation layer and location layer) with $2 np$ process units. The process units constitute $n + p$ groups, where only one process unit per group is active at the same time and process units of the same layer are updated in parallel. Moreover, the energy function (objective function) always decreases or remains constant as the system evolves according to the dynamical rule proposed. The effectiveness and efficiency of our algorithm for different problem sizes are analyzed in comparison to conventional heuristic methods. The results show that our recurrent neural network generates good solutions with a reasonable computational effort.
Geographical information systems (GIS) have occupied the attention of many researches involving a number of academic fields including geography, civil engineering, computer science, land use planning, and environmental sciences. GIS can support a wide range of spatial queries that can be used to support location studies. Model application and model development are the major impact of GIS on the field of location science. These systems are designed to store, retrieve, manipulate, analyze, and map geographical data. GIS can serve as the source of input data for a location model and it can also be used to present the model results. For example, if a p-median problem has to be solved, then a GIS executes a heuristic algorithm that reads the data from the GIS and it presents the results in real time.
Location–allocation models simultaneously locate facilities and allocate demand points to them. These models also arise in a variety of public and private sector problems. The p-median is the most widely used location–allocation model. The p-median model is NP-hard and its data set became very large in real problem, so heuristic solutions are required. As the size of the data set grow, the number of feasible solutions grows and the quality of solutions and computation times from the most commonly used heuristic are deteriorated. The purpose of this paper is to develop a neural model to be integrated in GIS software. Thus, we propose a new formulation for the p-median problem based on $2 np$ binary variables and $n + p$ equality linear constraints. A recurrent neural model is proposed for solving the p-median problem based on this formulation without the difficulty in selecting appropriate tuning parameters, since these parameters are avoided. Moreover, an np-parallel algorithm has been developed based on this formulation. The central property of this algorithm is that the objective function always decreases (or remains constant) as the algorithm evolves according to its dynamical rule. Moreover, this algorithm can be also implemented using optical or VLSI technology.
Heuristics and augmented neural networks for task scheduling with non-identical machines
2006, European Journal of Operational Research
Citation Excerpt :
This enhancement significantly reduced the inherent complexity of the Hopfield network. Smith et al. (1996) propose a hybrid of Hopfield network and self-organizing networks to overcome the deficiencies of each of these types of network. Smith et al. (1998) explain the Hopfield network, an improved Hopfield network and the self-organizing network for a general optimization problem.
We propose new heuristics along with an augmented-neural-network (AugNN) formulation for solving the makespan minimization task-scheduling problem for the non-identical machine environment. We explore four task and three machine-priority rules, resulting in 12 combinations of single-pass heuristics. The task-priority rules are Highest-Level-First (HLF), Highest-Total-Remaining-Processing-Time-First (HTRPTF), Smallest-Latest-Finish-Time-First (SLFTF) and Minimum-Slack-First (MSF). For machine priority, we propose a greedy rule called Fastest-Available-Machine-First (FAMF), and two non-greedy rules: (1) Fastest-Available-Machine-First-With-Conditional-Wait-1 (FAMF-CW-1), and (2) Fastest-Available-Machine-First-With-Conditional-Wait-2 (FMF-CW-2). The AugNN approach integrates neural-network learning with domain and problem specific knowledge through heuristics, to produce good results. A single-pass heuristic is embedded in a neural network designed specifically for each problem. We give the AugNN formulation for each of the 12 heuristics and show computational results on 100 randomly generated problems of sizes ranging from 20 to 70 tasks and 2 to 5 machines. Results demonstrate that AugNN provides significant improvement over single-pass heuristics. The reduction in the gap between the obtained solution and the lower-bound due to AugNN over single-pass heuristics ranged from 24.4% to 50%. As far as heuristics, the non-greedy machine-priority rules performed significantly better than the greedy rule. The average gaps for the non-greedy rules ranged from 16.1% to 23.5% compared to 33.7% to 40.4% for greedy. For AugNN, for non-greedy rules, the gap ranged from 11.5% to 15.5% compared to 18.4% to 25.0% for greedy. The HLF and HTRPTF task priority rules performed better than the other two. The HTRPTF/FAMF-CW-1 combination gave the best results, closely followed by HLF/FAMF-CW-2 and HTRPTF/FAMF-CW-2 combinations.
Solving terminal assignment problems with groups encoding: The wedding banquet problem
2006, Engineering Applications of Artificial Intelligence
Citation Excerpt :
An important class of combinatorial optimization problems with constraints (Smith et al., 1996; Smith, 1999) can be defined as the maximization or minimization of a goal function (usually in a binary search space) subject to a set of constraints, which have to be satisfied for a solution to be feasible.
This paper presents an extension of the terminal assignment problem (TA) in the case that groups of terminals must be assigned together. We analyze this situation by means of an equivalent problem: the wedding banquet problem (WBP). We provide a description of the problem and its mathematical definition. We also describe an application of the WBP to mobile communications network design. Two hybrid metaheuristics algorithms for the WBP are presented in the paper. We test their performance in several computational experiments, including synthetic instances of the WBP, and a mobile network design application.
A portable and scalable algorithm for a class of constrained combinatorial optimization problems
2005, Computers and Operations Research
Citation Excerpt :
In this paper, we refer to this kind of problems as constrained combinatorial optimization problems (CCOPs). Several methods which have been reported as good approaches to concrete examples of CCOPs can be found in the literature [10–14,30,31]. However, these existing algorithms have two main drawbacks: first, the majority of them show lack of scalability.
This paper presents a portable and scalable approach for a class of constrained combinatorial optimization problems (CCOPs) which requires to satisfy a set of constraints and to optimize and objective function simultaneously. In particular, this paper is focused on the class of CCOPs that admits a representation in terms of a square matrix of constraints C.
The algorithm consists of a hybrid neural-genetic algorithm, formed by a Hopfield Neural Network (HNN) which solves the problem's constraints, and a Genetic Algorithm (GA) for optimizing the objective function. This separated management of constraints and optimization procedures makes the proposed algorithm scalable and robust. The portability of the algorithm is given by the fact that the HNN dynamics depends only on the matrix C of constraints.
We show these properties of scalability and portability by solving three different CCOPs with our algorithm, the frequency assignment problem in a mobile telecommunications network, the reduction of the interference in satellite systems and the design of FPGAs with segmented channel routing architecture. We compare our results with previous approaches to these problems, obtaining very good results in all of them.

View all citing articles on Scopus

^†: Kate Smith received a Bachelor of Science degree in Mathematics from the University of Melbourne, Australia where she has recently completed Ph.D. studies in the area of neural networks for combinatorial optimization. She is currently a Lecturer in the Department of Business Systems at Monash University, Australia.

^§: M. Krishnamoorthy is a research scientist with the Operations Research Group at the CSIRO Division of Mathematics and Statistics in Melbourne, Australia. He obtained his M.Sc. and Ph.D. from Imperial College, University of London. He also has an M.Sc. (1985) in Operational Research from the University of Delhi. His research interests are in developing efficient algorithms for combinatorial, network and graph optimization problems.

View full text

A hybrid neural approach to combinatorial optimization

Abstract

Neural Networks

“Neural” computation of decisions in optimization problems

Biological Cybernet.

Simple neural optimization networks: an A/D converter, signal decision circuit and a linear programming circuit

IEEE Trans. Circuit Syst.

Solving Constraint Satisfaction Problems with neural networks

Generalised Hopfield Networks and nonlinear optimization

On the stability of the TSP algorithm of Hopfield and Tank

Biological Cybernet.

Determination of parameters in a Hopfield/Tank computational network

Dynamical stability and parameter selection in neural optimization

Genetic breeding of control parameters for the Hopfield.Tank neural net

Alternative networks for solving the Travelling Salesman Problem and the List-Matching Problem

A Travelling Salesman objective function that works

Limitations of neural networks for solving Traveling Salesman Problems

IEEE Trans. Neural Networks

Self-organized formation of topologically correct feature maps

Biological Cybernet.

Design of competition-based neural networks for combinatorial optimization

Int. J. Neural Syst.