Skip to main content
SpringerLink
Log in

Tabu Machine: A New Neural Network Solution Approach for Combinatorial Optimization Problems

  • Published:
Journal of Heuristics Aims and scope Submit manuscript

Abstract

A new artificial neural network solution approach is proposed to solve combinatorial optimization problems. The artificial neural network is called the Tabu Machine because it has the same structure as the Boltzmann Machine does but uses tabu search to govern its state transition mechanism. Similar to the Boltzmann Machine, the Tabu Machine consists of a set of binary state nodes connected with bidirectional arcs. Ruled by the transition mechanism, the nodes adjust their states in order to search for a global minimum energy state. Two combinatorial optimization problems, the maximum cut problem and the independent set problem, are used as examples to conduct a computational experiment. Without using overly sophisticated tabu search techniques, the Tabu Machine outperforms the Boltzmann Machine in terms of both solution quality and computation time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Aarts, E.H.L. and J.H. Korst. (1989a). Simulated Annealing and Boltzmann Machines: A Stochastic Approach to Combinatorial Optimization and Neural Computing. New York: Wiley.

    Google Scholar 

  • Aarts, E.H.L. and J.H. Korst. (1989b). “Boltzmann Machines for Travelling Salesman Problems.” European Journal of Operational Research 39(1), 79–95.

    Google Scholar 

  • Aarts, E.H.L., J.H. Korst, and P.J. Zwietering. (1996). “Deterministic and Randomized Local Search.” In P. Smolensky, M. Mozer, and D.E. Rumelhart (eds.), Mathematical Perspectives on Neural Networks. Mahwah, NJ: Lawerence Erlbaum, pp. 143–224.

    Google Scholar 

  • Aiyer, S.V. and F. Fallside. (1990). “A Theoretical Investigation into the Performance of the Hopfield Model.” IEEE Transactions on Neural Networks 1(2), 204–215.

    Google Scholar 

  • Battiti, R. and G. Tecchiolli. (1995). “Training Neural Nets with the Reactive Tabu Search.” IEEE Transactions on Neural Networks 6(5), 1185–1200.

    Google Scholar 

  • Beyer, D. and R. Ogier. (1991). “Tabu Learning: A Neural Network Search Method for Solving Nonconvex Optimization Problems.” In Proceedings of the International Joint Conference on Neural Networks, IEEE and INNS, Singapore, pp. 1–9.

  • Blum, A. and R. Rivest. (1992). “Training a 3-Node Neural Network is NP-Complete.” Neural Networks 5(1), 117–128.

    Google Scholar 

  • Brandt, R., Y. Wang, A. Laub, and S. Mitra. (1992). “Alternative Networks for Solving the Traveling Salesman Problem and the List-Matching Problem.” In Proceedings of International Conference on Neural Networks 2, pp. 330–340.

    Google Scholar 

  • Burke, L.I. (1992). “A Neural Design for Solution of the Maximal Independent Set Problem.” European Journal of Operational Research 62(2), 186–193.

    Google Scholar 

  • Campadelli, P., P. Mora, and R. Schettini. (1995). “Color Set Selection of Nominal Coding by Hopfield Networks.” The Visual Computer 11, 150–155.

    Google Scholar 

  • Chakrapani, J. and J. Skorin-Kapov. (1992). “A Connectionist Approach to the Quadratic Assignment Problem.” Computers & Operations Research 19(3/4), 287–295.

    Google Scholar 

  • Chakrapani, J. and J. Skorin-Kapov. (1993a). “Massively Parallel Tabu Search for the Quadratic Assignment Problem.” Annals of Operations Research 41, 327–341.

    Google Scholar 

  • Chakrapani, J. and J. Skorin-Kapov. (1993b). “Connection Machine Implementation of a Tabu Search Algorithm for the Traveling Salesman Problem.” Journal of Computing and Information Technology 1(1), 29–36.

    Google Scholar 

  • Cuykendail, R. and R. Reese. (1989). “Scaling the Neural TSP Algorithm.” Biological Cybernetics 60, 365–371.

    Google Scholar 

  • de Werra, D. and A. Hertz. (1989). “Tabu Search Techniques: A Tutorial and Applications to Neural Networks.” OR Spectrum 11, 131–141.

    Google Scholar 

  • Fang, L. and T. Li. (1990). “Design of Competition-Based Neural Networks for Combinatorial Optimization.” International Journal of Neural Systems 1(3), 221–235.

    Google Scholar 

  • Friden, C., A. Hertz, and D. de Werra. (1989). “STABULUS: A Technique for Finding Stable Sets in Large Graphs with Tabu Search.” Computing 45(2), 35–44.

    Google Scholar 

  • Friden, C., A. Hertz, and D. de Werra. (1990). “TABARIS: An Exact Algorithm Based on Tabu Search for Finding a Maximum Independent Set in a Graph.” Computers & Operations Research 17(5), 437–445.

    Google Scholar 

  • Garey, M.R. and D.S. Johnson. (1979). Computers and Intractability: A Guide to the Theory of NP -Completeness. San Francisco: W.H. Freeman.

    Google Scholar 

  • Glover, F. (1989). “Tabu Search, Part I.” ORSA Journal on Computing 1(3), 190–206.

    Google Scholar 

  • Glover, F. (1990a). “Tabu Search, Part II.” ORSA Journal on Computing 2(1), 4–32.

    Google Scholar 

  • Glover, F. (1990b).“Tabu Search: A Tutorial.” Interfaces 20(4), 74–94.

    Google Scholar 

  • Glover, F., G.A. Kochenberger, and B. Alidaee. (1998). “Adaptive Memory Tabu Search for Binary Quadratic Programs.” Management Science 44(3), 336–345.

    Google Scholar 

  • Glover, F. and M. Laguna. (1997). Tabu Search. Hingham, MA: Kluwer Academic Publishers.

    Google Scholar 

  • Haykin, S. (1994). Neural Networks: A Comprehensive Foundation. New York, NY: Macmillan.

    Google Scholar 

  • Hinton, G.E. and T.J. Sejnowski. (1983). “Optimal Perceptual Inference.” In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Washington D. C., pp. 448–453.

  • Hopfield, J.J. (1982). “Neural Networks and Physical Systems with Emergent Collective Computational Abilities.” Proceedings of National Academy of Sciences of the United States of America 79, 2554–2558.

    Google Scholar 

  • Hopfield, J.J. (1984). “Neurons with Graded Response Have Collective Computational Properties Like Those of Two-State Neurons.” Proceedings of National Academy of Sciences of the United States of America 81, 3088–3092.

    Google Scholar 

  • Hopfield, J.J. and D.W. Tank. (1985). “Neural Computation of Decisions in Optimization Problems.” Biological Cybernetics 52, 141–152.

    Google Scholar 

  • Jagota, A. (1995). “Approximating Maximum Clique with a Hopfield Network.” IEEE Transactions on Neural Networks 6(3), 724–735.

    Google Scholar 

  • Kamgar-Parsi, B. and B. Kamgar-Parsi. (1990).“On Problem Solving with Hopfield Neural Networks.” Biological Cybernetics 62, 415–423.

    Google Scholar 

  • Kaznachey, D. and A. Jagota. (1997). “Approximating Minimum Set Cover in a Hopfield-Style Network.” Information Sciences 98, 203–216.

    Google Scholar 

  • Kirkpatrick, S. (1984). “Optimization by Simulated Annealing: Quantitative Studies.” Journal of Statistical Physics 34, 975–986.

    Google Scholar 

  • Kohonen, T. (1988). Self-Organization and Associative Memory, 2nd edn. Berlin: Springer-Verlag.

    Google Scholar 

  • Looi, C.K. (1992). “Neural Network Methods in Combinatorial Optimization.” Computers & Operations Research 19(3/4), 191–208.

    Google Scholar 

  • Malek, M., M. Guruswamy, M. Pandya, and H. Owens. (1989). “Serial and Parallel Simulated Annealing and Tabu Search Algorithms for the Traveling Salesman Problem.” Annals of Operations Research 21, 59–84.

    Google Scholar 

  • Másson, E. and Y.J. Wang. (1990). “Introduction to Computation and Learning in Artificial Neural Networks.” European Journal of Operational Research 47(1), 1–28.

    Google Scholar 

  • Peterson, C. and B. Soderberg. (1989). “A New Method for Mapping Optimization Problems onto Neural Networks.” International Journal of Neural Systems 1(1), 3–22.

    Google Scholar 

  • Rumelhart, D.E., G.E. Hinton, and R.J. Williams. (1986). “Learning Internal Representations by Error Propagation.” In D.E. Rumelhart, J.L. McClelland and the PDP Research Group (eds.), Parallel Distributed Processing, Vol. 1: Foundations. Cambridge, MA: MIT Press. pp. 318–362.

    Google Scholar 

  • Samaad, T. and P. Harper. (1990). “High Order Hopfield and Tank Optimization Networks.” Parallel Computing 16, 287–292.

    Google Scholar 

  • Sexton, R., B. Alidaee, R. Dorsey, and J. Johnson. (1998). “Global Optimization for Artificial Neural Networks: A Tabu Search Application.” European Journal Of Operational Research 106(2/3), 570–584.

    Google Scholar 

  • Smith, K., M. Palaniswami, and M. Krishnamoorthy. (1996). “A Hybrid Neural Approach to Combinatorial Optimization.” Computers & Operations Research 23(6), 597–610.

    Google Scholar 

  • Sun, M., J.E. Aronson, P.G. McKeown, and D. Drinka. (1998). “A Tabu Search Heuristic Procedure for the Fixed Charge Transportation Problem.” European Journal of Operational Research 106(2/3), 441–456.

    Google Scholar 

  • Sun, K.T. and H.C. Fu. (1993). “A Hybrid Neural Network for Solving Optimization Problems.” IEEE Transactions on Computers 42(2), 218–227.

    Google Scholar 

  • Sun, M. and P.G. McKeown. (1993). “Tabu Search Applied to the General Fixed Charge Problem.” Annals of Operations Research 41, 405–420.

    Google Scholar 

  • Sun, M. and H.R. Nemati. (1994). “TABU MACHINE: A New Method of Solving Combinatorial Optimization Problems.” Presented at the TIMS/ORSA Joint National Meeting, Boston, MA.

  • Vaithyanathan, S., L. Burke, and M. Magent. (1996). “Massively Parallel Analog Tabu Search Using Neural Networks Applied to Simple Plant Location Problems.” European Journal of Operational Research 93(2), 317–330.

    Google Scholar 

  • Van den Bout, D.E. and T.K. Miller. (1989). “Improving the Performance of the Hopfield-Tank Neural Network Through Normalization and Annealing.” Biological Cybernetics 62, 129–139.

    Google Scholar 

  • Wang, J. (1994). “Neural Network Models and Their Role for Optimization: An Overview.” ORSA CSTS Newsletter 15(1).

  • Wilson, G. and G. Pawley. (1988). “On the Stability of the Travelling Salesman Problem Algorithm of Hopfield and Tank.” Biological Cybernetics 58, 63–70.

    Google Scholar 

  • Xu, X. and W.T. Tsai. (1991). “Effective Neural Algorithms for the Traveling Salesman Problem.” Neural Networks 4(2), 193–205.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Management Science and Statistics, College of Business, The University of Texas at San Antonio, San Antonio, TX, 78249–0632, USA

    Minghe Sun

  2. Department of Information Systems and Operations Management, Bryan School of Business and Economics, The University of North Carolina, Greensboro, NC, 27402–6165, USA

    Hamid R. Nemati

Authors
  1. Minghe Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hamid R. Nemati
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Sun, M., Nemati, H.R. Tabu Machine: A New Neural Network Solution Approach for Combinatorial Optimization Problems. Journal of Heuristics 9, 5–27 (2003). https://doi.org/10.1023/A:1021849410328

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1021849410328

Search

Navigation