Representation issues in Boltzmann machines

Abstract

We have postulated three major causes for the disappointing performance of the Boltzmann machine on a TSP in comparison with sequential simulated annealing. For each of these, we proposed, analyzed, and tested representational variations that could be expected to lead to some improvement. Whereas the experiments indeed indicate some improvement, the outcome is not overwhelming. Changes to the representation seem to sort much less effect on the Boltzmann machine than they do on the Hopfield-Tank network. Nevertheless, a conclusion that the Hopfield-Tank network is better suited to deal with combinatorial optimization problems than the Boltzmann machine would be premature. Firstly, Boltzmann machine simulations did perform well on other combinatorial optimization problems than the TSP. Secondly, there is, as yet, considerable uncertainty with respect to the speed and construction cost that can be expected of different kinds of neural network hardware in the future. At present, it is expected (Aarts and Korst, 1989b) that Boltzmann machines will be easier to implement in hardware than Hopfield-Tank networks.

We hope to have shown that massive parallelism in neural networks, promising as it may be, is not guaranteed to solve computational problems, that representation does matter in neural networks, and that finding a better representation is difficult. In this respect, we remark that the TSP should be qualified as a relatively easy problem in comparison with, for instance, job-shop scheduling problems. Attempts to solve nontrivial scheduling problems on the Boltzmann machine would stumble upon the same representation problem we encountered with the TSP, only (much) more vehemently (cf., Sadeh, 1991; p. 8).

Research was supported by the Netherlands Foundation for Scientific Research NWO under grant number 612-322-014