Abstract
The resolution of combinatorial optimization problems can greatly benefit from the parallel and distributed processing which is characteristic of neural network paradigms. Nevertheless, the fine grain parallelism of the usual neural models cannot be implemented in an entirely efficient way either in general-purpose multicomputers or in networks of computers, which are nowadays the most common parallel computer architectures. Therefore, we present a parallel implementation of a modified Boltzmann machine where the neurons are distributed among the processors of the multicomputer, which asynchronously compute the evolution of their subset of neurons using values for the other neurons that might not be updated, thus reducing the communication requirements. Several alternatives to allow the processors to work cooperatively are analyzed and their performance detailed. Among the proposed schemes, we have identified one that allows the corresponding Boltzmann Machine to converge to solutions with high quality and which provides a high acceleration over the execution of the Boltzmann machine in uniprocessor computers.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
E.H.L. Aarts and J.H.M. Korst, Simulated Annealing and Boltzmann Machines, New York: Wiley, 1988.
D.H. Oh, J.H. Nang, H. Yoon and S.R. Maeng, “An efficient mapping of Boltzmann Machine computations onto distributed-memory multiprocessors”, Microprocessing and Microprogramming, 33, pp. 223-236, 1991/92.
A. De Gloria, P. Faraboschi and S. Ridella, “A dedicated Massively Parallel Architecture for the Boltzmann Machine”, Parallel Comp., Vol. 18, No. 1, pp. 57-75, 1993.
A. De Gloria and M. Olivieri, “An asynchronous distributed architecture model for the Boltzmann machine control mechanism”, IEEE Trans. on Neural Networks, Vol. 7, No. 6., pp. 1538-1541. November, 1996.
O. Martin, S.W. Otto and E.W. Felten, “Large-step Markov chains for the TSP incorporation local search heuristics”, Operations Research Letters, 11, pp. 219-224. May, 1992.
H.R. Lourenço, “Job-shop scheduling: Computational study of local search and large-step optimization methods”, European J. of Operation Research, 83, pp. 347-364, 1995.
M. Livesey, “Clamping in Boltzmann machines”, IEEE Trans. on Neural Networks, Vol. 2, No. 1, pp. 143-148. January, 1991.
C.-E. Hong and B.M. McMillin, “Relaxing Synchronization in Distributed Simulated Annealing”, IEEE Trans. on Parallel and Distributed Systems, Vol. 6, No. 2, pp. 189-195, February, 1995.
V. Zissimopoulos, V.T. Paschos and F. Pekergin, “On the approximation of NP-complete problems by using the Boltzmann Machine method: The cases of some covering and packing problems”, IEEE Trans. on Computers, Vol. 40, No. 12, pp. 1413-1418, December, 1991.
I. Pramanick and J.G. Kuhl, “An inherently Parallel Method for Heuristic Problem-Solving: Part I - General Framework”, IEEE Trans. on Parallel and Distributed Systems, Vol. 6, No. 10, pp. 1006-1015, October, 1995.
S.-Y. Lee and K.G. Lee, “Synchronous and asynchronous parallel simulated annealing with multiple Markov chains”, IEEE Trans. on Parallel and Distributed Systems, Vol. 7, No. 10, pp. 993-1007, October, 1996.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Ortega, J., Rojas, I., Diaz, A.F. et al. Parallel Coarse Grain Computing of Boltzmann Machines. Neural Processing Letters 7, 169–184 (1998). https://doi.org/10.1023/A:1009689415274
Issue Date:
DOI: https://doi.org/10.1023/A:1009689415274