Skip to main content
Log in

Parallel Coarse Grain Computing of Boltzmann Machines

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

The resolution of combinatorial optimization problems can greatly benefit from the parallel and distributed processing which is characteristic of neural network paradigms. Nevertheless, the fine grain parallelism of the usual neural models cannot be implemented in an entirely efficient way either in general-purpose multicomputers or in networks of computers, which are nowadays the most common parallel computer architectures. Therefore, we present a parallel implementation of a modified Boltzmann machine where the neurons are distributed among the processors of the multicomputer, which asynchronously compute the evolution of their subset of neurons using values for the other neurons that might not be updated, thus reducing the communication requirements. Several alternatives to allow the processors to work cooperatively are analyzed and their performance detailed. Among the proposed schemes, we have identified one that allows the corresponding Boltzmann Machine to converge to solutions with high quality and which provides a high acceleration over the execution of the Boltzmann machine in uniprocessor computers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. E.H.L. Aarts and J.H.M. Korst, Simulated Annealing and Boltzmann Machines, New York: Wiley, 1988.

    Google Scholar 

  2. D.H. Oh, J.H. Nang, H. Yoon and S.R. Maeng, “An efficient mapping of Boltzmann Machine computations onto distributed-memory multiprocessors”, Microprocessing and Microprogramming, 33, pp. 223-236, 1991/92.

    Google Scholar 

  3. A. De Gloria, P. Faraboschi and S. Ridella, “A dedicated Massively Parallel Architecture for the Boltzmann Machine”, Parallel Comp., Vol. 18, No. 1, pp. 57-75, 1993.

    Google Scholar 

  4. A. De Gloria and M. Olivieri, “An asynchronous distributed architecture model for the Boltzmann machine control mechanism”, IEEE Trans. on Neural Networks, Vol. 7, No. 6., pp. 1538-1541. November, 1996.

    Google Scholar 

  5. O. Martin, S.W. Otto and E.W. Felten, “Large-step Markov chains for the TSP incorporation local search heuristics”, Operations Research Letters, 11, pp. 219-224. May, 1992.

    Google Scholar 

  6. H.R. Lourenço, “Job-shop scheduling: Computational study of local search and large-step optimization methods”, European J. of Operation Research, 83, pp. 347-364, 1995.

    Google Scholar 

  7. M. Livesey, “Clamping in Boltzmann machines”, IEEE Trans. on Neural Networks, Vol. 2, No. 1, pp. 143-148. January, 1991.

    Google Scholar 

  8. C.-E. Hong and B.M. McMillin, “Relaxing Synchronization in Distributed Simulated Annealing”, IEEE Trans. on Parallel and Distributed Systems, Vol. 6, No. 2, pp. 189-195, February, 1995.

    Google Scholar 

  9. V. Zissimopoulos, V.T. Paschos and F. Pekergin, “On the approximation of NP-complete problems by using the Boltzmann Machine method: The cases of some covering and packing problems”, IEEE Trans. on Computers, Vol. 40, No. 12, pp. 1413-1418, December, 1991.

    Google Scholar 

  10. I. Pramanick and J.G. Kuhl, “An inherently Parallel Method for Heuristic Problem-Solving: Part I - General Framework”, IEEE Trans. on Parallel and Distributed Systems, Vol. 6, No. 10, pp. 1006-1015, October, 1995.

    Google Scholar 

  11. S.-Y. Lee and K.G. Lee, “Synchronous and asynchronous parallel simulated annealing with multiple Markov chains”, IEEE Trans. on Parallel and Distributed Systems, Vol. 7, No. 10, pp. 993-1007, October, 1996.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ortega, J., Rojas, I., Diaz, A.F. et al. Parallel Coarse Grain Computing of Boltzmann Machines. Neural Processing Letters 7, 169–184 (1998). https://doi.org/10.1023/A:1009689415274

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1009689415274

Navigation