Abstract
The fine grain, data-driven parallelism shown by neural models as the Boltzmann machine cannot be implemented in an entirely efficient way either in general-purpose multicomputers or in networks of computers, which are nowadays the most common parallel computer architectures.
In this paper we present a parallel implementation of a modified Boltzmann machine where the processors, with disjoint subsets of neurons allocated, asynchronously compute the evolution of their neurons by using values that might not be updated for the remaining neurons, thus reducing interprocessor communication requirements. An evolutionary algorithm is used to learn the rules that allow the processors to cooperate by interchanging the local optima that they find while concurrently exploring different zones of the Boltzmann machine state space. Thus, the way the processors interact changes dynamically during execution of the algorithm, adapted to the problem at hand. Good figures for speedup with respect to the Boltzmann machine computation in a uniprocessor computer have been experimentally obtained.
Preview
Unable to display preview. Download preview PDF.
References
Anderson, T.E.; Culler, T.E.; Patterson, D.A.; and the NOW team: “A Case for NOW (Networks of Workstations”. IEEE Micro, pp. 54–64. February, 1995.
Wolpert, D.H.; Macready, W.G.: “No Free Lunch Theorems for Optimization”. IEEE Trans. on Evolutionary Computation, Vol. 1, No. 1, pp. 67–82. April, 1997.
Aarts, E.H.L.; Korst, J.H.M.: “Simulated Annealing and Boltzmann Machines”. Wiley, 1988.
Oh, D.H.; Nang, J.H.; Yoon, H.; Maeng, S.R.: “An efficient mapping of Boltzmann Machine computations onto distributed-memory multiprocessors”. Microprocessing and Microprogramming, Vol. 33, pp. 223–236, 1991/92.
De Gloria, A.; Faraboschi, P.; Ridella, S.: “A dedicated Massively Parallel Architecture for the Boltzmann Machine”. Parallel Comp., Vol. 18, No. 1, pp. 57–75, 1993.
Ortega, J.; Rojas, I.; Diaz, A.F.; Prieto, A.: “Parallel Coarse Grain Computing of Boltzmann Machines”. Neural Processing Letters, Vol. 7, No. 3, pp. 1–16, 1998.
Hong, C.-E.; McMillin, B.M.: “Relaxing Synchronization in Distributed Simulated Annealing”. IEEE Trans. on Parallel and Distributed Systems, Vol. 6, No. 2, pp. 189–195. Febuary, 1995.
Pramanick, I.; Kuhl, J.G.: “An inherently Parallel Method for Heuristic Problem-Solving: Part I—General Framework”. IEEE Trans. on Parallel and Distributed Systems, Vol. 6, No. 10, pp. 1006–1015. October, 1995.
Zissimopoulos, V.; Paschos, V.T.; Pekergin, F.: “On the approximation of NP-complete problems by using the Boltzmann Machine method: The cases of some covering and packing problems”. IEEE Trans. on Computers, Vol. 40, No. 12, pp. 1413–1418. December, 1991.
Potter, M.A.; De Jong, K.A.: “A Cooperative Coevolutionary Aproach to Function Optimization”. In Third Conference on Parallel Problem Solving from Nature, Y. Davidor and H.P. Schwefel (Eds.), Lecture Notes in Computer Science, Vol. 866, Springer-Verlag, pp. 249–257, 1994.
Martin, O.; Otto, S.W.; Felten, E.W.: “Large-step Markov chains for the TSP incorporation local search heuristics”. Operations Research Letters, 11, pp. 219–224. May 1992.
Lourenço, H.R.: “Job-shop scheduling: Computational study of local search and large-step optimization methods”. European J. of Operation Research, 83, pp. 347–364, 1995.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1999 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ortega, J., Parrilla, L., Bernier, J.L., Gil, C., Pino, B., Anguita, M. (1999). Adaptive cooperation between processors in a parallel Boltzmann machine implementation. In: Mira, J., Sánchez-Andrés, J.V. (eds) Engineering Applications of Bio-Inspired Artificial Neural Networks. IWANN 1999. Lecture Notes in Computer Science, vol 1607. Springer, Berlin, Heidelberg . https://doi.org/10.1007/BFb0100487
Download citation
DOI: https://doi.org/10.1007/BFb0100487
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-66068-2
Online ISBN: 978-3-540-48772-2
eBook Packages: Springer Book Archive