Abstract
In this paper we describe a parallel implementation of a multi-layer perceptron for a message-passing parallel architecture following the vertical-slicing approach. A theoretical analysis shows that linear scalability may be achieved both in recognition and learning, at the expense of a proper replication of data structures in order to optimize the communication phase.
Scalability is a function of the number of neurons per processor, of the communication bandwidth and of the ratio between processing time and communication time. We show how, given a particular neural network, the number of processing elements that minimizes the execution time can be determined.
The theoretical analysis has been confirmed by an actual implementation in the case of a Transputer-based system with 40 processing nodes.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Mc Clelland, J., Rumelhart, D.E. et al.: “Parallel Distributed Processing”, vol. 1, Cambridge MA, MIT Press, (1986).
Fogelman-Soulie, F., Gallinari, P., Le Cun, Y. & S. Thiria: “Network Learning”, to appear in “Machine Learning”, vol. 3, Kodratoff, R. Michalski Eds.
Beynon, T. & N. Dodd: “The implementation of multi-layer perceptrons on transputer networks”, Proc. of the 7th Occam User Group, Grenoble, (1987).
Robert, F. & S. Wang: “Implementation of a Neural Network on a Hypercube F.P.S. T20”, IFIP Working Group 10.3, Conference on Parallel Processing, Pisa, Italy (April 1988).
Forrest, B., Roweth, D., Stroud, N., Wallace, D., & G. Wilson: “Neural network models”, Edinburgh preprint no. 87/419, (1987).
Baiardi, F., Mussardo, R., Serra, R. & G. Valastro: “Feedforward Layered Networks on Message Passing Parallel Computers”, to appear in “Proceedings of the 1989 Workshop on Parallel Architectures and Neural Networks”, E.R. Caianiello editor, World Scientific Publishers, London, in press
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1990 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Baiardi, F., Mussardo, R., Serra, R., Valastro, G. (1990). Parallel Implementation of a Multi-Layer Perceptron. In: Soulié, F.F., Hérault, J. (eds) Neurocomputing. NATO ASI Series, vol 68. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-76153-9_21
Download citation
DOI: https://doi.org/10.1007/978-3-642-76153-9_21
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-76155-3
Online ISBN: 978-3-642-76153-9
eBook Packages: Springer Book Archive