Performance prediction of large MIMD systems for parallel neural network simulations

doi:10.1016/0167-739X(94)00064-L

Future Generation Computer Systems

Volume 11, Issue 2, March 1995, Pages 221-232

https://doi.org/10.1016/0167-739X(94)00064-L Get rights and content

Abstract

In this paper, we present a performance prediction model for indicating the performance range of MIMD parallel processor systems for neural network simulations. The model expresses the total execution time of a simulation as a function of the execution times of a small number of kernel functions, which have to be measured on only one processor and one physical communication link. The functions depend on the type of neural network, its geometry, decomposition and the connection structure of the MIMD machine. Using the model, the execution time, speedup, scalability and efficiency of large MIMD systems can be predicted. The model is validated quantitatively by applying it to two popular neural networks, backpropagation and the Kohonen self-organizing feature map, decomposed on a GCel-512, a 512 transputer system. Measurements are taken from network simulations decomposed via dataset and network decomposition techniques. Agreement of the model with the measurements is within 1–14%. Estimates are given for the performances that can be expected for the new T9000 transputer systems. The presented method can also be used for other application areas such as image processing.

References (17)

M. Berry et al.
Scientific benchmark characterizations
Parallel Comput.
(1991)
G.A. Carpenter et al.
A massively parallel architecture for a self-organizing neural pattern recognition machine
Computer Vision, Graphics, and Image Processing
(1987)
L-C. Chu et al.
Optimal mapping of neural network learning on message-passing multicomputers
J. Parallel Distributed Comput.
(1992)
A. d'Acierno et al.
Fully connected neural networks: Simulation on massively parallel computers
A.J.G. Hey
The Genesis distributed memory benchmarks
Parallel Comput.
(1991)
T. Tollenaere et al.
Simulating modular neural networks on message-passing multiprocessors
Parallel Comput.
(1991)
M. Witbrock et al.
An implementation of back-propagation learning on GF11, a large SIMD parallel computer
Parallel Comput.
(1990)
C-H. Wu et al.
Parallelizing the self-organizing feature map on multiprocessor systems
Parallel Comput.
(1991)

There are more references available in the full text version of this article.

Cited by (5)

Parallel codebook design for vector quantization on a message passing MIMD architecture
2002, Parallel Computing
Citation Excerpt :
It is not reasonable to compare those results with the ones reported here for many reasons: the platform is different, the parallelization method is not the same, and the speed-up is calculated differently. There are some MIMD parallel implementations of neural networks which have been trained using the competitive learning algorithms such as the Kohonen maps on the GCEL-512 which is a 512-transputer system [20]. In this paper, a master/worker parallel implementation of a VQ algorithm to train a codebook on gray image database has been evaluated using the Alex AVX-2 parallel computer.
Vector quantization (VQ) is a widely used algorithm in speech and image data compression. One of the problems of the VQ methodology is that it requires large computation time especially for large codebook size. This paper addresses two issues. The first deals with the parallel construction of the VQ codebook which can drastically reduce the training time. A master/worker parallel implementation of a VQ algorithm is proposed. The algorithm is executed on the DM-MIMD Alex AVX-2 machine using a pipeline architecture. The second issue deals with the ability of accurately predicting the machine performance. Using communication and computation models, a comparison between expected and real performance is carried out. Results show that the two models can accurately predict the performance of the machine for image data compression. Analysis of metrics normally used in parallel realization is conducted.
Performance of the Alex AVX-2 MIMD architecture in learning the NetTalk database
2004, IEEE Transactions on Neural Networks
Performance of a backpropagation trained feedforward network on an MIMD architecture
1998, Concurrency Practice and Experience
On the implementation of backpropagation on the Alex AVX-2 parallel system
1997, IEEE International Conference on Neural Networks - Conference Proceedings
Estimating the parallel start-up overhead for parallelizing compilers
1997, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

^☆: We would like to acknowledge the University of Amsterdam and Parsytec Gmbh for allowing us to use the GCel located in Amsterdam during the CAMPP '93 programme.

View full text

PaperPerformance prediction of large MIMD systems for parallel neural network simulations☆

Abstract

Parallel Comput.

Computer Vision, Graphics, and Image Processing

J. Parallel Distributed Comput.

Parallel Comput.

Parallel Comput.

Parallel Comput.

Parallel Comput.

Paper
Performance prediction of large MIMD systems for parallel neural network simulations☆