Abstract
Three parallel physical optimization algorithms for allocating irregular data to multicomputer nodes are presented. They are based on simulated annealing, neural networks and genetic algorithms. All three algorithms deviate from the sequential versions in order to achieve acceptable speedups. The parallel simulated annealing (PSA) and neural network (PNN) algorithms include communication schemes that are adapted to the properties of the allocation problem and of the algorithms themselves for maintaining both good solutions and reasonable execution times. The parallel genetic algorithm (PGA) is based on a natural model of evolution. The performances of these algorithms are evaluated and compared. The three parallel algorithms maintain the good solution qualities of their sequential counterparts. Their comparison shows their suitability for different applications. For example, PGA yields the best solutions, but it is the slowest of the three. PNN is the fastest, but it yields lower quality solutions. PSA's performance lies in the middle.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aarts, E., and Korst, J. 1989.Simulated Annealing and Boltzmann Machines. New York, Wiley.
Berger, M., and Bokhari, S. 1987. A partitioning strategy for nonuniform problems on multiprocessors.IEEE Trans. Comps., C-36, 5(May):570–580.
Bout, D.E. Van den, and Miller, T.K. 1990. Graph partitioning using annealed neural networks.IEEE Trans. Neural Networks, 1,2:192–203.
Byun, H., Kortesis, S.K., and Houstis, E.N. 1992. A workload partitioning strategy for PDEs by a generalized neural network. Purdue Univ., Tech. Rept. CSD-TR-92-015.
Chrisochoides, N.P., Houstis, E.N., and Houstis, C.E. 1991. Geometry based mapping strategies for PDE computations. InInternat. Conf. on Supercomputing, ACM Press, pp. 115–127.
Eglese, R.W. 1990. Simulated annealing: A tool for operational research.Eur. J. Operational Research, 46:271–281.
Ercal, F. 1988. Heuristic approaches to task allocation for parallel computing. Ph.D. thesis, Ohio State Univ., Columbus, Oh.
Flower, J., Otto, S., and Salama, M. 1987. A preprocessor for finite element problems. InProc., Symp. on Parallel Computations and Their Impact on Mechanics, ASME Winter Meeting (Dec).
Fox, G.C. 1988. A graphical approach to load balancing and sparse matrix vector multiplication on the hypercube. InNumerical Algorithms for Modern Parallel Computers (M. Schultz, ed.), Berlin, Springer-Verlag.
Fox, G.C. 1990. Physical computation. InProc., Internat. Conf. on Parallel Computing: Achievements, Problems and Prospects (Italy, June).
Fox, G.C., and Furmanski, W. 1988. Load balancing loosely synchronous problems with a neural network. InProc., 3rd Conf. on Hypercube Concurrent Computers, and Applications, pp. 241–278.
Fox, G.C., Johnson, M., Lyzenga, G., Otto, S., Salmon, J., and Walker, D. 1988.Solving Problems on Concurrent Processors. Englewood Cliffs, N.J., Prentice-Hall.
Fox, G.C., Hiranandani, S., Kennedy, K., Koelbel, C., Kremer, U., Tseng, C., and Wu, M.-Y. 1990. Fortran D language specification. Syracuse Univ., NPAC, SCCS-42.
Garey, M.R., and Johnson, D.S. 1979.Computers and Intractability. Freeman.
Goldberg, D.E. 1989.Genetic Algorithms in Search, Optimization and Machine Learning. Reading, Mass., Addison-Wesley.
Greening, D.R. 1990. Parallel simulated annealing techniques.Physica D, 42:293–306.
Holland, J.H. 1975.Adaptation in Natural and Artificial Systems. Univ. of Michigan Press, Ann Arbor.
Hopfield, J.J., and Tank, D.W. 1986. Computing with neural circuits: A model.Science, 233:625–639.
Houstis, E.N., Rice, J.R., Chrisochoides, N.P., Karathonases, H.C., Papachiou, P.N., Samartzis, M.K., Vavalis, E.A., Wang, K. Y., and Weerawarana, S. 1990.//ELLPACK: A numerical simulation programming environment for parallel MIMD machines. InInternat. Conf. on Supercomputing, ACM Press, pp. 3–23.
Johnson, D.S., Aragon, C.R., McGeoch, L.A., and Schevon, C. 1989. Optimization by simulated annealing: An experimental evaluation; part 1, graph partitioning.Operations Research, 37,6:865–892.
Kirkpatrick, S., Gelatt, C.D., and Vecchi, M.P. 1983. Optimization by simulated annealing.Science, 220:671–680.
Mansour, N. 1992. Physical optimization algorithms for mapping data to distributed-memory multiprocessors. Ph.D. diss., Comp. Sci., Syracuse Univ., Syracuse, N.Y.
Mansour, N., and Fox, G.C. 1991. A hybrid genetic algorithm for task allocation. InProc., Internat. Conf. on Genetic Algorithms (July), pp. 466–473.
Mansour, N., and Fox, G.C. 1992a. Allocating data to multicomputer nodes by physical optimization algorithms for loosely synchronous computations.Concurrency: Practice and Experience, 4,7:557–574.
Mansour, N., and Fox, G.C. 1992b. Parallel genetic algorithms with application to load balancing parallel computations. InSupercomputing Symp. (Montreal, June), Atmospheric Environment Services, pp. 128–139.
Nolting, S. 1991. Nonlinear adaptive finite element systems on distributed memory computers. InEur. Distributed Memory Computing Conf. (Apr.), pp. 283–293.
Pothen, A., Simon, H., and Liou, K.-P. 1990. Partitioning sparse matrices with eigenvectors of graphs.SIAM J. Matrix Anal. Appl., 11, 3 (July):430–452.
Raman, S., and Patnaik, L.M. 1990. Annealing-based circuit partitioner for hypercube architectures.Internat. J. High Speed Computing, 2,1:69–84.
Simon, H. 1991. Partitioning of unstructured mesh problems for parallel processing. InProc., Conf. on Parallel Methods on Large Scale Structural Analysis and Physics Applications, Permagon Press.
Walker, D. 1990. Characterizing the parallel performance of a large-scale, particle-in-cell plasma simulation code.Concurrency: Practice and Experience (Dec.):257–288.
Williams, R.D. 1991. Performance of dynamic load balancing algorithms for unstructured mesh calculations.Concurrency: Practice and Experience, 3,5:457–481.
Wright, S. 1977.Evolution and the Genetics of Populations, vol. 3. Univ. of Chicago Press.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Mansour, N., Fox, G.C. Parallel physical optimization algorithms for allocating data to multicomputer nodes. J Supercomput 8, 53–80 (1994). https://doi.org/10.1007/BF01666908
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/BF01666908