Bhaskar Ghosh
ABSTRACT
The fundamental problems in dynamic load balancing and job scheduling in parallel and distributed computers involve moving load between processors. In this paper, we consider a new model for load movement in synchronous parallel and distributed machines. In each step of our model, each processor can transfer load to at most one neighbor; also, any amount of load can be moved along a communication link between two processors in one step. This is a reasonable model for load movement in significant classes of dynamic load balancing problems.
We derive efficient algorithms for a number of task reallocation problems under our model of load movement. These include dynamic load balancing on processor networks, adaptive mesh re-partitioning such as those in finite element methods, and progressive job migration under dynamic generation and consumption of load.
To obtain the above-mentioned results, we introduce and solve the abstract problem of Incremental Weight Migration (IWM) on arbitrary graphs. Our main result is a simple, randomized, algorithm for this problem which provably results in asymptotically optimal convergence towards the state where weights on the nodes of the graph are all equal. This algorithm utilizes an appropriate random set of edges forming a matching. Our algorithm for the IWM problem is used in deriving efficient algorithms for all the problems mentioned above.
Our results are very general. The algorithms we derive are local, and hence, scalable. They work for arbitrary load distributions and for networks of arbitrary topology which can possibly undergo link failures. Of independent interest is our proof technique which we use to lower bound the convergence of our algorithms in terms of the eigenstructure of the underlying graph.
Finally, we present preliminary experimental results analyzing issues in load balancing related to our algorithms.
- AA+93.W. Aiello, B. Awerbuch, B. Maggs, and S. l#ao. Approximate Load Balancing on Dynamic and Asynchronous Networks. In Proc. of 25th A CM Symp on Theory of Computing, 632-641, May 1993. Google Scholar
Digital Library
- AB92.Y. Aumann and M. Ben-Or. Computing with Faulty Arrays. In Proc of 24th A CM Syrup on Theory of Computing, 162-169, May 1992. Google ScholarDigital Library
- AHS91.J. Aspnes, M. Herlihy and N. Shavit. Counting Networks and Multiprocessor coordination, in Proc of 23rd A CM Syrup on Theory of Computing, 348-358, May 1991. Google ScholarDigital Library
- B87.B. Bollobas. Random Graphs. Academic Press, New York. 1987.Google Scholar
- BB87.M.J. Berger and S. H. Bokhari. A Partitioning Strategy for Nonuniform Problems on Multiprocessors. IEEE Trans. on Computers, Vol. C-36, No. 5,570-580, 1987. Google ScholarDigital Library
- CA87.G. Cybenko and T. G. Allen. Parallel Algorithms for Classification and Clustering. In Proc. SPIE Conference on Advanced Architectures and Algorithms for Signal Processing, San Diego, CA 1987.Google Scholar
- C89.G. Cybenko. Dynamic Load Balancing for Distributed Memory Multiprocessors. Journal of Parallel and Distributed Computing, Vol. 2, No. 7, 279-301, 1989. Google ScholarDigital Library
- F93.R. Feldmann. Game Tree Search on Massively Parallel Systems. PhD Thesis, Dept. of Mathematics and Computer Science, University of Paderborn. August 1993.Google Scholar
- FG91.M. Factor and D. Gelernter. Software Backplanes, Realtime Data Fusion and the Process Trellis. Research Report YALEU/DCS/TR-852, Yale Computer Science Department, March 1991.Google Scholar
- GH89.B. Goldberg and P. Hudak. Implementing Functional Programs on a Hypercube Multiprocessor. In Proc. of the 4th Conference on Hypercubes, Concurrent Computers and Applications, Vol. 1,489-503, 1989. Google ScholarDigital Library
- HCT89.J. Hong, M. Chen and X. Tan. Dynamic Cyclic Load Balancing on Hypercubes. in Proc. of the 4th Conference on Hypercubes, Concurrent Computers and Applications, Vol. 1,595-598, 1989.Google Scholar
- HLS92.M. Herlihy, B. Lim, and N. Shavit. Low contention load balancing on large-scale multiprocessors. In Proc. of 4th A CM Syrup on Parallel Algorithms and Architectures, 219-227, 1992. Google ScholarDigital Library
- HT93.A. Heirich and S. Taylor. A Parabolic Theory of Load Balance. Research Report Caltech-CS- TR-93-25, Caltech Scalable Concurrent Computation Lab, March 1993. Google ScholarDigital Library
- K88.L.V. Kale. Comparing the Performance of Two Dynamic Load Distribution Methods. In Proc. of International Conference on Parallel Processing, Vol. 1, August 1988.Google Scholar
- KZ88.R. Karp and Y. Zhang. A randomized parallel branch-and-bound procedure. In Proc. of 20lh A CM Syrup on Theory of Computing, 290-300, 1988. Google ScholarDigital Library
- LK87.F.C.H. Lin and R. M. Keller. The Gradient Model Load Balancing Method. IEEE Transactions on Software Engineering, Vol. 13, No. 1, 32-38, 1987. Google ScholarDigital Library
- LM93.R. Lueling and B. Monien. A Dynamic Distributed Load Balancing Algorithm with Provable Good Performance. In Proe. of 5th A CM Syrup on Parallel Algorithms and Architectures, 164-172, 1993. Google ScholarDigital Library
- LMR91.R. Lueling, B. Monien and F. Ramme. Load Balancing in Large Networks" A Comparative Study. In Proc. of IEEE Symp on Parallel and Distributed Computing, Dallas, 1991.Google ScholarDigital Library
- LN+89.T. Leighton, M. Newman, A. Ranade and E. Schwabe. Dynamic tree embeddings on butterflies and hypercubes. In Proc. of 1st A CM Syrup on Parallel Algorithms and Architectures, 224-234, 1989. Google ScholarDigital Library
- M89.M. Mihail. Conductance and Convergence of Markov Chains - A Combinatorial Treatment of Expanders. In Proe. of 30th IEEE Symp on Foundations of Computer Science, 526-531, October 1989.Google Scholar
- MP92.B. Mohar and S. Poljak. Eigenvalues in Combinatorial Optimization. Research Report 92752, IMA, Minneapolis, 1992.Google Scholar
- N92.D. Nicol. Communication Efficient Global Load Balancing. In Proc. of Scalable High Performance Computing Conference, 292- 299. Williamsburg, VA. April 1992.Google ScholarCross Ref
- NX+85.L. M. Ni, C. W. Xu and T. B. Gendreau. Drafting Algorithm- A Dynamic Process Migration Protocol for Distributed Systems. In Proc. of Int. Conf. on Distributed Computing Systems, 539-546, 1985.Google Scholar
- P89.C.G. Plaxton. Load Balancing, Selection and Sorting on the ttypercube. Proc. of 1st A CM Symp on Parallel Algorithms and Architectures, 64-73, 1989. Google ScholarDigital Library
- PU89.D. Peleg and E. Upfal. The token distribution problem. SIAM J. on Computing, Volume 18, 229-243, 1989. Google ScholarDigital Library
- R89.M.O. Rabin. Efficient dispersal of information for security, load balancing and fault tolerance. Journal of the A CM, Vol. 36, No. 3, 335-348, 1989. Google ScholarDigital Library
- R91.A. Ranade. Optimal speedup for backtrack search on a butterfly network. In Proc. 3rd A CM Syrup on Parallel Algorithms and Architectures, 40-49, 1991. Google ScholarDigital Library
- RSU91.L. Rudolph, M. Slivkin-Allalouf, and E. Upfal. A simple load balancing scheme for task allocation in parallel machines. In Proc. 3rd A CM Syrup on Parallel Algorithms and Architectures, 237-243, 1991. Google ScholarDigital Library
- W91.R.D. Williams. Performance of dynamic load balancing algorithms for unstructured mesh calculations. Concurrency: Pracice and Experience, Vol. 3, No. 5, 457-481, 1991. Google ScholarDigital Library
Index Terms
- Dynamic load balancing in parallel and distributed networks by random matchings (extended abstract)
Recommendations
Dynamic Load Balancing by Random Matchings
The fundamental problems in dynamic load balancing and job scheduling in parallel and distributed networks involve moving load between processors. In this paper we consider a new model for load movement in synchronous machines. In each step of our model,...
Adaptive Load Balancing Dashboard in Dynamic Distributed Systems
Considering the dynamic nature of new generation scientific problems, load balancing is a necessity to manage the load in an efficient manner. Load balancing systems are used to optimize the resource consumption, maximize the throughput, minimize ...
Comments