Abstract
Multiprocessor load balancing aims to improve performance by moving jobs from highly loaded processors to more lightly loaded processors. Some schemes allow only migration of new jobs upon arrival, while other schemes allow migration of jobs in progress. A difficulty with all these schemes, however, is that they require continuously maintaining detailed state information. In this paper we consider the alternative of periodic load balancing, in which the loads are balanced only at each T time units for some appropriate T. With periodic load balancing, state information is only needed at the balancing times. Moreover, it is often possible to use slightly stale information collected during the interval between balancing times. In this paper we study the performance of periodic load balancing. We consider multiple queues in parallel with unlimited waiting space to which jobs come either in separate independent streams or by assignment (either random or cyclic) from a single stream. Resource sharing is achieved by periodically redistributing the jobs or the work in the system among the queues. The performance of these systems of queues coupled by periodic load balancing depends on the transient behavior of a single queue. We focus on useful approximations obtained by considering a large number of homogeneous queues and a heavy load. When the number of queues is sufficiently large, the number of jobs or quantity of work at each queue immediately after redistribution tends to evolve deterministically, by the law of large numbers. The steady-state (limiting) value of this deterministic sequence is obtained as the solution of a fixed point equation, where the initial value is equal to the expected transient value over the interval between successive redistributions conditional on the initial value. A refined approximation based on the central limit theorem is a normal distribution, where the mean and variance are obtained by solving a pair of fixed-point equations. With higher loads, which is natural to consider when load balancing is performed, a heavy-traffic limit theorem shows that one-dimensional reflected Brownian motion can be used to approximately describe system performance, even with general arrival and service processes. With these approximations, we show how performance depends on the assumed arrival pattern of jobs and the model parameters. We do numerical calculations and conduct simulation experiments to show the accuracy of the approximations.
Similar content being viewed by others
References
J. Abate and W. Whitt, Transient behavior of regulated Brownian motion, I and II, Adv. in Appl. Probab. 19 (1987) 560-631.
J. Abate and W. Whitt, Transient behavior of the M/M/1 queue via Laplace transforms, Adv. in Appl. Probab. 20 (1988) 145-178.
J. Abate and W. Whitt, Calculating time-dependent performance measures for the M/M/1 queue, IEEE Trans. Commun. 37 (1989) 1102-1104.
J. Abate and W. Whitt, The Fourier-series method for inverting transforms of probability distributions, Queueing Systems 10 (1992) 5-88.
J. Abate and W. Whitt, Numerical inversion of Laplace transforms of probability distributions, ORSA J. Comput. 7 (1995) 36-43.
M. Abramowitz and I.A. Stegun, Handbook of Mathematical Functions (Dover, New York, 1972).
F. Baccelli and P. Brémaud, Elements of Queueing Theory (Springer, New York, 1994).
A. Barak, G. Shai and R.G. Wheeler, The MOSIX Distributed Operating System: Load Balancing for UNIX (Springer, Berlin, 1993).
P. Billingsley, Convergence of Probability Measures (Wiley, New York, 1968).
A.A. Borovkov, Stochastic Processes in Queueing Theory (Springer, New York, 1976).
G.L. Choudhury, D.M. Lucantoni and W. Whitt, Multidimensional transform inversion with applications to the transient M/G/1 queue, Ann. Appl. Probab. 4 (1994) 719-740.
E. Çinlar, Superpositions of point processes, in: Stochastic Point Processes: Statistical Analysis, Theory and Applications, ed. P.A.W. Lewis (Wiley, New York, 1972) pp. 549-606.
J.L. Davis, W.A. Massey and W. Whitt, Sensitivity to the service-time distribution in the nonstationary Erlang loss model, Managm. Sci. 41 (1995) 1107-1116.
D.L. Eager, E.D. Lazowska and J. Zahorjan, Adaptive load balancing in homogeneous distributed systems, IEEE Trans. Software Engrg. 12 (1986) 662-675.
S.N. Ethier and T.G. Kurtz, Characterization and Approximation of Markov Processes (Wiley, New York, 1986).
W. Feller, An Introduction to Probability Theory and Its Applications, Vol. I (Wiley, New York, 1968).
K.W. Fendick, V.R. Saksena and W. Whitt, Dependence in packet queues, IEEE Trans. Commun. 37 (1989) 1173-1183.
G.J. Foschini, Unobtrusive communication of status in a packet network in heavy traffic, AT&T Tech. J. 64 (1985) 463-479.
G.J. Foschini and J. Salz, A basic dynamic routing problem and diffusion, IEEE Trans. Commun. 26 (1978) 320-327.
B. Hajek, Performance of global load balancing by local adjustment, IEEE Trans. Inform. Theory 36 (1990) 1398-1414.
M. Harchol-Balter and A.B. Downey, Exploiting process lifetime distributions for dynamic load balancing, in: Proc. SIGMETRICS '96 (1996).
G. Hjálmtýsson, Lightweight call setup — supporting connection and connectionless services, in: Teletraffic Contributions for the Information Age, Proc. of the 15th Internat. Teletraffic Congress, eds. V. Ramaswami and P.E. Wirth (Elsevier, Amsterdam, 1997) pp. 35-45.
G. Hjálmtýsson and K.K. Ramakrishnan, UNITE — An architecture for lightweight signalling in ATM networks, in: IEEE Infocom '98 (1998) pp. 832-840.
D.L. Iglehart and W. Whitt, Multiple channel queues in heavy traffic, I and II, Adv. in Appl. Probab. 2 (1970) 150-177 and 355–369.
O. Kella and W. Whitt, Diffusion approximations for queues with server vacations, Adv. in Appl. Probab. 22 (1990) 706-729.
J. Köllerström, Heavy traffic theory for queues with several servers, I, J. Appl. Probab. 11 (1974) 544-552.
C.N. Laws, Resource pooling in queueing networks with dynamic routing, Adv. in Appl. Probab. 24 (1992) 699-726.
W.E. Leland and T.J. Ott, Load balancing heuristics and process behavior, Sigmetrics 86(14) (1986) 54-69.
A. Mandelbaum and M.I. Reiman, On pooling in queueing networks, Managm. Sci., to appear.
W.A. Massey and W. Whitt, Peak congestion in multi-server service systems with slowly varying arrival rates, Queueing Systems 25 (1997) 157-172.
S.P. Meyn and R.L. Tweedie, Markov Chains and Stochastic Stability (Springer, New York, 1993).
M.I. Reiman, Some diffusion approximate with state space collapse, in: Modelling and Performance Evaluation Methodology, eds. F. Baccelli and G. Fayolle (Springer, Berlin, 1984) pp. 209-240.
M.H. Rothkopf and P. Rech, Perspectives on queues: Combining queues is not always beneficial, Oper. Res. 35 (1987) 906-909.
H. Sakasegawa, An approximate formula Lq = αβρ/(1 — ρ), Ann. Inst. Statist. Math. 29 (1977) 67-75.
R.F. Serfozo, Partitions of point processes: multivariate Poisson approximations, Stochastic Process. Appl. 20 (1985) 281-294.
D.R. Smith and W. Whitt, Resource sharing for efficiency in traffic systems, Bell System Tech. J. 60 (1981) 39-55.
D. Stoyan, Comparison Methods for Queues and Other Stochastic Models (Wiley, New York, 1983).
M.R. Taaffe and K.L. Ong, Approximating Ph(t)/M(t)/S/C queueing systems, Ann. Oper. Res. 8 (1987) 103-116.
L. Takács, Introduction to the Theory of Queues (Oxford Univ. Press, New York, 1962).
E. van Doorn, Stochastic Monotonicity and Queueing Applications of Birth-Death Processes (Springer, New York, 1981).
R.W. Weber, On the optimal assignment of customers to parallel servers, J. Appl. Probab. 15 (1978) 406-413.
W. Whitt, Weak convergence theorems for priority queues: Preemptive-resume discipline, J. Appl. Probab. 8 (1971) 74-94.
W. Whitt, Some useful functions for functional limit theorems, Math. Oper. Res. 5 (1980) 67-85.
W. Whitt, Comparing counting processes and queues, Adv. in Appl. Probab. 13 (1981) 207-220.
W. Whitt, Deciding which queue to join: Some counterexamples, Oper. Res. 34 (1986) 55-62.
W. Whitt, Planning queueing simulations, Managm. Sci. 35 (1989) 1341-1366.
W. Whitt, Understanding the efficiency of multi-server service systems, Managm. Sci. 38 (1992) 708-723.
W. Whitt, Approximations for the GI/G/m queue, Production Oper. Managm. 2 (1993) 114-160.
M.H. Willebeck-LeMair and A.P. Reeves, Strategies for dynamic load balancing on highly parallel computers, IEEE Trans. Parallel Distrib. Systems 9 (1993) 979-993.
W. Winston, Optimality of the shortest line discipline, J. Appl. Probab. 14 (1977) 181-189.
R.W. Wolff, An upper bound for multi-channel queues, J. Appl. Probab. 14 (1977) 884-888.
H. Zhang, G. Hsu and R. Wang, Heavy traffic limit theorems for a sequence of shortest queueing systems, Queueing Systems 21 (1995) 217-238.
S. Zhou, A trace-driven simulation study of dynamic load balancing, IEEE Trans. Software Engrg. 14 (1988) 1327-1341.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Hjálmtýsson, G., Whitt, W. Periodic load balancing. Queueing Systems 30, 203–250 (1998). https://doi.org/10.1023/A:1019165004692
Issue Date:
DOI: https://doi.org/10.1023/A:1019165004692