Illés Antal Horváth
Ziv Scully
Abstract
Load balancing plays a crucial role in many large scale computer systems. Much prior work has focused on systems with First-Come-First-Served (FCFS) servers. However, servers in practical systems are more complicated. They serve multiple jobs at once, and their service rate can depend on the number of jobs in service. Motivated by this, we study load balancing for systems using Limited-Processor-Sharing (LPS). Our model has heterogeneous servers, meaning the service rate curve and multiprogramming level (limit on the number of jobs sharing the processor) differs between servers. We focus on a specific load balancing policy: Join-Below-Threshold (JBT), which associates a threshold with each server and, whenever possible, dispatches to a server which has fewer jobs than its threshold. Given this setup, we ask: how should we configure the system to optimize objectives such as mean response time? Configuring the system means choosing both a load balancing threshold and a multiprogramming level for each server. To make this question tractable, we study the many-server mean field regime. In this paper we provide a comprehensive study of JBT in the mean field regime. We begin by developing a mean field model for the case of exponentially distributed job sizes. The evolution of our model is described by a differential inclusion, which complicates its analysis. We prove that the sequence of stationary measures of the finite systems converges to the fixed point of the differential inclusion, provided a unique fixed point exists. We derive simple conditions on the service rate curves to guarantee the existence of a unique fixed point. We demonstrate that when these conditions are not satisfied, there may be multiple fixed points, meaning metastability may occur. Finally, we give a simple method for determining the optimal system configuration to minimize the mean response time and related metrics. While our theoretical results are proven for the special case of exponentially distributed job sizes, we provide evidence from simulation that the system becomes insensitive to the job size distribution in the mean field regime, suggesting our results are more generally applicable.
- J. Abate and W. Whitt. Numerical inversion of Laplace transforms of probability distributions. ORSA Journal on computing, 7(1):36--43, 1995.Google Scholar
Cross Ref
- M. Bramson, Y. Lu, and B. Prabhakar. Randomized load balancing with general service time distributions. In ACM SIGMETRICS 2010, pages 275--286, 2010.Google ScholarDigital Library
- S. L. Brumelle. A generalization of Erlang's loss system to state dependent arrival and service rates. Mathematics of Operations Research, 3(1):10--16, 1978.Google ScholarDigital Library
- D. Gamarnik, J. N. Tsitsiklis, and M. Zubeldia. Delay, memory, and messaging tradeoffs in distributed service systems. SIGMETRICS Perform. Eval. Rev., 44(1):1--12, June 2016.Google ScholarDigital Library
- A. Ganesh, S. Lilienthal, D. Manjunath, A. Proutiere, and F. Simatos. Load balancing via random local search in closed and open systems. SIGMETRICS Perform. Eval. Rev., 38(1):287--298, June 2010.Google ScholarDigital Library
- N. Gast and B. Gaujal. A mean field model of work stealing in large-scale systems. SIGMETRICS Perform. Eval. Rev., 38(1):13--24, June 2010.Google ScholarDigital Library
- N. Gast and B. Gaujal. Markov chains with discontinuous drifts have differential inclusion limits. Performance Evaluation, 69(12):623 -- 642, 2012.Google ScholarDigital Library
- N. Gast and B. Van Houdt. A refined mean field approximation. Proc. ACM Meas. Anal. Comput. Syst., 1(2):33:1--33:28, December 2017.Google ScholarDigital Library
- I. Grosof, Z. Scully, and M. Harchol-Balter. Load balancing guardrails: Keeping your heavy traffic on the road to low response times. Proc. ACM Meas. Anal. Comput. Syst., 3(2):42:1--42:31, June 2019.Google ScholarDigital Library
- V. Gupta and M. Harchol-Balter. Self-adaptive admission control policies for resource-sharing systems. SIGMETRICS Perform. Eval. Rev., 37(1):311--322, June 2009.Google ScholarDigital Library
- M.W. Hirsch and H. Smith. Monotone dynamical systems. In Handbook of differential equations: ordinary differential equations, volume 2, pages 239--357. Elsevier, 2006.Google Scholar
- G. Horváth, I. Horváth, S. Al-Deen Almousa, and M. Telek. High order low variance matrix-exponential distributions. The Tenth International Conference on Matrix-Analytic Methods in Stochastic Models (MAM10), 02 2019.Google Scholar
- Y. Lu, Q. Xie, G. Kliot, A. Geller, J. R. Larus, and A. Greenberg. Join-idle-queue: A novel load balancing algorithm for dynamically scalable web services. Perform. Eval., 68:1056--1071, 2011.Google ScholarDigital Library
- M. Mitzenmacher. The power of two choices in randomized load balancing. IEEE Trans. Parallel Distrib. Syst., 12:1094--1104, October 2001.Google ScholarDigital Library
- M. Mitzenmacher. Analyzing distributed join-idle-queue: A fluid limit approach. In 2016 54th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pages 312--318, Sept 2016.Google ScholarCross Ref
- G. Roth and W.H. Sandholm. Stochastic approximations with constant step size and differential inclusions. SIAM Journal on Control and Optimization, 51(1):525--555, 2013.Google ScholarDigital Library
- A.L. Stolyar. Pull-based load distribution in large-scale heterogeneous service systems. Queueing Systems, 80(4):341--361, 2015.Google ScholarDigital Library
- M. Telek and B. Van Houdt. Response time distribution of a class of limited processor sharing queues. SIGMETRICS Perform. Eval. Rev., 45(3):143--155, March 2018.Google ScholarDigital Library
- T. Vasantam, A. Mukhopadhyay, and R. R. Mazumdar. The mean-field behavior of processor sharing systems with general job lengths under the sq(d) policy. Performance Evaluation, 127--128:120 -- 153, 2018.Google ScholarCross Ref
- N.D. Vvedenskaya, R.L. Dobrushin, and F.I. Karpelevich. Queueing system with selection of the shortest of two queues: an asymptotic approach. Problemy Peredachi Informatsii, 32:15--27, 1996.Google Scholar
- X. Zhou, J. Tan, and N. Shroff. Heavy-traffic delay optimality in pull-based load balancing systems: Necessary and sufficient conditions. Proceedings of the ACM on Measurement and Analysis of Computing Systems, 2(3):41, 2018.Google ScholarDigital Library
Index Terms
- Mean Field Analysis of Join-Below-Threshold Load Balancing for Resource Sharing Servers
Recommendations
Mean Field Analysis of Join-Below-Threshold Load Balancing for Resource Sharing Servers
Load balancing plays a crucial role in many large scale computer systems. Much prior work has focused on systems with First-Come- First-Served (FCFS) servers. However, servers in practical systems are more complicated. They serve multiple jobs at once, ...
Mean Field Analysis of Join-Below-Threshold Load Balancing for Resource Sharing Servers
SIGMETRICS '20: Abstracts of the 2020 SIGMETRICS/Performance Joint International Conference on Measurement and Modeling of Computer SystemsLoad balancing plays a crucial role in many large scale computer systems. Much prior work has focused on systems with First-Come-First-Served (FCFS) servers. However, servers in practical systems are more complicated. They serve multiple jobs at once, ...
Market-Based Load Balancing for Distributed Heterogeneous Multi-Resource Servers
ICPADS '09: Proceedings of the 2009 15th International Conference on Parallel and Distributed SystemsTo cope with rapidly increasing Internet usage nowadays, providing Internet services using multiple servers has become a necessity. To ensure sufficient service quality and server utilization at the same time, effective methods are needed to spread load ...
Comments