Cited By
View all- Goldsztajn D(2025)Fluid Limits and Optimal Task Assignment Policies for Locally Pooled Service SystemsACM SIGMETRICS Performance Evaluation Review10.1145/3712170.371217452:3(7-10)Online publication date: 9-Jan-2025
Server Saturation in Skewed Networks
Article No.: 26, Pages 1 - 37
Abstract
We consider a model inspired by compatibility constraints that arise between tasks and servers in data centers, cloud computing systems and content delivery networks. The constraints are represented by a bipartite graph or network that interconnects dispatchers with compatible servers. Each dispatcher receives tasks over time and sends every task to a compatible server with the least number of tasks, or to a server with the least number of tasks among \mathrmd compatible servers selected uniformly at random. We focus on networks where the neighborhood of at least one server is skewed in a limiting regime. This means that a diverging number of dispatchers are in the neighborhood which are each compatible with a uniformly bounded number of servers; thus, the degree of the central server approaches infinity while the degrees of many neighboring dispatchers remain bounded. We prove that each server with a skewed neighborhood saturates, in the sense that the mean number of tasks queueing in front of it in steady state approaches infinity. Paradoxically, this pathological behavior can even arise in random networks where nearly all the servers have at most one task in the limit.
References
[1]
Robert B Ash. 1990. Information theory. Dover.
[2]
Jacques Bahi, Raphaël Couturier, and Flavien Vernier. 2003. Broken edges and dimension exchange algorithms on hypercube topology. In Eleventh Euromicro Conference on Parallel, Distributed and Network-Based Processing. IEEE, 140--145.
[3]
Béla Bollobás. 1981. Degree sequences of random graphs. Discrete Mathematics, Vol. 33, 1 (1981), 1--19.
[4]
Mark noopBoor van der Boor, Sem C Borst, Johan S H van Leeuwaarden, and Debankur Mukherjee. 2022. Scalable load balancing in networked systems: A survey of recent advances. SIAM Rev., Vol. 64, 3 (2022), 554--622.
[5]
Maury Bramson. 2011. Stability of join the shortest queue networks. The Annals of Applied Probability, Vol. 21, 4 (2011), 1568--1625.
[6]
Anton Braverman, JG Dai, and Jiekun Feng. 2016. Stein's method for steady-state diffusion approximations: an introduction through the Erlang-A and Erlang-C models. Stochastic Systems, Vol. 6, 2 (2016), 301--366.
[7]
Amarjit Budhiraja, Debankur Mukherjee, and Ruoyu Wu. 2019. Supermarket model on graphs. The Annals of Applied Probability, Vol. 29, 3 (2019), 1740--1777.
[8]
Ellen Cardinaels, Sem C Borst, and Johan S H van Leeuwaarden. 2019. Job assignment in large-scale service systems with affinity relations. Queueing Systems, Vol. 93 (2019), 227--268.
[9]
James Cruise, Matthieu Jonckheere, and Seva Shneer. 2020. Stability of JSQ in queues with general server-job class compatibilities. Queueing Systems, Vol. 95 (2020), 271--279.
[10]
George Cybenko. 1989. Dynamic load balancing for distributed memory multiprocessors. Journal of parallel and distributed computing, Vol. 7, 2 (1989), 279--301.
[11]
Robert Elsasser, Burkhard Monien, and Stefan Schamberger. 2004. Load balancing in dynamic networks. In 7th International Symposium on Parallel Architectures, Algorithms and Networks. IEEE, 193--200.
[12]
Atilla Eryilmaz and Rayadurgam Srikant. 2012. Asymptotically tight steady-state queue length bounds implied by drift conditions. Queueing Systems, Vol. 72 (2012), 311--359.
[13]
Stewart N Ethier and Thomas G Kurtz. 2009. Markov processes: characterization and convergence. John Wiley & Sons.
[14]
Serguei G Foss and Natalia I Chernova. 1998. On the stability of a partially accessible multi-station queue with state-dependent routing. Queueing Systems, Vol. 29 (1998), 55--73.
[15]
Nicolas Gast. 2015. The power of two choices on graphs: the pair-approximation is accurate? ACM SIGMETRICS Performance Evaluation Review, Vol. 43, 2 (2015), 69--71.
[16]
Nicolas Gast and Benny Van Houdt. 2017. A refined mean field approximation. Proceedings of the ACM on Measurement and Analysis of Computing Systems, Vol. 1, 2 (2017), 1--28.
[17]
Seth Gilbert, Uri Meir, Ami Paz, and Gregory Schwartzman. 2021. On the complexity of load balancing in dynamic networks. In Proceedings of the 33rd ACM Symposium on Parallelism in Algorithms and Architectures. 254--264.
[18]
Peter W Glynn and Assaf Zeevi. 2008. Bounding stationary expectations of Markov processes. In Markov processes and related topics: a Festschrift for Thomas G. Kurtz. Vol. 4. Institute of Mathematical Statistics, 195--214.
[19]
Diego Goldsztajn, Sem C Borst, and Johan S H van Leeuwaarden. 2023. Load balancing with sparse dynamic random graphs. arXiv preprint arXiv:2305.13054 (2023).
[20]
Itai Gurvich. 2014. Diffusion models and steady-state approximations for exponentially ergodic Markovian queues. The Annals of Applied Probability, Vol. 24, 6 (2014), 2527--2559.
[21]
Krishnaram Kenthapadi and Rina Panigrahy. 2006. Balanced allocation on graphs. In SODA, Vol. 6. 434--443.
[22]
Xin Liu and Lei Ying. 2020. Steady-state analysis of load-balancing algorithms in the sub-Halfin--Whitt regime. Journal of Applied Probability, Vol. 57, 2 (2020), 578--596.
[23]
Siva Theja Maguluri and R Srikant. 2016. Heavy traffic queue length behavior in a switch under the MaxWeight algorithm. Stochastic Systems, Vol. 6, 1 (2016), 211--250.
[24]
Ronald Menich and Richard F Serfozo. 1991. Optimality of routing and servicing in dependent parallel processing systems. Queueing Systems, Vol. 9 (1991), 403--418.
[25]
Michael Mitzenmacher. 2001. The power of two choices in randomized load balancing. IEEE Transactions on Parallel and Distributed Systems, Vol. 12, 10 (2001), 1094--1104.
[26]
Debankur Mukherjee, Sem C Borst, and Johan S H van Leeuwaarden. 2018. Asymptotically optimal load balancing topologies. Proceedings of the ACM on Measurement and Analysis of Computing Systems, Vol. 2, 1 (2018), 1--29.
[27]
Erik Nygren, Ramesh K Sitaraman, and Jennifer Sun. 2010. The Akamai network: a platform for high-performance internet applications. ACM SIGOPS Operating Systems Review, Vol. 44, 3 (2010), 2--19.
[28]
Daan Rutten and Debankur Mukherjee. 2023 a. Load balancing under strict compatibility constraints. Mathematics of Operations Research, Vol. 48, 1 (2023), 227--256.
[29]
Daan Rutten and Debankur Mukherjee. 2023 b. Mean-field Analysis for Load Balancing on Spatial Graphs. ACM SIGMETRICS Performance Evaluation Review, Vol. 51, 1 (2023), 27--28.
[30]
Daan Rutten and Debankur Mukherjee. 2023 c. Mean-field Analysis for Load Balancing on Spatial Graphs. arXiv preprint arXiv:2301.03493 (2023).
[31]
Panayotis D Sparaggis, D Towsley, and CG Cassandras. 1993. Extremal properties of the shortest/longest non-full queue policies in finite-capacity systems with state-dependent service rates. Journal of Applied Probability, Vol. 30 (1993), 223--236.
[32]
Dengwang Tang and Vijay G Subramanian. 2019. Random walk based sampling for load balancing in multi-server systems. Proceedings of the ACM on Measurement and Analysis of Computing Systems, Vol. 3, 1 (2019), 1--44.
[33]
Stephen RE Turner. 1998. The effect of increasing routing choice on resource pooling. Probability in the Engineering and Informational Sciences, Vol. 12 (1998), 109--124.
[34]
Nikita Dmitrievna Vvedenskaya, Roland L'vovich Dobrushin, and Fridrikh Izrailevich Karpelevich. 1996. Queueing system with selection of the shortest of two queues: An asymptotic approach. Problemy Peredachi Informatsii, Vol. 32, 1 (1996), 20--34.
[35]
Weina Wang, Siva Theja Maguluri, R Srikant, and Lei Ying. 2018. Heavy-traffic delay insensitivity in connection-level models of data transfer with proportionally fair bandwidth sharing. ACM SIGMETRICS Performance Evaluation Review, Vol. 45, 3 (2018), 232--245.
[36]
Weina Wang, Kai Zhu, Lei Ying, Jian Tan, and Li Zhang. 2016. MapTask scheduling in MapReduce with data locality: Throughput and heavy-traffic optimality. IEEE/ACM Transactions on Networking, Vol. 24, 1 (2016), 190--203.
[37]
Wentao Weng, Xingyu Zhou, and R Srikant. 2020a. Optimal load balancing in bipartite graphs. arXiv preprint arXiv:2008.08830 (2020).
[38]
Wentao Weng, Xingyu Zhou, and R Srikant. 2020b. Optimal load balancing with locality constraints. Proceedings of the ACM on Measurement and Analysis of Computing Systems, Vol. 4, 3 (2020), 1--37.
[39]
Udi Wieder. 2017. Hashing, load balancing and multiple choice. Foundations and Trends® in Theoretical Computer Science, Vol. 12, 3--4 (2017), 275--379.
[40]
Qiaomin Xie and Yi Lu. 2015. Priority algorithm for near-data scheduling: Throughput and heavy-traffic optimality. In 2015 IEEE International Conference on Computer Communications (INFOCOM). IEEE, 963--972.
[41]
Qiaomin Xie, Ali Yekkehkhany, and Yi Lu. 2016. Scheduling with multi-level data locality: Throughput and heavy-traffic optimality. In 2016 IEEE International Conference on Computer Communications (INFOCOM). IEEE, 1--9.
[42]
Lei Ying. 2017. Stein's method for mean field approximations in light and heavy traffic regimes. Proceedings of the ACM on Measurement and Analysis of Computing Systems, Vol. 1, 1 (2017), 1--27.
[43]
Zhisheng Zhao and Debankur Mukherjee. 2023. Optimal Rate-Matrix Pruning For Large-Scale Heterogeneous Systems. arXiv preprint arXiv:2306.00274 (2023).
[44]
Zhisheng Zhao, Debankur Mukherjee, and Ruoyu Wu. 2022. Exploiting Data Locality to Improve Performance of Heterogeneous Server Clusters. arXiv preprint arXiv:2211.16416 (2022).
Index Terms
- Server Saturation in Skewed Networks
Recommendations
Server Saturation in Skewed Networks
SIGMETRICS '24We use bipartite graphs to model compatibility constraints that arise between tasks and servers in data centers, cloud computing systems and content delivery networks. We prove that servers with skewed graph neighborhoods saturate with tasks in a ...
Server Saturation in Skewed Networks
SIGMETRICS/PERFORMANCE '24: Abstracts of the 2024 ACM SIGMETRICS/IFIP PERFORMANCE Joint International Conference on Measurement and Modeling of Computer SystemsWe use bipartite graphs to model compatibility constraints that arise between tasks and servers in data centers, cloud computing systems and content delivery networks. We prove that servers with skewed graph neighborhoods saturate with tasks in a ...
A Technique to Assign an Appropriate Server to a Client, for a CDN Consists of Servers at the Global Internet and Hierarchical Private Networks
COMPSACW '14: Proceedings of the 2014 IEEE 38th International Computer Software and Applications Conference WorkshopsThis paper discusses a technique to assign an appropriate server to a client for a content delivery network (CDN). We assume that the CDN consists of not only servers in the global Internet but also servers in hierarchical private networks. To use a ...
Comments
Information & Contributors
Information
Published In
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 29 May 2024
Published in POMACS Volume 8, Issue 2
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- Netherlands Organisation for Scientific Research (NWO)
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 71Total Downloads
- Downloads (Last 12 months)71
- Downloads (Last 6 weeks)1
Reflects downloads up to 03 Mar 2025
Other Metrics
Citations
Cited By
View all- Goldsztajn D(2025)Fluid Limits and Optimal Task Assignment Policies for Locally Pooled Service SystemsACM SIGMETRICS Performance Evaluation Review10.1145/3712170.371217452:3(7-10)Online publication date: 9-Jan-2025
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in