skip to main content
10.1145/3618260.3649608acmconferencesArticle/Chapter ViewAbstractPublication PagesstocConference Proceedingsconference-collections
research-article

Breaking the VLB Barrier for Oblivious Reconfigurable Networks

Published: 11 June 2024 Publication History

Abstract

In a landmark 1981 paper, Valiant and Brebner gave birth to the study of oblivious routing and, simultaneously, introduced its most powerful and ubiquitous method: Valiant load balancing (VLB). By routing messages through a randomly sampled intermediate node, VLB lengthens routing paths by a factor of two but gains the crucial property of obliviousness: it balances load in a completely decentralized manner, with no global knowledge of the communication pattern. Forty years later, with datacenters handling workloads whose communication pattern varies too rapidly to allow centralized coordination, oblivious routing is as relevant as ever, and VLB continues to take center stage as a widely used — and in some settings, provably optimal — way to balance load in the network obliviously to the traffic demands. However, the ability of the network to rapidly reconfigure its interconnection topology gives rise to new possibilities.
In this work we revisit the question of whether VLB remains optimal in the novel setting of reconfigurable networks. Prior work showed that VLB achieves the optimal tradeoff between latency and guaranteed throughput. In this work we show that a strictly superior latency-throughput tradeoff is achievable when the throughput bound is relaxed to hold with high probability. The same improved tradeoff is also achievable with guaranteed throughput under time-stationary demands, provided the latency bound is relaxed to hold with high probability and that the network is allowed to be semi-oblivious, using an oblivious (randomized) connection schedule but demand-aware routing. We prove that the latter result is not achievable by any fully-oblivious reconfigurable network design, marking a rare case in which semi-oblivious routing has a provable asymptotic advantage over oblivious routing. Our results are enabled by a novel oblivious routing scheme that improves VLB by stretching routing paths the minimum possible amount — an additive stretch of 1 rather than a multiplicative stretch of 2 — yet still manages to balance load with high probability when either the traffic demand matrix or the network’s interconnection schedule are shuffled by a uniformly random permutation. To analyze our routing scheme we prove an exponential tail bound which may be of independent interest, concerning the distribution of values of a bilinear form on an orbit of a permutation group action.

References

[1]
Vamsi Addanki, Chen Avin, and Stefan Schmid. 2023. Mars: Near-optimal throughput with shallow buffers in reconfigurable datacenter networks. Proceedings of the ACM on Measurement and Analysis of Computing Systems, 7, 1 (2023), 1–43.
[2]
William A. Aiello, F. T. Leighton, Bruce M. Maggs, and Mark Newman. 1991. Fast algorithms for bit-serial routing on a hypercube. Mathematical systems theory, 24, 1 (1991), 253–271. isbn:1433-0490 https://doi.org/10.1007/BF02090402
[3]
Daniel Amir, Tegan Wilson, Vishal Shrivastav, Hakim Weatherspoon, Robert Kleinberg, and Rachit Agarwal. 2022. Optimal Oblivious Reconfigurable Networks. In Proceedings of the 54th Annual ACM SIGACT Symposium on Theory of Computing (STOC 2022). Association for Computing Machinery, New York, NY, USA. 1339–1352. isbn:9781450392648 https://doi.org/10.1145/3519935.3520020
[4]
David L. Applegate and Edith Cohen. 2003. Making intra-domain routing robust to changing and uncertain traffic demands: understanding fundamental tradeoffs. In Proceedings of the ACM SIGCOMM 2003 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication, August 25-29, 2003, Karlsruhe, Germany, Anja Feldmann, Martina Zitterbart, Jon Crowcroft, and David Wetherall (Eds.). ACM, 313–324. https://doi.org/10.1145/863955.863991
[5]
Yossi Azar, Edith Cohen, Amos Fiat, Haim Kaplan, and Harald Räcke. 2003. Optimal Oblivious Routing in Polynomial Time. In Proceedings of the Thirty-Fifth Annual ACM Symposium on Theory of Computing (STOC ’03). Association for Computing Machinery, New York, NY, USA. 383–388. isbn:1581136749 https://doi.org/10.1145/780542.780599
[6]
Moshe Babaioff and John Chuang. 2007. On the optimality and interconnection of valiant load-balancing networks. In IEEE INFOCOM 2007-26th IEEE International Conference on Computer Communications. 80–88.
[7]
Hitesh Ballani, Paolo Costa, Raphael Behrendt, Daniel Cletheroe, Istvan Haller, Krzysztof Jozwik, Fotini Karinou, Sophie Lange, Kai Shi, and Benn Thomsen. 2020. Sirius: A Flat Datacenter Network with Nanosecond Optical Switching. In Proceedings of the Annual conference of the ACM Special Interest Group on Data Communication on the applications, technologies, architectures, and protocols for computer communication. 782–797.
[8]
Kashinath Basu, Ali Maqousi, and Frank Ball. 2020. Architecture of an end-to-end energy consumption model for a Cloud Data Center. In 2020 12th International Symposium on Communication Systems, Networks and Digital Signal Processing (CSNDSP). 1–6.
[9]
Marcin Bienkowski, Miroslaw Korzeniowski, and Harald Räcke. 2003. A Practical Algorithm for Constructing Oblivious Routing Schemes. In Proceedings of the Fifteenth Annual ACM Symposium on Parallel Algorithms and Architectures (SPAA ’03). Association for Computing Machinery, New York, NY, USA. 24–33. isbn:1581136617 https://doi.org/10.1145/777412.777418
[10]
Allan Borodin and John E. Hopcroft. 1985. Routing, Merging, and Sorting on Parallel Models of Computation. J. Comput. Syst. Sci., 30 (1985), 130–145.
[11]
Q. Cheng, A. Wonfor, J. L. Wei, R. V. Penty, and I. H. White. 2014. Demonstration of the feasibility of large-port-count optical switching using a hybrid Mach–Zehnder interferometer–semiconductor optical amplifier switch module in a recirculating loop. Opt. Lett., 39, 18 (2014), Sep, 5244–5247. https://doi.org/10.1364/OL.39.005244
[12]
M. Ding, A. Wonfor, Q. Cheng, R. V. Penty, and I. H. White. 2017. Scalable, low-power-penalty nanosecond reconfigurable hybrid optical switches for data centre networks. In 2017 Conference on Lasers and Electro-Optics (CLEO). 1–2.
[13]
Devdatt P Dubhashi and Desh Ranjan. 1996. Balls and bins: A study in negative dependence. BRICS Report Series, 3, 25 (1996).
[14]
Jittat Fakcharoenphol, Satish Rao, and Kunal Talwar. 2004. A tight bound on approximating arbitrary metrics by tree metrics. J. Comput. Syst. Sci., 69, 3 (2004), 485–497. https://doi.org/10.1016/j.jcss.2004.04.011
[15]
Nathan Farrington, George Porter, Sivasankar Radhakrishnan, Hamid Hajabdolali Bazzaz, Vikram Subramanya, Yeshaiahu Fainman, George Papen, and Amin Vahdat. 2010. Helios: a hybrid electrical/optical switch architecture for modular data centers. In Proceedings of ACM SIGCOMM.
[16]
Mohsen Ghaffari, Bernhard Haeupler, and Goran Zuzic. 2021. Hop-Constrained Oblivious Routing. In Proceedings of the 53rd Annual ACM SIGACT Symposium on Theory of Computing (STOC 2021). Association for Computing Machinery, New York, NY, USA. 1208–1220. isbn:9781450380539 https://doi.org/10.1145/3406325.3451098
[17]
Monia Ghobadi, Ratul Mahajan, Amar Phanishayee, Nikhil Devanur, Janardhan Kulkarni, Gireeja Ranade, Pierre-Alexandre Blanche, Houman Rastegarfar, Madeleine Glick, and Daniel Kilper. 2016. ProjecToR: Agile Reconfigurable Data Center Interconnect. In Proceedings of the 2016 ACM SIGCOMM Conference (SIGCOMM ’16). Association for Computing Machinery, New York, NY, USA. 216–229. isbn:9781450341936 https://doi.org/10.1145/2934872.2934911
[18]
Soudeh Ghorbani, Zibin Yang, P. Brighten Godfrey, Yashar Ganjali, and Amin Firoozshahian. 2017. DRILL: Micro Load Balancing for Low-Latency Data Center Networks. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication (SIGCOMM ’17). Association for Computing Machinery, New York, NY, USA. 225–238. isbn:9781450346535 https://doi.org/10.1145/3098822.3098839
[19]
Albert Greenberg, James Hamilton, David A Maltz, and Parveen Patel. 2008. The cost of a cloud: research problems in data center networks. 68–73 pages.
[20]
Chen Griner, Johannes Zerwas, Andreas Blenk, Manya Ghobadi, Stefan Schmid, and Chen Avin. 2021. Cerberus: The power of choices in datacenter topology design-a throughput perspective. Proceedings of the ACM on Measurement and Analysis of Computing Systems, 5, 3 (2021), 1–33.
[21]
Anupam Gupta, Mohammad Taghi Hajiaghayi, and Harald Räcke. 2006. Oblivious network design. In Proceedings of the Seventeenth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2006, Miami, Florida, USA, January 22-26, 2006. ACM Press, 970–979. http://dl.acm.org/citation.cfm?id=1109557.1109665
[22]
MohammadTaghi Hajiaghayi, Jeong Han Kim, Tom Leighton, and Harald Räcke. 2005. Oblivious Routing in Directed Graphs with Random Demands. In Proceedings of the Thirty-Seventh Annual ACM Symposium on Theory of Computing (STOC ’05). Association for Computing Machinery, New York, NY, USA. 193–201. isbn:1581139608 https://doi.org/10.1145/1060590.1060619
[23]
Navid Hamedazimi, Zafar Qazi, Himanshu Gupta, Vyas Sekar, Samir R. Das, Jon P. Longtin, Himanshu Shah, and Ashish Tanwer. 2014. FireFly: A Reconfigurable Wireless Data Center Fabric Using Free-Space Optics. In Proceedings of the 2014 ACM Conference on SIGCOMM (SIGCOMM ’14). Association for Computing Machinery, New York, NY, USA. 319–330. isbn:9781450328364 https://doi.org/10.1145/2619239.2626328
[24]
Chris Harrelson, Kirsten Hildrum, and Satish Rao. 2003. A polynomial-time tree decomposition to minimize congestion. In SPAA 2003: Proceedings of the Fifteenth Annual ACM Symposium on Parallelism in Algorithms and Architectures, June 7-9, 2003, San Diego, California, USA (part of FCRC 2003), Arnold L. Rosenberg and Friedhelm Meyer auf der Heide (Eds.). ACM, 34–43. https://doi.org/10.1145/777412.777419
[25]
Christos Kaklamanis, Danny Krizanc, and Thanasis Tsantilas. 1991. Tight Bounds for Oblivious Routing in the Hypercube. Math. Syst. Theory, 24, 4 (1991), 223–232. https://doi.org/10.1007/BF02090400
[26]
Isaac Keslassy, Cheng-Shang Chang, Nick McKeown, and Duan-Shin Lee. 2005. Optimal load-balancing. In Proceedings IEEE 24th Annual Joint Conference of the IEEE Computer and Communications Societies. 3, 1712–1722.
[27]
Praveen Kumar, Yang Yuan, Chris Yu, Nate Foster, Robert Kleinberg, Petr Lapukhov, Chiunlin Lim, and Robert Soulé. 2018. Semi-Oblivious Traffic Engineering: The Road Not Taken. In 15th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2018, Renton, WA, USA, April 9-11, 2018, Sujata Banerjee and Srinivasan Seshan (Eds.). USENIX Association, 157–170. https://www.usenix.org/conference/nsdi18/presentation/kumar
[28]
He Liu, Feng Lu, Alex Forencich, Rishi Kapoor, Malveeka Tewari, Geoffrey M. Voelker, George Papen, Alex C. Snoeren, and George Porter. 2014. Circuit Switching Under the Radar with REACToR. In 11th USENIX Symposium on Networked Systems Design and Implementation (NSDI 14). USENIX Association, Seattle, WA. 1–15. isbn:978-1-931971-09-6 https://www.usenix.org/conference/nsdi14/technical-sessions/presentation/liu_he
[29]
William M. Mellette, Rajdeep Das, Yibo Guo, Rob McGuinness, Alex C. Snoeren, and George Porter. 2020. Expanding across time to deliver bandwidth efficiency and low latency. In 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI 20). USENIX Association, Santa Clara, CA. 1–18. isbn:978-1-939133-13-7 https://www.usenix.org/conference/nsdi20/presentation/mellette
[30]
Ted Nesson and Lennart Johnsson. 1994. ROMM routing: A class of efficient Minimal routing algorithms. In Parallel Computer Routing and Communication, Kevin Bolding and Lawrence Snyder (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg. 185–199. isbn:978-3-540-48787-6
[31]
Ted Nesson and S. Lennart Johnsson. 1995. ROMM Routing on Mesh and Torus Networks. In Proceedings of the Seventh Annual ACM Symposium on Parallel Algorithms and Architectures (SPAA ’95). Association for Computing Machinery, New York, NY, USA. 275–287. isbn:0897917170 https://doi.org/10.1145/215399.215455
[32]
George Porter, Richard Strong, Nathan Farrington, Alex Forencich, Pang Chen-Sun, Tajana Rosing, Yeshaiahu Fainman, George Papen, and Amin Vahdat. 2013. Integrating Microsecond Circuit Switching into the Data Center. In Proceedings of the ACM SIGCOMM 2013 Conference on SIGCOMM (SIGCOMM ’13). Association for Computing Machinery, New York, NY, USA. 447–458. isbn:9781450320566 https://doi.org/10.1145/2486001.2486007
[33]
H. Räcke. 2002. Minimizing congestion in general networks. In The 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002. Proceedings. 43–52. https://doi.org/10.1109/SFCS.2002.1181881
[34]
Harald Räcke. 2008. Optimal Hierarchical Decompositions for Congestion Minimization in Networks. STOC ’08. Association for Computing Machinery, New York, NY, USA. 10 pages. isbn:9781605580470 https://doi.org/10.1145/1374376.1374415
[35]
Vishal Shrivastav, Asaf Valadarsky, Hitesh Ballani, Paolo Costa, Ki Suh Lee, Han Wang, Rachit Agarwal, and Hakim Weatherspoon. 2019. Shoal: A Network Architecture for Disaggregated Racks. In 16th USENIX Symposium on Networked Systems Design and Implementation (NSDI 19). USENIX Association, Boston, MA. https://www.usenix.org/conference/nsdi19/presentation/shrivastav
[36]
Valiant. 1983. Optimality of a Two-Phase Strategy for Routing in Interconnection Networks. IEEE Trans. Comput., C-32, 9 (1983), 861–863. https://doi.org/10.1109/TC.1983.1676335
[37]
Leslie G. Valiant. 1982. A Scheme for Fast Parallel Communication. SIAM J. Comput., 11, 2 (1982), 350–361. https://doi.org/10.1137/0211027
[38]
Leslie G. Valiant and Gordon J. Brebner. 1981. Universal Schemes for Parallel Communication. 263–277. https://doi.org/10.1145/800076.802479
[39]
Guohui Wang, David G. Andersen, Michael Kaminsky, Konstantina Papagiannaki, T.S. Eugene Ng, Michael Kozuch, and Michael Ryan. 2010. C-Through: Part-Time Optics in Data Centers. In Proceedings of the ACM SIGCOMM 2010 Conference (SIGCOMM ’10). Association for Computing Machinery, New York, NY, USA. 327–338. isbn:9781450302012 https://doi.org/10.1145/1851182.1851222
[40]
Tegan Wilson, Daniel Amir, Vishal Shrivastav, Hakim Weatherspoon, and Robert Kleinberg. 2023. Extending Optimal Oblivious Reconfigurable Networks to All N. In Proceedings of the SIAM Symposium on Algorithmic Principles of Computer Systems (APOCS).
[41]
Rui Zhang-Shen and Nick McKeown. 2005. Designing a predictable internet backbone with valiant load-balancing. In Quality of Service–IWQoS 2005: 13th International Workshop, IWQoS 2005, Passau, Germany, June 21-23, 2005. Proceedings 13. 178–192.
[42]
Xia Zhou, Zengbin Zhang, Yibo Zhu, Yubo Li, Saipriya Kumar, Amin Vahdat, Ben Y. Zhao, and Haitao Zheng. 2012. Mirror Mirror on the Ceiling: Flexible Wireless Links for Data Centers. In Proceedings of the ACM SIGCOMM 2012 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication (SIGCOMM ’12). Association for Computing Machinery, New York, NY, USA. 443–454. isbn:9781450314190 https://doi.org/10.1145/2342356.2342440
[43]
Goran Zuzic, Bernhard Haeupler, and Antti Roeyskoe. 2023. Sparse Semi-Oblivious Routing: Few Random Paths Suffice. In Proceedings of the 2023 ACM Symposium on Principles of Distributed Computing (PODC ’23). Association for Computing Machinery, New York, NY, USA. 222–232. isbn:9798400701214 https://doi.org/10.1145/3583668.3594585

Cited By

View all
  • (2024)Semi-Oblivious Reconfigurable Datacenter NetworksProceedings of the 23rd ACM Workshop on Hot Topics in Networks10.1145/3696348.3696860(150-158)Online publication date: 18-Nov-2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
STOC 2024: Proceedings of the 56th Annual ACM Symposium on Theory of Computing
June 2024
2049 pages
ISBN:9798400703836
DOI:10.1145/3618260
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 June 2024

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Oblivious routing
  2. reconfigurable networks
  3. tail inequalities
  4. valiant load balancing

Qualifiers

  • Research-article

Funding Sources

  • NSF (National Science Foundation)
  • Google
  • Cisco Systems
  • Microsoft Research

Conference

STOC '24
Sponsor:
STOC '24: 56th Annual ACM Symposium on Theory of Computing
June 24 - 28, 2024
BC, Vancouver, Canada

Acceptance Rates

Overall Acceptance Rate 1,469 of 4,586 submissions, 32%

Upcoming Conference

STOC '25
57th Annual ACM Symposium on Theory of Computing (STOC 2025)
June 23 - 27, 2025
Prague , Czech Republic

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)91
  • Downloads (Last 6 weeks)10
Reflects downloads up to 19 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Semi-Oblivious Reconfigurable Datacenter NetworksProceedings of the 23rd ACM Workshop on Hot Topics in Networks10.1145/3696348.3696860(150-158)Online publication date: 18-Nov-2024

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media