research-article

Free access

Improving datacenter performance and robustness with multipath TCP

Authors:

Mark HandleyAuthors Info & Claims

SIGCOMM '11: Proceedings of the ACM SIGCOMM 2011 conference

Pages 266 - 277

https://doi.org/10.1145/2018436.2018467

Published: 15 August 2011 Publication History

PDF eReader

Abstract

The latest large-scale data centers offer higher aggregate bandwidth and robustness by creating multiple paths in the core of the net- work. To utilize this bandwidth requires different flows take different paths, which poses a challenge. In short, a single-path transport seems ill-suited to such networks.

We propose using Multipath TCP as a replacement for TCP in such data centers, as it can effectively and seamlessly use available bandwidth, giving improved throughput and better fairness on many topologies. We investigate what causes these benefits, teasing apart the contribution of each of the mechanisms used by MPTCP.

Using MPTCP lets us rethink data center networks, with a different mindset as to the relationship between transport protocols, rout- ing and topology. MPTCP enables topologies that single path TCP cannot utilize. As a proof-of-concept, we present a dual-homed variant of the FatTree topology. With MPTCP, this outperforms FatTree for a wide range of workloads, but costs the same.

In existing data centers, MPTCP is readily deployable leveraging widely deployed technologies such as ECMP. We have run MPTCP on Amazon EC2 and found that it outperforms TCP by a factor of three when there is path diversity. But the biggest benefits will come when data centers are designed for multipath transports.

Supplementary Material

MP4 File (sigcomm_8_3.mp4)

Download
109.85 MB

References

[1]

M. Al-Fares, A. Loukissas, and A. Vahdat. A scalable, commodity data center network architecture. In Proc. SIGCOMM 2010.

Digital Library

Google Scholar

[2]

M. Al-Fares, S. Radhakrishnan, B. Raghavan, N. Huang, and A. Vahdat. Hedera: Dynamic flow scheduling for data center networks. In Proc. Usenix NSDI 2010.

Digital Library

Google Scholar

[3]

C. Clos. A study of non-blocking switching networks. Bell System Technical Journal, 32(5):406--424, 1952.

Google Scholar

[4]

A. Ford, C. Raiciu, M. Handley, and O. Bonaventure. TCP Extensions for Multipath Operation with Multiple Addresses. Internet-draft, IETF, 2011.

Google Scholar

[5]

P. Geoffray and T. Hoefler. Adaptive routing strategies for modern high performance networks. In Proceedings of the 2008 16th IEEE Symposium on High Performance Interconnects, pages 165--172, Washington, DC, USA, 2008. IEEE Computer Society.

Digital Library

Google Scholar

[6]

A. Greenberg el al. VL2: a scalable and flexible data center network. In Proc. ACM Sigcomm 2009.

Digital Library

Google Scholar

[7]

C. Guo, G. Lu, D. Li, H. Wu, X. Zhang, Y. Shi, C. Tian, Y. Zhang, and S. Lu. Bcube: a high performance, server-centric network architecture for modular data centers. In Proc. SIGCOMM 2009.

Digital Library

Google Scholar

[8]

K. Holmberg. Optimization models for routing in switching networks of clos type with many stages. AMO - Advanced Modeling and Optimization, 10(1), 2008.

Google Scholar

[9]

J. Mudigonda, P. Yalagandula, M. Al-Fares, and J. C. Mogul. Spain: Cots data-center ethernet for multipathing over arbitrary topologies. In Proc. NSDI 2010.

Digital Library

Google Scholar

[10]

C. Raiciu, M. Handley, and A. Ford. Multipath TCP design decisions. Work in progress, www.cs.ucl.ac.uk/staff/C.Raiciu/files/mtcp-design.pdf, 2009.

Google Scholar

[11]

C. Raiciu, M. Handley, and D. Wischik. Coupled Congestion Control for Multipath Transport Protocols. Internet-draft, IETF, 2011.

Google Scholar

[12]

A. Smiljanic. Rate and delay guarantees provided by clos packet switches with load balancing. IEEE/ACM Trans. Netw., 16:170--181, February 2008.

Digital Library

Google Scholar

[13]

D. Wischik, C. Raiciu, A. Greenhalgh, and M. Handley. Design, implementation and evaluation of congestion control for multipath TCP. In Proc. Usenix NSDI 2011.

Digital Library

Google Scholar

[14]

E. Oki, Z. Jing, R. Rojas-Cessa, and H. J. Chao. Concurrent round-robin-based dispatching schemes for clos-network switches. IEEE/ACM Trans. Netw., 10:830--844, December 2002.

Digital Library

Google Scholar

Cited By

View all

Blach NBesta MDe Sensi DDomke JHarake HLi SIff PKonieczny MLakhotia KKubicek AFerrari MPetrini FHoefler TVanbever LZhang I(2024)A high-performance design, implementation, deployment, and evaluation of the slim fly networkProceedings of the 21st USENIX Symposium on Networked Systems Design and Implementation10.5555/3691825.3691882(1025-1044)Online publication date: 16-Apr-2024
https://dl.acm.org/doi/10.5555/3691825.3691882
Xue SWu CHan JZhan A(2024)PDASTSGAT: An STSGAT-Based Multipath Data Scheduling AlgorithmAlgorithms10.3390/a1704014517:4(145)Online publication date: 30-Mar-2024
https://doi.org/10.3390/a17040145
Basu PZhao LFantl JPal SKrishnamurthy AKhoury JMencagli GDazzi PLowenthal DBadia R(2024)Efficient all-to-all Collective Communication Schedules for Direct-connect TopologiesProceedings of the 33rd International Symposium on High-Performance Parallel and Distributed Computing10.1145/3625549.3658656(28-41)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3625549.3658656
Show More Cited By

Index Terms

Improving datacenter performance and robustness with multipath TCP
1. Networks
  1. Network protocols

Recommendations

Improving datacenter performance and robustness with multipath TCP
SIGCOMM '11

The latest large-scale data centers offer higher aggregate bandwidth and robustness by creating multiple paths in the core of the net- work. To utilize this bandwidth requires different flows take different paths, which poses a challenge. In short, a ...
An enhanced socket API for Multipath TCP
ANRW '16: Proceedings of the 2016 Applied Networking Research Workshop

Multipath TCP is a TCP extension that enables hosts to send data belonging to a single TCP connection over different paths. It was designed as an incrementally deployable evolution of TCP. For this reason, the Multipath TCP specification assumes that ...
Evaluating TCP-friendliness in light of Concurrent Multipath Transfer

In prior work, a CMT protocol using SCTP multihoming (termed SCTP-based CMT) was proposed and investigated for improving application throughput. SCTP-based CMT was studied in (bottleneck-independent) wired networking scenarios with ns-2 simulations. ...

Comments

Information & Contributors

Information

Published In

SIGCOMM '11: Proceedings of the ACM SIGCOMM 2011 conference

August 2011

502 pages

ISBN:9781450307970

DOI:10.1145/2018436

General Chairs:
Srinivasan Keshav
University of Waterloo, Canada
,
Jörg Liebeherr
University of Toronto, Canada
,
Program Chairs:
John Byers
Boston University, USA
,
Jeffrey Mogul
HP Labs, USA

ACM SIGCOMM Computer Communication Review Volume 41, Issue 4
SIGCOMM '11
August 2011
480 pages
ISSN:0146-4833
DOI:10.1145/2043164
Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 August 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGCOMM '11

Sponsor:

SIGCOMM

SIGCOMM '11: ACM SIGCOMM 2011 Conference

August 15 - 19, 2011

Ontario, Toronto, Canada

Acceptance Rates

SIGCOMM '11 Paper Acceptance Rate 32 of 223 submissions, 14%;

Overall Acceptance Rate 462 of 3,389 submissions, 14%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

603
Total Citations
View Citations
4,773
Total Downloads

Downloads (Last 12 months)472
Downloads (Last 6 weeks)49

Reflects downloads up to 18 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Blach NBesta MDe Sensi DDomke JHarake HLi SIff PKonieczny MLakhotia KKubicek AFerrari MPetrini FHoefler TVanbever LZhang I(2024)A high-performance design, implementation, deployment, and evaluation of the slim fly networkProceedings of the 21st USENIX Symposium on Networked Systems Design and Implementation10.5555/3691825.3691882(1025-1044)Online publication date: 16-Apr-2024
https://dl.acm.org/doi/10.5555/3691825.3691882
Xue SWu CHan JZhan A(2024)PDASTSGAT: An STSGAT-Based Multipath Data Scheduling AlgorithmAlgorithms10.3390/a1704014517:4(145)Online publication date: 30-Mar-2024
https://doi.org/10.3390/a17040145
Basu PZhao LFantl JPal SKrishnamurthy AKhoury JMencagli GDazzi PLowenthal DBadia R(2024)Efficient all-to-all Collective Communication Schedules for Direct-connect TopologiesProceedings of the 33rd International Symposium on High-Performance Parallel and Distributed Computing10.1145/3625549.3658656(28-41)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3625549.3658656
Zhuang RHan JXue KLi JSun QLu J(2024)ProactMP: A Proactive Multipath Transport Protocol for Low-Latency DatacentersIEEE Transactions on Network and Service Management10.1109/TNSM.2024.339902821:4(3919-3932)Online publication date: Aug-2024
https://doi.org/10.1109/TNSM.2024.3399028
Hu JHe YLuo WHuang JWang J(2024)Enhancing Load Balancing With In-Network Recirculation to Prevent Packet Reordering in Lossless Data CentersIEEE/ACM Transactions on Networking10.1109/TNET.2024.340367132:5(4114-4127)Online publication date: Oct-2024
https://doi.org/10.1109/TNET.2024.3403671
Champagne LDonnet B(2024)Smoothie: Efficient and Flexible Load-Balancing in Data CenterNOMS 2024-2024 IEEE Network Operations and Management Symposium10.1109/NOMS59830.2024.10575467(1-7)Online publication date: 6-May-2024
https://doi.org/10.1109/NOMS59830.2024.10575467
Shalev LAyoub HBshara NFatael YGolan OIlany OLevin AMachulsky ZMilczewski KOlson MPriescu VRajagopal SSaidi A(2024)The Tail at Amazon Web Services ScaleIEEE Micro10.1109/MM.2024.342007044:5(23-29)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1109/MM.2024.3420070
Hu JHe YLuo WHuang JWang JWang J(2024)TaLB: Tensor-aware Load Balancing for Distributed DNN Training Acceleration2024 IEEE/ACM 32nd International Symposium on Quality of Service (IWQoS)10.1109/IWQoS61813.2024.10682910(1-10)Online publication date: 19-Jun-2024
https://doi.org/10.1109/IWQoS61813.2024.10682910
Hu JHe YWang JLuo WHuang J(2023)RLB: Reordering-Robust Load Balancing in Lossless Datacenter NetworksProceedings of the 52nd International Conference on Parallel Processing10.1145/3605573.3605617(576-584)Online publication date: 7-Aug-2023
https://dl.acm.org/doi/10.1145/3605573.3605617
Dai WFoerster KFuchssteiner DSchmid S(2023)Load-optimization in Reconfigurable Data-center Networks: Algorithms and Complexity of Flow RoutingACM Transactions on Modeling and Performance Evaluation of Computing Systems10.1145/35972008:3(1-30)Online publication date: 18-Jul-2023
https://dl.acm.org/doi/10.1145/3597200
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Improving datacenter performance and robustness with multipath TCP

An enhanced socket API for Multipath TCP

Evaluating TCP-friendliness in light of Concurrent Multipath Transfer