skip to main content
10.1145/3544216.3544222acmconferencesArticle/Chapter ViewAbstractPublication PagescommConference Proceedingsconference-collections
research-article
Open access

Continuous in-network round-trip time monitoring

Published: 22 August 2022 Publication History

Abstract

Round-trip time (RTT) is a central metric that influences end-user QoE and can expose traffic-interception attacks. Many popular RTT monitoring techniques either send active probes (that do not capture application-level RTTs) or passively monitor only the TCP handshake (which can be inaccurate, especially for long-lived flows). High-speed programmable switches present a unique opportunity to monitor the RTTs continuously and react in real time to improve performance and security. In this paper, we present Dart, an inline, real-time, and continuous RTT measurement system that can enable automated detection of network events and adapt (e.g., routing, scheduling, marking, or dropping traffic) inside the network. However, designing Dart is fraught with challenges, due to the idiosyncrasies of the TCP protocol and the resource constraints in high-speed switches. Dart overcomes these challenges by strategically limiting the tracking of packets to only those that can generate useful RTT samples, and by identifying the synergy between per-flow state and per-packet state for efficient memory use. We present a P4 prototype of Dart for the Tofino switch, as well our experiments on a campus testbed and simulations using anonymized campus traces. Dart, running in real time and with limited data-plane memory, is able to collect 99% of the RTT samples of an offline, software baseline---a variant of the popular tcptrace tool that has access to unlimited memory.

Supplementary Material

PDF File (p473-sengupta-supp.pdf)
Supplemental material.

References

[1]
2021. NVIDIA Mellanox NIC's Performance Report with DPDK 21.05. http://fast.dpdk.org/doc/perf/DPDK_21_05_Mellanox_NIC_performance_report.pdf (2021).
[2]
Anurag Agrawal and Changhoon Kim. 2020. Intel Tofino2: A 12.9 Tbps P4-Programmable Ethernet Switch. In IEEE Hot Chips Symposium (HCS). IEEE Computer Society, 1--32.
[3]
Aditya Akella, Jeffrey Pang, Bruce Maggs, Srinivasan Seshan, and Anees Shaikh. 2004. A comparison of overlay routing and multihoming route control. ACM SIGCOMM Computer Communication Review 34, 4 (2004), 93--106.
[4]
Maria Apostolaki, Ankit Singla, and Laurent Vanbever. 2021. Performance-Driven Internet Path Selection. In ACM SIGCOMM Symposium on SDN Research (SOSR). 41--53.
[5]
Axel Arnbak and Sharon Goldberg. 2014. Loopholes for circumventing the constitution: Unrestricted bulk surveillance on americans by collecting network traffic abroad. Michigan Telecommunications and Technology Law Review 21 (2014), 317.
[6]
Debopam Bhattacherjee, Muhammad Tirmazi, and Ankit Singla. 2017. A cloud-based content gathering network. In USENIX Workshop on Hot Topics in Cloud Computing.
[7]
Henry Birge-Lee, Liang Wang, Jennifer Rexford, and Prateek Mittal. 2019. Sico: Surgical interception attacks by manipulating BGP communities. In ACM SIGSAC Conference on Computer and Communications Security. 431--448.
[8]
D. Borman, B. Braden, V. Jacobson, and R. Scheffenegger. 2014. TCP Extensions for High Performance. RFC 7323. RFC Editor.
[9]
Pat Bosshart, Dan Daly, Glen Gibb, Martin Izzard, Nick McKeown, Jennifer Rexford, Cole Schlesinger, Dan Talayco, Amin Vahdat, George Varghese, and David Walker. 2014. P4: Programming protocol-independent packet processors. ACM SIGCOMM Computer Communication Review 44, 3 (2014), 87--95.
[10]
Francesco Bronzino, Paul Schmitt, Sara Ayoubi, Guilherme Martins, Renata Teixeira, and Nick Feamster. 2019. Inferring streaming video quality from encrypted traffic: Practical models and deployment experience. In ACM SIGMETRICS. 1--25.
[11]
Kuan-Ta Chen, Yu-Chun Chang, Po-Han Tseng, Chun-Ying Huang, and Chin-Laung Lei. 2011. Measuring the latency of cloud gaming systems. In Proceedings of the 19th ACM international conference on Multimedia. 1269--1272.
[12]
Xiaoqi Chen, Hyojoon Kim, Javed M Aman, Willie Chang, Mack Lee, and Jennifer Rexford. 2020. Measuring TCP round-trip time in the data plane. In ACM SIGCOMM Workshop on Secure Programmable Network Infrastructure. 35--41.
[13]
Yunhua Deng, Yusen Li, Xueyan Tang, and Wentong Cai. 2016. Server allocation for multiplayer cloud gaming. In Proceedings of the 24th ACM international conference on Multimedia. 918--927.
[14]
Hao Ding and Michael Rabinovich. 2015. TCP stretch acknowledgements and timestamps: Findings and implications for passive RTT measurement. ACM SIGCOMM Computer Communication Review 45, 3 (2015), 20--27.
[15]
Jon Dugan, Seth Elliott, Bruce A Mah, Jeff Poskanzer, and Kaustubh Prabhu. 2014. iperf3, tool for active measurements of the maximum achievable bandwidth on IP networks. (2014). https://github.com/esnet/iperf.
[16]
Mojgan Ghasemi, Theophilus Benson, and Jennifer Rexford. 2017. Dapper: Data plane performance diagnosis of TCP. In ACM SIGCOMM Symposium on SDN Research (SOSR). ACM, 61--74.
[17]
Nicholas Hopper, Eugene Y Vasserman, and Eric Chan-Tin. 2010. How much anonymity does network latency leak? ACM Transactions on Information and System Security (TISSEC) 13, 2 (2010), 1--28.
[18]
Hao Jiang and Constantinos Dovrolis. 2002. Passive estimation of TCP round-trip times. ACM SIGCOMM Computer Communication Review 32, 3 (2002), 75--88.
[19]
Daehyeok Kim, Zaoxing Liu, Yibo Zhu, Changhoon Kim, Jeongkeun Lee, Vyas Sekar, and Srinivasan Seshan. 2020. TEA: Enabling state-intensive network functions on programmable switches. In ACM SIGCOMM. 90--106.
[20]
Adam Langley, Alistair Riddoch, Alyssa Wilk, Antonio Vicente, Charles Krasic, Dan Zhang, Fan Yang, Fedor Kouranov, Ian Swett, Janardhan Iyengar, et al. 2017. The QUIC transport protocol: Design and internet-scale deployment. In ACM SIGCOMM. 183--196.
[21]
Changhyun Lee, Chunjong Park, Keon Jang, Sue Moon, and Dongsu Han. 2015. Accurate latency-based congestion feedback for datacenters. In USENIX Annual Technical Conference (ATC). 403--415.
[22]
Sanghwan Lee, Zhi-Li Zhang, and Srihari Nelakuditi. 2004. Exploiting as hierarchy for scalable route selection in multi-homed stub networks. In ACM Internet Measurement Conference. 294--299.
[23]
Zaoxing Liu, Samson Zhou, Ori Rottenstreich, Vladimir Braverman, and Jennifer Rexford. 2020. Memory-efficient performance monitoring on programmable switches with lean algorithms. In Symposium on Algorithmic Principles of Computer Systems (APoCS). SIAM, 31--44.
[24]
Giovane CM Moura, John Heidemann, Wes Hardaker, Pithayuth Charnsethikul, Jeroen Bulten, Joao Ceron, and Cristian Hesselman. 2022. Old but Gold: Prospecting TCP to Engineer and Real-time Monitor DNS Anycast. In Passive and Active Measurement Conference.
[25]
RIPE NCC. 2021. RIPE Atlas. https://atlas.ripe.net/. (2021).
[26]
Kathleen Nichols. 2017. pping (Pollere passive ping). https://github.com/pollere/pping. (2017).
[27]
Shawn Ostermann. 2007. tcptrace Homepage. http://www.tcptrace.org/ (2007).
[28]
Stefan Savage, Neal Cardwell, David Wetherall, and Tom Anderson. 1999. TCP congestion control with a misbehaving receiver. ACM SIGCOMM Computer Communication Review 29, 5 (1999), 71--78.
[29]
Brandon Schlinker, Todd Arnold, Italo Cunha, and Ethan Katz-Bassett. 2019. PEERING: Virtualizing BGP at the Edge for Research. In ACM SIGCOMM International Conference on Emerging Networking Experiments And Technologies. 51--67.
[30]
Satadal Sengupta, Hyojoon Kim, and Jennifer Rexford. 2021. Fine-Grained RTT Monitoring Inside the Network. Measuring Network Quality for End-Users (2021).
[31]
Yixin Sun, Anne Edmundson, Laurent Vanbever, Oscar Li, Jennifer Rexford, Mung Chiang, and Prateek Mittal. 2015. RAPTOR: Routing attacks on privacy in Tor. In USENIX Security Symposium. 271--286.
[32]
Yufei Zheng, Xiaoqi Chen, Mark Braverman, and Jennifer Rexford. 2022. Unbiased Delay Measurement in the Data Plane. In Symposium on Algorithmic Principles of Computer Systems (APoCS). SIAM, 15--30.

Cited By

View all
  • (2025)Compact Data Structures for Network TelemetryACM Computing Surveys10.1145/3716819Online publication date: 12-Feb-2025
  • (2024)Adaptive Address Family Selection for Latency-Sensitive Applications on Dual-Stack HostsACM SIGCOMM Computer Communication Review10.1145/3717512.371751454:4(2-13)Online publication date: 30-Oct-2024
  • (2024)NetGSR: Towards Efficient and Reliable Network Monitoring with Generative Super ResolutionProceedings of the ACM on Networking10.1145/36964002:CoNEXT4(1-27)Online publication date: 25-Nov-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SIGCOMM '22: Proceedings of the ACM SIGCOMM 2022 Conference
August 2022
858 pages
ISBN:9781450394208
DOI:10.1145/3544216
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 August 2022

Permissions

Request permissions for this article.

Check for updates

Badges

Author Tags

  1. highspeed programmable switch
  2. network monitoring
  3. passive measurement
  4. round-trip time

Qualifiers

  • Research-article

Funding Sources

Conference

SIGCOMM '22
Sponsor:
SIGCOMM '22: ACM SIGCOMM 2022 Conference
August 22 - 26, 2022
Amsterdam, Netherlands

Acceptance Rates

Overall Acceptance Rate 462 of 3,389 submissions, 14%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)1,013
  • Downloads (Last 6 weeks)91
Reflects downloads up to 14 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Compact Data Structures for Network TelemetryACM Computing Surveys10.1145/3716819Online publication date: 12-Feb-2025
  • (2024)Adaptive Address Family Selection for Latency-Sensitive Applications on Dual-Stack HostsACM SIGCOMM Computer Communication Review10.1145/3717512.371751454:4(2-13)Online publication date: 30-Oct-2024
  • (2024)NetGSR: Towards Efficient and Reliable Network Monitoring with Generative Super ResolutionProceedings of the ACM on Networking10.1145/36964002:CoNEXT4(1-27)Online publication date: 25-Nov-2024
  • (2024)SetD4: Sets With Deletions and Decay in the Data PlaneProceedings of the ACM on Networking10.1145/36963912:CoNEXT4(1-22)Online publication date: 25-Nov-2024
  • (2024)Zoom2Net: Constrained Network Telemetry ImputationProceedings of the ACM SIGCOMM 2024 Conference10.1145/3651890.3672225(764-777)Online publication date: 4-Aug-2024
  • (2024)Keeping an Eye on Congestion Control in the Wild with NebbyProceedings of the ACM SIGCOMM 2024 Conference10.1145/3651890.3672223(136-150)Online publication date: 4-Aug-2024
  • (2024)Taming the Elephants: Affordable Flow Length Prediction in the Data PlaneProceedings of the ACM on Networking10.1145/36494732:CoNEXT1(1-24)Online publication date: 28-Mar-2024
  • (2024)Measuring Network Latency from a Wireless ISP: Variations Within and Across SubnetsProceedings of the 2024 ACM on Internet Measurement Conference10.1145/3646547.3688438(29-43)Online publication date: 4-Nov-2024
  • (2024)DeviceRadar: Online IoT Device Fingerprinting in ISPs Using Programmable SwitchesIEEE/ACM Transactions on Networking10.1109/TNET.2024.339877832:5(3854-3869)Online publication date: Oct-2024
  • (2024)SpinTrap: Catching Speeding QUIC FlowsNOMS 2024-2024 IEEE Network Operations and Management Symposium10.1109/NOMS59830.2024.10575719(1-10)Online publication date: 6-May-2024
  • Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media