research-article

Open access

Keeping an Eye on Congestion Control in the Wild with Nebby

Authors:

Lakshay Rastogi,

Ben LeongAuthors Info & Claims

ACM SIGCOMM '24: Proceedings of the ACM SIGCOMM 2024 Conference

Pages 136 - 150

https://doi.org/10.1145/3651890.3672223

Published: 04 August 2024 Publication History

Abstract

The Internet congestion control landscape is rapidly evolving. Since the introduction of BBR and the deployment of QUIC, it has become increasingly commonplace for companies to modify and implement their own congestion control algorithms (CCAs). To respond effectively to these developments, it is crucial to understand the state of CCA deployments in the wild. Unfortunately, existing CCA identification tools are not future-proof and do not work well with modern CCAs and encrypted protocols like QUIC. In this paper, we articulate the challenges in designing a future-proof CCA identification tool and propose a measurement methodology that directly addresses these challenges. The resulting measurement tool, called Nebby, can identify all the CCAs currently available in the Linux kernel and BBRv2 with an average accuracy of 96.7%. We found that among the Alexa Top 20k websites, the share of BBR has shrunk since 2019 and that only 8% of them responded to QUIC requests. Among these QUIC servers, CUBIC and BBR seem equally popular. We show that Nebby is extensible by extending it for Copa and an undocumented family of CCAs that is deployed by 6% of the measured websites, including major corporations like Hulu and Apple.

References

[1]

2022. Alibaba's QUIC implementation, xquic. (2022). https://github.com/alibaba/xquic

[2]

2022. Amazon Web Services's QUIC implementation, s2n-quic. (2022). https://github.com/aws/s2n-quic

[3]

2022. Cloudflare's QUIC implementation, quiche. (2022). https://github.com/cloudflare/quiche

[4]

2022. Facebook's QUIC implementation, mvfst. (2022). https://github.com/facebookincubator/mvfst

[5]

2022. Google's QUIC implementation, chromium. (2022). https://www.chromium.org/quic/playing-with-quic

[6]

2022. Go's QUIC implementation, quic-go. (2022). https://github.com/lucas-clemente/quic-go

[7]

2022. H2O's QUIC implementation, quicly. (2022). https://github.com/h2o/quicly

[8]

2022. LiteSpeed's QUIC implementation, lsquic. (2022). https://github.com/litespeedtech/lsquic

[9]

2022. Microsoft's QUIC implementation, msquic. (2022). https://github.com/microsoft/msquic

[10]

2022. Mozilla's QUIC implementation, neqo. (2022). https://github.com/mozilla/neqo

[11]

2022. Rust's QUIC implementation, quinn. (2022). https://github.com/quinn-rs/quinn

[12]

2023. Alexa Top Websites - Last Save. (2023). https://www.expireddomains.net/alexa-top-websites/

[13]

2023. BBRv3: Algorithm Bug Fixes and Public Internet Deployment. (2023). http://tinyurl.com/bbrv3ietf

[14]

2024. Nebby. (2024). www.github.com/NUS-SNL/Nebby

[15]

2024. Selenium. (2024). https://www.selenium.dev

[16]

Guido Appenzeller, Isaac Keslassy, and Nick McKeown. 2004. Sizing router buffers. ACM SIGCOMM CCR 34, 4 (2004).

[17]

Venkat Arun and Hari Balakrishnan. 2018. Copa: Practical Delay-Based Congestion Control for the Internet. In Proceedings of NSDI.

Digital Library

[18]

Andrea Baiocchi, Angelo P Castellani, and Francesco Vacirca. 2007. YeAH-TCP: yet another highspeed TCP. In Proceedings of PFLDnet.

[19]

Lawrence S Brakmo, Sean W O'Malley, and Larry L Peterson. 1994. TCP Vegas: New techniques for congestion detection and avoidance. In Proceedings of SIGCOMM.

Digital Library

[20]

Neal Cardwell, Yuchung Cheng, C. Stephen Gunn, Soheil Hassas Yeganeh, and Van Jacobson. 2017. BBR: Congestion-based Congestion Control. CACM 60, 2 (2017), 58--66.

Digital Library

[21]

Neal Cardwell, Yuchung Cheng, Soheil Hassas Yeganeh, and Van Jacobson. 2017. BBR Congestion Control. IETF Draft. (2017). https://datatracker.ietf.org/doc/html/draft-cardwell-iccrg-bbr-congestion-control-00

Digital Library

[22]

Neal Cardwell, Yuchung Cheng, Soheil Hassas Yeganeh, Ian Swett, Victor Vasiliev, Priyaranjan Jha, Yousuk Seung, Matt Mathis, and Van Jacobson. 2019. BBR v2 - A Model-based Congestion Control. ICCRG at IETF 104. (2019). https://bit.ly/2HgGOuQ

[23]

Claudio Casetti, Mario Gerla, Saverio Mascolo, Medy Y Sanadidi, and Ren Wang. 2002. TCP Westwood: end-to-end congestion control for wired/wireless networks. Wireless Networks 8, 5 (2002), 467--479.

Digital Library

[24]

Xiaoyu Chen, Shugong Xu, Xudong Chen, Shan Cao, Shunqing Zhang, and Yanzan Sun. 2019. Passive TCP identification for wired and wireless networks: A long-short term memory approach. In Proceedings of IWCMC.

[25]

Mo Dong, Tong Meng, Doron Zarchy, Engin Arslan, Yossi Gilad, Brighten Godfrey, and Michael Schapira. 2018. PCC Vivace: Online-Learning Congestion Control. In Proceedings of NSDI.

[26]

Eirini Kakogianni Erik Carlsson. 2018. Smoother Streaming with BBR. (2018). https://engineering.atspotify.com/2018/08/smoother-streaming-with-bbr/

[27]

Margarida Ferreira, Akshay Narayan, Inês Lynce, Ruben Martins, and Justine Sherry. 2021. Counterfeiting Congestion Control Algorithms. In Proceedings of Hotnets.

Digital Library

[28]

Sally Floyd. 2003. HighSpeed TCP for Large Congestion Windows. RFC 3649. (2003).

[29]

Cheng Peng Fu and S. C. Liew. 2006. TCP Veno: TCP Enhancement for Transmission over Wireless Access Networks. IEEE JSAC 21, 2 (2006), 216--228.

Digital Library

[30]

Nitin Garg. 2019. Engineering at Meta: Evaluating COPA congestion control for improved video performance. (2019). https://engineering.fb.com/2019/11/17/video-engineering/copa/

[31]

Sishuai Gong, Usama Naseer, and Theophilus A Benson. 2020. Inspector Gadget: A Framework for Inferring TCP Congestion Control Algorithms and Protocol Configurations. In Network Traffic Measurement and Analysis Conference.

[32]

Google. 2023. Personal Correspondance. (2023).

[33]

Google Cloud Blogs 2017. TCP BBR congestion control comes to GCP: your Internet just got faster. (2017). https://bit.ly/2Hk4WLH

[34]

Sangtae Ha, Injong Rhee, and Lisong Xu. 2008. CUBIC: A New TCP-friendly High-speed TCP Variant. SIGOPS Operating Systems Review 42, 5 (2008), 64--74.

Digital Library

[35]

Desta Haileselassie Hagos, Paal E Engelstad, Anis Yazidi, and Øivind Kure. 2018. General TCP state inference model from passive measurements using machine learning techniques. IEEE Access 6 (2018), 28372--28387.

[36]

T. Henderson, S. Floyd, A. Gurtov, and Y. Nishida. 2012. The NewReno Modification to TCP's Fast Recovery Algorithm. (2012). https://tools.ietf.org/html/rfc6582

[37]

Mario Hock, Roland Bless, and Martina Zitterbart. 2017. Experimental Evaluation of BBR Congestion Control. In Proceedings of ICNP.

[38]

Alexa Internet Inc. 2023. The Top 500 websites on the Internet. (2023). https://www.alexa.com/topsites

[39]

Jana Iyengar and Martin Thomson. 2021. QUIC: A UDP-Based Multiplexed and Secure Transport, RFC 9000. (2021). https://datatracker.ietf.org/doc/html/rfc9000

Digital Library

[40]

Van Jacobson. 1988. Congestion Avoidance and Control. ACM SIGCOMM Computer Communication Review 18, 4 (1988).

Digital Library

[41]

Sharad Jaiswal, Gianluca Iannaccone, Christophe Diot, Jim Kurose, and Don Towsley. 2004. Inferring TCP connection characteristics through passive measurements. In Proceedings of INFOCOM.

[42]

Matt Joras and Yang Chi. 2020. How Facebook is bringing QUIC to billions. (2020). https://engineering.fb.com/2020/10/21/networking-traffic/how-facebook-is-bringing-quic-to-billions/

[43]

Tom Kelly. 2003. Scalable TCP: Improving Performance in Highspeed Wide Area Networks. SIGCOMM CCR 33, 2 (2003), 83--91.

Digital Library

[44]

Douglas Leith, R Shorten, and Y Lee. 2005. H-TCP: A framework for congestion control in high-speed and long-distance networks. In Proceedings of PFLDnet.

[45]

Shao Liu, Tamer Başar, and R. Srikant. 2006. TCP-Illinois: A Loss and Delay-based Congestion Control Algorithm for High-speed Networks. In Proceedings of VALUETOOLS.

[46]

Alberto Medina, Mark Allman, and Sally Floyd. 2005. Measuring the Evolution of Transport Protocols in the Internet. SIGCOMM CCR 35, 2 (2005), 37--52.

Digital Library

[47]

Ayush Mishra and Ben Leong. 2023. Containing the Cambrian Explosion in QUIC Congestion Control. In Proceedings of IMC.

Digital Library

[48]

Ayush Mishra, Sherman Lim, and Ben Leong. 2022. Understanding Speciation in QUIC Congestion Control. In Proceedings of IMC.

Digital Library

[49]

Ayush Mishra, Xiangpeng Sun, Atishya Jain, Sameer Pande, Raj Joshi, and Ben Leong. 2019. Gordon: Congestion Control Identification Tool. (2019). https://github.com/NUS-SNL/Gordon

[50]

Ayush Mishra, Xiangpeng Sun, Atishya Jain, Sameer Pande, Raj Joshi, and Ben Leong. 2019. The Great Internet TCP Congestion Control Census. In Proceedings of SIGMETRICS.

Digital Library

[51]

Ayush Mishra, Wee Han Tiu, and Ben Leong. 2022. Are we heading towards a BBR-dominant Internet?. In Proceedings of IMC.

Digital Library

[52]

NetFlix. 2023. Personal Correspondance. (2023).

[53]

Ravi Netravali, Anirudh Sivaraman, Somak Das, Ameesh Goyal, Keith Winstein, James Mickens, and Hari Balakrishnan. 2015. Mahimahi: Accurate Record-andReplay for HTTP. In Proceedings of ATC.

[54]

Jitendra Padhye and Sally Floyd. 2001. On Inferring TCP Behavior. In Proceedings of SIGCOMM.

[55]

Vern Paxson and Mark Allman. 2009. TCP Congestion Control. RFC 5681. (2009).

[56]

Canada Sandvine Inc. Waterloo, ON. 2022. The 2022 Global Internet Phenomena Report. (2022). https://www.sandvine.com/phenomena

[57]

Satadal Sengupta, Hyojoon Kim, and Jennifer Rexford. 2022. Continuous innetwork round-trip time monitoring. In Proceedings of SIGCOMM.

Digital Library

[58]

Bruce Spang, Serhat Arslan, and Nick McKeown. 2022. Updating the theory of buffer sizing. In Proceedings of SIGMETRICS.

Digital Library

[59]

Kun Tan, Jingmin Song, Qian Zhang, and Murad Sridharan. 2006. A compound TCP approach for high-speed and long distance networks. In Proceedings of INFOCOM.

[60]

Ranysha Ware, Matthew K Mukerjee, Srinivasan Seshan, and Justine Sherry. 2019. Beyond jain's fairness index: Setting the bar for the deployment of congestion control algorithms. In Proceedings of Hotnets.

Digital Library

[61]

Ranysha Ware, Matthew K. Mukerjee, Srinivasan Seshan, and Justine Sherry. 2019. Modeling BBR's Interactions with Loss-Based Congestion Control. In Proceedings of IMC.

Digital Library

[62]

Lisong Xu, K. Harfoush, and Injong Rhee. 2004. Binary increase congestion control (BIC) for fast long-distance networks. In Proceedings of INFOCOM.

[63]

Peng Yang, Juan Shao, Wen Luo, Lisong Xu, Jitendra Deogun, and Ying Lu. 2011. TCP Congestion Avoidance Algorithm Identification. IEEE/ACM Transactions on Networking 22, 4 (2011), 1311--1324.

Digital Library

Cited By

Mishra AGondaliya HKumaar LBhatnagar AJoshi RLeong BSekar VYu MSeneviratne AVeitch D(2024)POSTER: Expanding the Design Space for in-Network Congestion Control on the InternetProceedings of the ACM SIGCOMM 2024 Conference: Posters and Demos10.1145/3672202.3673733(42-44)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3672202.3673733
Welzl MIslam Svon Stephanides M(2024)Real-Time TCP Packet Loss Prediction Using Machine LearningIEEE Access10.1109/ACCESS.2024.348851112(159622-159634)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3488511

Index Terms

Keeping an Eye on Congestion Control in the Wild with Nebby
1. General and reference
  1. Cross-computing tools and techniques
    1. Measurement
2. Networks
  1. Network protocols
    1. Transport protocols
  2. Network types
    1. Public Internet

Recommendations

The Great Internet TCP Congestion Control Census
SIGMETRICS '20: Abstracts of the 2020 SIGMETRICS/Performance Joint International Conference on Measurement and Modeling of Computer Systems

In 2016, Google proposed and deployed a new TCP variant called BBR. BBR represents a major departure from traditional congestion control as it uses estimates of bandwidth and round-trip delays to regulate its sending rate. BBR has since been introduced ...
The Great Internet TCP Congestion Control Census
SIGMETRICS

In 2016, Google proposed and deployed a new TCP variant called BBR. BBR represents a major departure from traditional congestion-window-based congestion control. Instead of using loss as a congestion signal, BBR uses estimates of the bandwidth and round-...
The Great Internet TCP Congestion Control Census

In 2016, Google proposed and deployed a new TCP variant called BBR. BBR represents a major departure from traditional congestion control as it uses estimates of bandwidth and round-trip delays to regulate its sending rate. BBR has since been introduced ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ACM SIGCOMM '24: Proceedings of the ACM SIGCOMM 2024 Conference

August 2024

1033 pages

ISBN:9798400706141

DOI:10.1145/3651890

Co-chairs:
Aruna Seneviratne,
Darryl Veitch,
Program Co-chairs:
Vyas Sekar,
Minlan Yu

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGCOMM: ACM Special Interest Group on Data Communication

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 August 2024

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Ministry of Education, Singapore

Conference

ACM SIGCOMM '24

Sponsor:

SIGCOMM

ACM SIGCOMM '24: ACM SIGCOMM 2024 Conference

August 4 - 8, 2024

NSW, Sydney, Australia

Acceptance Rates

Overall Acceptance Rate 462 of 3,389 submissions, 14%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
1,379
Total Downloads

Downloads (Last 12 months)1,379
Downloads (Last 6 weeks)235

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Mishra AGondaliya HKumaar LBhatnagar AJoshi RLeong BSekar VYu MSeneviratne AVeitch D(2024)POSTER: Expanding the Design Space for in-Network Congestion Control on the InternetProceedings of the ACM SIGCOMM 2024 Conference: Posters and Demos10.1145/3672202.3673733(42-44)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3672202.3673733
Welzl MIslam Svon Stephanides M(2024)Real-Time TCP Packet Loss Prediction Using Machine LearningIEEE Access10.1109/ACCESS.2024.348851112(159622-159634)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3488511

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten