Robin Marx
ABSTRACT
The QUIC and HTTP/3 protocols are quickly maturing together with their implementations, though many of their low-level behaviours are not yet well-understood. To help improve this, we empirically compare 15 IETF QUIC+HTTP/3 implementations for advanced features like Flow and Congestion Control, 0-RTT, Multiplexing, and Packetization. We find a large heterogeneity between stacks, discuss uncovered bugs and conclude that most implementations are not fully optimized or validated yet. We argue that future work must prioritize rigorous root-cause analysis of observed behaviours, and show this is possible by employing our qlog and qvis tools.
- 2020. (2020). https://github.com/rmarx/aioquic.Google Scholar
- 2020. Active QUIC implementations. (2020). https://github.com/quicwg/base-drafts/wiki/Implementations.Google Scholar
- 2020. ns-3: a discrete-event network simulator for Internet systems. (May 2020). https://www.nsnam.org/.Google Scholar
- 2020. QUIC Interop Runner. (2020). https://interop.seemann.io/.Google Scholar
- 2020. QUICdev Slack Group. (2020). https://quicdev.slack.com/.Google Scholar
- Amit Aggarwal, Stefan Savage, and Thomas Anderson. 2000. Understanding the performance of TCP pacing. In Proceedings of the Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE INFOCOM 2000)., Vol. 3. IEEE, 1157--1165. https://doi.org/10.1109/INFCOM.2000.832483Google ScholarCross Ref
- Venkat Arun and Hari Balakrishnan. 2018. Copa: Practical Delay-Based Congestion Control for the Internet. In 15th USENIX Symposium on Networked Systems Design and Implementation (NSDI 18). USENIX Association, Renton, WA, 329--342. https://www.usenix.org/conference/nsdi18/presentation/arunGoogle ScholarDigital Library
- Mike Bishop. 2020. Hypertext Transfer Protocol Version 3 (HTTP/3). Internet-Draft. https://tools.ietf.org/html/draft-ietf-quic-http-27Google Scholar
- Prasenjeet Biswal and Omprakash Gnawali. 2016. Does quic make the web faster?. In IEEE Global Communications Conference (GLOBECOM 2016). IEEE, 1--6. https://doi.org/10.1109/GLOCOM.2016.7841749Google ScholarCross Ref
- Enrico Bocchi, Luca De Cicco, and Dario Rossi. 2016. Measuring the Quality of Experience of Web Users. SIGCOMM Comput. Commun. Rev. 46, 4 (Dec. 2016), 8--13. https://doi.org/10.1145/3027947.3027949Google ScholarDigital Library
- Neal Cardwell, Yuchung Cheng, C. Stephen Gunn, Soheil Hassas Yeganeh, and Van Jacobson. 2017. BBR: Congestion-Based Congestion Control. Commun. ACM 60, 2 (Jan. 2017), 58--66. https://doi.org/10.1145/3009824Google ScholarDigital Library
- Gaetano Carlucci, Luca De Cicco, and Saverio Mascolo. 2015. HTTP over UDP: An Experimental Investigation of QUIC. In Proceedings of the 30th Annual ACM Symposium on Applied Computing (SAC '15). Association for Computing Machinery, New York, NY, USA, 609--614. https://doi.org/10.1145/2695664.2695706Google ScholarDigital Library
- Junho Choi. 2020. CUBIC and HyStart++ Support in quiche. (2020). https://blog.cloudflare.com/cubic-and-hystart-support-in-quiche/.Google Scholar
- Junho Choi. 2020. quiche: update receiver flow control. (May 2020). https://github.com/cloudflare/quiche/pull/529.Google Scholar
- Andy Davies and Patrick Meenan. 2018. Tracking HTTP/2 Prioritization Issues. (December 2018). https://github.com/andydavies/http2-prioritization-issues.Google Scholar
- Willem de Bruijn and Eric Dumazet. 2018. Optimizing UDP for content delivery: GSO, pacing and zerocopy. In Linux Plumbers Conference. http://vger.kernel.org/lpc_net2018_talks/willemdebruijn-lpc2018-udpgso-paper-DRAFT-1.pdf.Google Scholar
- Fairhurst et al. 2020. Packetization Layer Path MTU Discovery for Datagram Transports. Internet-Draft. https://tools.ietf.org/html/draft-ietf-tsvwg-datagram-plpmtud-20Google Scholar
- Gorry Fairhurst and Ana Custura. 2020. Changing QUIC defaultto ACK 1:10. (2020). https://erg.abdn.ac.uk/users/gorry/ietf/QUIC/QUIC-ack10-24-april-00.pdf.Google Scholar
- Alessandro Ghedini. 2020. Accelerating UDP packet transmission for QUIC. (2020). https://blog.cloudflare.com/accelerating-udp-packet- transmission-for-quic/.Google Scholar
- Christian Huitema. 2020. Faster slow start for satellite links? (2020). https://huitema.wordpress.com/2020/04/21/faster-slow-start-for-satellite-links/.Google Scholar
- Jana Iyengar and Ian Swett. 2020. QUIC Loss Detection and Congestion Control. Internet-Draft. https://tools.ietf.org/html/draft-ietf-quic-recovery-27Google Scholar
- Jana Iyengar and Ian Swett. 2020. Sender Control of Acknowledgement Delays in QUIC. Internet-Draft. https://tools.ietf.org/html/draft-iyengar-quic-delayed-ack-00Google Scholar
- Jana Iyengar and Martin Thomson. 2020. QUIC: A UDP-Based Multiplexed and Secure Transport. Internet-Draft. https://tools.ietf.org/html/draft-ietf-quic-transport-27Google Scholar
- Arash Molavi Kakhki, Samuel Jero, David Choffnes, Cristina Nita-Rotaru, and Alan Mislove. 2017. Taking a Long Look at QUIC: An Approach for Rigorous Evaluation of Rapidly Evolving Transport Protocols. In Proceedings of the 2017 Internet Measurement Conference (IMC '17). Association for Computing Machinery, New York, NY, USA, 290--303. https://doi.org/10.1145/3131365.3131368Google ScholarDigital Library
- Jana Iyengar Kazuho Oku. 2020. Can QUIC match TCP's computational efficiency? (2020). https://www.fastly.com/blog/measuring-quic-vs-tcp-computational-efficiency.Google Scholar
- Adam Langley, Alistair Riddoch, Alyssa Wilk, Antonio Vicente, Charles Krasic, Dan Zhang, Fan Yang, Fedor Kouranov, Ian Swett, Janardhan Iyengar, Jeff Bailey, Jeremy Dorfman, Jim Roskind, Joanna Kulik, Patrik Westin, Raman Tenneti, Robbie Shade, Ryan Hamilton, Victor Vasiliev, Wan-Teh Chang, and Zhongyi Shi. 2017. The QUIC Transport Protocol: Design and Internet-Scale Deployment. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication (SIGCOMM '17). Association for Computing Machinery, New York, NY, USA, 183--196. https://doi.org/10.1145/3098822.3098842Google ScholarDigital Library
- Robin Marx. 2020. qvis toolsuite live. (2020). https://qvis.edm.uhasselt.be.Google Scholar
- Robin Marx., Tom De Decker., Peter Quax., and Wim Lamotte. 2019. Of the Utmost Importance: Resource Prioritization in HTTP/3 over QUIC. In Proceedings of the 15th International Conference on Web Information Systems and Technologies (WEBIST 2019). INSTICC, SciTePress, 130--143. https://doi.org/10.5220/0008191701300143Google ScholarCross Ref
- Robin Marx, Wim Lamotte, Jonas Reynders, Kevin Pittevils, and Peter Quax. 2018. Towards QUIC Debuggability. In Proceedings of the Workshop on the Evolution, Performance, and Interoperability of QUIC (EPIQ '18). Association for Computing Machinery, New York, NY, USA, 1--7. https://doi.org/10.1145/3284850.3284851Google ScholarDigital Library
- Robin Marx, Maxime Piraux, Peter Quax, and Wim Lamotte. 2020. Debugging Modern Web Protocols with qlog. In Proceedings of the Applied Networking Research Workshop (ANRW 2020). https://qlog.edm.uhasselt.be/anrw/Google Scholar
- Robin Marx, Marten Seemann, and Jeremy Lainé. 2019. The IETF I-D documents for the qlog format. (2019). https://github.com/quiclog/internet-drafts.Google Scholar
- Patrick Meenan. 2019. Better HTTP/2 Prioritization for a Faster Web. (2019). https://blog.cloudflare.com/better-http- 2-prioritization-for-a-faster-web/.Google Scholar
- Péter Megyesi, Zsolt Krämer, and Sándor Molnár. 2016. How quick is QUIC?. In IEEE International Conference on Communications (ICC 2016). IEEE, 1--6. https://doi.org/10.1109/ICC.2016.7510788Google ScholarCross Ref
- Kazuho Oku and Lucas Pardue. 2020. Extensible Prioritization Scheme for HTTP. Internet-Draft. https://tools.ietf.org/html/draft-ietf-httpbis-priority-00Google Scholar
- Mirko Palmer, Thorben Krüger, Balakrishnan Chandrasekaran, and Anja Feldmann. 2018. The QUIC Fix for Optimal Video Streaming. In Proceedings of the Workshop on the Evolution, Performance, and Interoperability of QUIC (EPIQ '18). Association for Computing Machinery, New York, NY, USA, 43--49. https://doi.org/10.1145/3284850.3284857Google ScholarDigital Library
- James Pavur, Martin Strohmeier, Vincent Lenders, and Ivan Martinovic. 2020. QPEP: A QUIC-Based Approach to Encrypted Performance Enhancing Proxies for High-Latency Satellite Broadband. arXiv preprint (2020). https://arxiv.org/pdf/2002.05091.pdf.Google Scholar
- Maxime Piraux, Quentin De Coninck, and Olivier Bonaventure. 2018. Observing the Evolution of QUIC Implementations. In Proceedings of the Workshop on the Evolution, Performance, and Interoperability of QUIC (EPIQ '18). Association for Computing Machinery, New York, NY, USA, 8--14. https://doi.org/10.1145/3284850.3284852Google ScholarDigital Library
- Jan Rüth, Ike Kunze, and Oliver Hohlfeld. 2019. TCP's Initial Window---Deployment in the Wild and Its Impact on Performance. IEEE Transactions on Network and Service Management (TNSM 2019) 16, 2 (2019), 389--402. https://doi.org/10.1109/TNSM.2019.2896335Google Scholar
- Darius Saif, Chung-Horng Lung, and Ashraf Matrawy. 2020. An Early Benchmark of Quality of Experience Between HTTP/2 and HTTP/3 using Lighthouse. arXiv preprint (2020). https://arxiv.org/pdf/2004.01978.pdf.Google Scholar
- Robbie Shade. 2016. Flow Control in Google QUIC. (2016). https://docs.google.com/document/d/1F2YfdDXKpy20WVKJueEf4abn_LVZHhMUMS5gX6Pgjl4.Google Scholar
- Daniel Stenberg. 2020. qlog with curl. (2020). https://daniel.haxx.se/blog/2020/05/07/qlog-with-curl/.Google Scholar
- Ian Swett and Robin Marx. 2019. HTTP Priority design team update - IETF 107. (18 November 2019). https://github.com/httpwg/wg-materials/blob/gh-pages/ietf106/priorities.pdf.Google Scholar
- Martin Thomson and Sean Turner. 2020. Using TLS to Secure QUIC. Internet-Draft. https://tools.ietf.org/html/draft-ietf-quic-tls-27Google Scholar
- Eric Weigle and Wu-chun Feng. 2002. A comparison of TCP automatic tuning techniques for distributed computing. In Proceedings 11th IEEE International Symposium on High Performance Distributed Computing (HPDC 2002). IEEE, 265--272. https://doi.org/10.1109/HPDC.2002.1029926Google ScholarCross Ref
- Maarten Wijnants, Robin Marx, Peter Quax, and Wim Lamotte. 2018. HTTP/2 Prioritization and Its Impact on Web Performance. In Proceedings of the 2018 World Wide Web Conference (WWW'18). ACM, 1755--1764. https://doi.org/10.1145/3178876.3186181Google ScholarDigital Library
- Konrad Wolsing, Jan Rüth, Klaus Wehrle, and Oliver Hohlfeld. 2019. A Performance Perspective on Web Optimized Protocol Stacks: TCP+TLS+HTTP/2 vs. QUIC. In Proceedings of the Applied Networking Research Workshop (ANRW '19). Association for Computing Machinery, New York, NY, USA, 1--7. https://doi.org/10.1145/3340301.3341123Google ScholarDigital Library
Index Terms
- Same Standards, Different Decisions: A Study of QUIC and HTTP/3 Implementation Diversity
Recommendations
Dissecting Performance of Production QUIC
WWW '21: Proceedings of the Web Conference 2021IETF QUIC, the standardized version of Google’s UDP-based layer-4 network protocol, has seen increasing adoption from large Internet companies for its benefits over TCP. Yet despite its rapid adoption, performance analysis of QUIC in production is ...
ECN with QUIC: Challenges in the Wild
IMC '23: Proceedings of the 2023 ACM on Internet Measurement ConferenceTCP and QUIC can both leverage ECN to avoid congestion loss and its retransmission overhead. However, both protocols require support of their remote endpoints and it took two decades since the initial standardization of ECN for TCP to reach 80% ECN ...
Poster: QUIC is not Quick Enough over Fast Internet
IMC '23: Proceedings of the 2023 ACM on Internet Measurement ConferenceQUIC is a multiplexed transport-layer protocol over UDP and comes with enforced encryption. It is expected to be a game-changer in improving web application performance. Together with the network layer and layers below, UDP, QUIC, and HTTP/3 form a new ...
Comments