skip to main content
10.1145/3359989.3365406acmconferencesArticle/Chapter ViewAbstractPublication PagesconextConference Proceedingsconference-collections
research-article

HyperTester: high-performance network testing driven by programmable switches

Published: 03 December 2019 Publication History

Abstract

Modern network research and operations are inseparable from network testers to evaluate performance limits of proofs-of-concept, troubleshoot failures, etc. Existing network testers suffer from either constrained flexibility or a low performance-cost ratio. In this paper, we propose a new network tester, HyperTester. The core of HyperTester is to leverage new-generation programmable switches for generating and capturing test traffic with high performance, low cost, and remarkable flexibility. We design a series of efficient mechanisms, including template-based packet generation, false-positive-free counter-based queries, and stateless connections to realize various network testing tasks upon switches with limited programmability and resources. Meanwhile, to facilitate developing testing tasks upon HyperTester, we provide a high-level network testing API. We have implemented HyperTester on the Tofino switch and built dozens of network testing tasks. The evaluations on the hardware testbed show that HyperTester supports line-rate packet generation (400Gbps in the testbed) with highly-accurate rate control, while HyperTester can save $40150 per Tps and 9225W per Tbps when compared with the software network testers.

References

[1]
Zakir Durumeric, Eric Wustrow, and J. Alex Halderman. Zmap: Fast internet-wide scanning and its security applications. In Proceedings of NDSS, 2013.
[2]
Jan Rüth, Christian Bormann, and Oliver Hohlfeld. Large-scale scanning of tcp's initial window. In Proceedings of IMC, 2017.
[3]
Platon Kotzias, Abbas Razaghpanah, Johanna Amann, Kenneth G. Paterson, Narseo Vallina-Rodriguez, and Juan Caballero. Coming of age: A longitudinal study of tls deployment. In Proceedings of IMC, 2018.
[4]
pktgen. Website, 2019. https://www.kernel.org/doc/Documentation/networking/pktgen.txt.
[5]
Paul Emmerich, Sebastian Gallenmüller, Daniel Raumer, Florian Wohlfart, and Georg Carle. Moongen: A scriptable high-speed packet generator. In Proceedings of IMC, 2015.
[6]
pktgen-dpdk. Website, 2019. http://git.dpdk.org/apps/pktgen-dpdk.
[7]
Chuanxiong Guo, Lihua Yuan, Dong Xiang, Yingnong Dang, Ray Huang, Dave Maltz, Zhaoyi Liu, Vin Wang, Bin Pang, Hua Chen, Zhi-Wei Lin, and Varugis Kurien. Pingmesh: A large-scale system for data center network latency measurement and analysis. In Proceedings of SIGCOMM, 2015.
[8]
Yilong Geng, Shiyu Liu, Zi Yin, Balaji Prabhakar, Mendel Rosenblum, Ashish Naik, and Amin Vahdat. Simon: A simple and scalable method for sensing, inference and measurement in data center networks. In Proceedings of NSDI, 2019.
[9]
Tian Bu, Nick Duffield, Francesco Lo Presti, and Don Towsley. Network tomography on general topologies. In Proceedings of SIGMETRICS, 2002.
[10]
Yanghua Peng, Ji Yang, Chuan Wu, Chuanxiong Guo, Chengchen Hu, and Zongpeng Li. detector: a topology-aware monitoring system for data center networks. In Proceedings of ATC, 2017.
[11]
Hongyi Zeng, Peyman Kazemian, George Varghese, and Nick McKeown. Automatic test packet generation. IEEE/ACM Trans. Netw., 22(2), 2014.
[12]
Yu Zhao, Huazhe Wang, Xin Lin, Tingting Yu, and Chen Qian. Pronto: Efficient test packet generation for dynamic network data planes. In Proceedings of ICDCS, 2017.
[13]
Peter Perešíni, Maciej Kuźniar, and Dejan Kostić. Monocle: Dynamic, fine-grained data plane monitoring. In Proceedings of CoNEXT, 2015.
[14]
Peng Zhang, Cheng Zhang, and Chengchen Hu. Fast testing network data plane with rulechecker. In Proceedings of ICNP, 2017.
[15]
Xitao Wen, Kai Bu, Bo Yang, Yan Chen, Li Erran Li, Xiaolin Chen, Jianfeng Yang, and Xue Leng. Is every flow on the right track?: Inspect sdn forwarding with rulescope. In Proceedings of INFOCOM, 2016.
[16]
Yibo Zhu, Ben Y. Zhao, Haitao Zheng, Nanxi Kang, Jiaxin Cao, Albert Greenberg, Guohan Lu, Ratul Mahajan, Dave Maltz, Lihua Yuan, and Ming Zhang. Packet-level telemetry in large datacenter networks. In Proceedings of SIGCOMM, 2015.
[17]
Francois Aubry, David Lebrun, Stefano Vissicchio, Minh Thanh Khong, Yves Deville, and Olivier Bonaventure. Scmon: Leveraging segment routing to improve network monitoring. In Proceedings of INFOCOM, 2016.
[18]
Yu Zhou, Jun Bi, Yunsenxiao Lin, Yangyang Wang, Dai Zhang, Zhaowei Xi, Jiamin Cao, and Chen Sun. P4tester: Efficient runtime rule fault detection for programmable data planes. In Proceedings of IWQoS, 2019.
[19]
IXIA. Test hardware. Website, 2019. https://ixia.keysight.com/products/test-hardware.
[20]
Spirent. Spirent testcenter. Website, 2019. https://www.spirent.com/products/testcenter.
[21]
Gianni Antichi, Charalampos Rotsos, and Andrew W. Moore. Enabling performance evaluation beyond 10 gbps. SIGCOMM Comput. Commun. Rev., 45(4):369--370, August 2015.
[22]
G. Adam Covington, Glen Gibb, John W. Lockwood, and Nick McKeown. A packet generator on the netfpga platform. In Proceedings of FCCM, 2009.
[23]
Charalampos Rotsos, Nadi Sarrar, Steve Uhlig, Rob Sherwood, and Andrew W Moore. Oflops: An open framework for openflow switch evaluation. In Proceedings of PAM, 2012.
[24]
Paul Emmerich, Sebastian Gallenmuller, Gianni Antichi, Andrew W. Moore, and Georg Carle. Mind the gap - a comparison of software packet generators. In Proceedings of ANCS, 2017.
[25]
Pat Bosshart, Glen Gibb, Hun-seok Kim, George Varghese, Nick Mckeown, Martin Izzard, Fernando Mujica, and Mark Horowitz. Forwarding metamorphosis: Fast programmable match-action processing in hardware for sdn. In Proceedings of SIGCOMM, 2013.
[26]
Sharad Chole, Isaac Keslassy, Ariel Orda, Tom Edsall, Andy Fingerhut, Sha Ma, Anirudh Sivaraman, Shay Vargaftik, Alon Berger, Gal Mendelson, Mohammad Alizadeh, and Shang-Tse Chuang. drmt: Disaggregated programmable switching. In Proceedings of SIGCOMM, 2017.
[27]
Cavium. Xpliant ethernet switch product family. Website. https://www.cavium.com/xpliant-ethernet-switch-product-family.html.
[28]
Barefoot Networks. Tofino. Website, 2019. https://www.barefootnetworks.com/products/brief-tofino/.
[29]
Barefoot Networks. Tofino2. Website, 2019. https://www.barefootnetworks.com/products/brief-tofino-2/.
[30]
John Sonchack, Adam J. Aviv, Eric Keller, and Jonathan M. Smith. Turboflow: Information rich flow record generation on commodity switches. In Proceedings of EuroSys, 2018.
[31]
Vibhaalakshmi Sivaraman, Srinivas Narayana, Ori Rottenstreich, S. Muthukrishnan, and Jennifer Rexford. Heavy-hitter detection entirely in the data plane. In Proceedings of the Symposium on SDN Research, SOSR, pages 164--176. ACM, 2017.
[32]
Tong Yang, Jie Jiang, Peng Liu, Qun Huang, Junzhi Gong, Yang Zhou, Rui Miao, Xiaoming Li, and Steve Uhlig. Elastic sketch: Adaptive and Fast Network-wide Measurements. In Proceedings of SIGCOMM, 2018.
[33]
Qun Huang, Patrick P. C. Lee, and Yungang Bao. Sketchlearn: Relieving User Burdens in Approximate Measurement with Automated Statistical Inference. In Proceedings of SIGCOMM, 2018.
[34]
The P4 Language Consortium. The p4 language specificatio. Website. https://goo.gl/Sq3TRK.
[35]
Muhammad Shahbaz, Lalith Suresh, Jennifer Rexford, Nick Feamster, Ori Rottenstreich, and Mukesh Hira. Elmo: Source routed multicast for public clouds. In Proceedings of SIGCOMM, 2019.
[36]
Marvell. Zen and the art of network timestamping. Website. https://www.marvell.com/documents/7a47sa82moydu7z8uvam/.
[37]
Repositories for hypertester with strict anonymity. Website, 2019. https://github.com/hypertester/.
[38]
Pat Bosshart, Glen Gibb, Hun-seok Kim, George Varghese, Nick Mckeown, Martin Izzard, Fernando Mujica, and Mark Horowitz. P4: Programming protocol-independent packet processors. SIGCOMM CCR, 44(3), 2014.
[39]
Noa Zilberman, Yury Audzevich, Georgina Kalogeridou, Neelakandan Manihatty-Bojan, Jingyun Zhang, and Andrew Moore. Netfpga: Rapid prototyping of networking devices in open source. SIGCOMM CCR, 45(4):363--364, August 2015.
[40]
Bojie Li, Kun Tan, Layong Larry Luo, Yanqing Peng, Renqian Luo, Ningyi Xu, Yongqiang Xiong, Peng Cheng, and Enhong Chen. Clicknp: Highly flexible and high performance network processing with reconfigurable hardware. In Proceedings of SIGCOMM, 2016.
[41]
Han Wang, Robert Soulé, Huynh Tu Dang, Ki Suh Lee, Vishal Shrivastav, Nate Foster, and Hakim Weatherspoon. P4fpga: A rapid prototyping framework for p4. In Proceedings of SOSR, pages 122--135, 2017.
[42]
Dijilent. Netfpga-sume virtex-7 fpga development board. Website. https://store.digilentinc.com/netfpga-sume-virtex-7-fpga-development-board/.
[43]
Scapy. Scapy project. Website. https://scapy.net.
[44]
Data plane development kit. Website, 2019. https://www.dpdk.org.
[45]
Luigi Rizzo. netmap: A novel framework for fast packet i/o. In Proceedings of ATC, 2012.
[46]
Xin Jin, Xiaozhou Li, Haoyu Zhang, Robert Soulé, Jeongkeun Lee, Nate Foster, Changhoon Kim, and Ion Stoica. Netcache: Balancing key-value stores with fast in-network caching. In Proceedings of SOSP, 2017.
[47]
Xin Jin, Xiaozhou Li, Haoyu Zhang, Robert Soulé, Jeongkeun Lee, Nate Foster, Changhoon Kim, and Ion Stoica. Netchain: Scale-free sub-rtt coordination. In Proceedings of NSDI, 2018.
[48]
Huynh Tu Dang, Daniele Sciascia, Marco Canini, Fernando Pedone, and Robert Soulé. Netpaxos: Consensus at network speed. In Proceedings of SOSR, 2015.
[49]
Rui Miao, Hongyi Zeng, Changhoon Kim, Jeongkeun Lee, and Minlan Yu. Silkroad: Making stateful layer-4 load balancing fast and cheap using switching asics. In Proceedings of SIGCOMM, 2017.
[50]
Naga Katta, Mukesh Hira, Changhoon Kim, Anirudh Sivaraman, and Jennifer Rexford. Hula: Scalable load balancing using programmable data planes. In Proceedings of SOSR, 2016.
[51]
Mojgan Ghasemi, Theophilus Benson, and Jennifer Rexford. Dapper: Data plane performance diagnosis of tcp. In Proceedings of SOSR, 2017.
[52]
Yuliang Li, Rui Miao, Changhoon Kim, and Minlan Yu. Lossradar: Fast detection of lost packets in data center networks. In Proceedings of CoNEXT, 2016.
[53]
Yuliang Li, Rui Miao, Changhoon Kim, and Minlan Yu. Flowradar: A better netflow for data centers. In Proceedings of NSDI, 2016.
[54]
Li Chen, Ge Chen, Justinas Lingys, and Kai Chen. Programmable switch as a parallel computing device. CoRR, abs/1803.01491, 2018.
[55]
Arpit Gupta, Rob Harrison, Marco Canini, Nick Feamster, Jennifer Rexford, and Walter Willinger. Sonata: Query-driven network telemetry. In Proceedings of SIGCOMM, 2018.
[56]
Srinivas Narayana, Anirudh Sivaraman, Vikram Nathan, Prateesh Goyal, Venkat Arun, Mohammad Alizadeh, Vimalkumar Jeyakumar, and Changhoon Kim. Language-Directed Hardware Design for Network Performance Monitoring. In Proceedings of SIGCOMM, 2017.
[57]
Sheng Liu, Theophilus A. Benson, and Michael K. Reiter. Efficient and safe network updates with suffix causal consistency. In Proceedings of EuroSys, 2019.
[58]
Ahmed El-Hassany, Petar Tsankov, Laurent Vanbever, and Martin Vechev. Net-complete: Practical network-wide configuration synthesis with autocompletion. In Proceedings of NSDI, 2018.
[59]
Eric Osterweil, Angelos Stavrou, and Lixia Zhang. 20 years of ddos: a call to action. CoRR, abs/1904.02739, 2019.
[60]
David Moore, Geoffrey M. Voelker, and Stefan Savage. Inferring internet denial-of-service activity. In Proceedings of SSYM, 2001.
[61]
Akamai. Akamai ddos protection. Website, 2019. https://www.akamai.com/us/en/resources/ddos-protection.jsp.
[62]
A10 Networks. Thunder tps. Website, 2019. https://www.a10networks.com/products/thunder-tps/.
[63]
Akamai. Akamai ddos protection. Website, 2019. https://www.akamai.com/us/en/resources/ddos-protection.jsp.
[64]
CloudFlare. Cloudflare advanced ddos attack protection. Website, 2019. https://www.cloudfare.com/ddos/.
[65]
The Apache Software Foundation. Flink: Stateful computations over data streams. Website. https://flink.apache.org.
[66]
Wikipedia. Inverse transform sampling. Website. https://en.wikipedia.org/wiki/Inverse_transform_sampling.
[67]
Burton H. Bloom. Space/time trade-offs in hash coding with allowable errors. Commun. ACM, 13(7), July 1970.
[68]
Graham Cormode and Shan Muthukrishnan. An improved data stream summary: the count-min sketch and its applications. Journal of Algorithms, 55(1):58--75, 2005.
[69]
R. Ben-Basat, X. Chen, G. Einziger, and O. Rottenstreich. Efficient measurement on programmable switches using probabilistic recirculation. In Proceedings of ICNP, 2018.
[70]
Bin Fan, Dave G. Andersen, Michael Kaminsky, and Michael D. Mitzenmacher. Cuckoo Filter: Practically Better Than Bloom. In Proceedings of CoNEXT, 2014.
[71]
The P4 Language Consortium. Consolidated switch repository. Website, 2019. https://github.com/p4lang/switch.
[72]
A10. Testing ddos defense effectiveness at 300 gbps scale and beyond. Website. https://www.a10networks.com/marketing-comms/white-papers/testing-ddos-defense-effectiveness-300-gbps-scale/.

Cited By

View all
  • (2024)DEMO: P4 Replay (P4R): Reproducing Packet Traces and Stateful Connections at Line-Rate on Your P4-capable HardwareProceedings of the ACM SIGCOMM 2024 Conference: Posters and Demos10.1145/3672202.3673743(122-124)Online publication date: 4-Aug-2024
  • (2024)Toward Scalable and Low-Cost Traffic Testing for Evaluating DDoS Defense SolutionsIEEE/ACM Transactions on Networking10.1109/TNET.2023.328144932:1(191-206)Online publication date: Feb-2024
  • (2024)PIPO-TG: Parameterizable High-Performance Traffic GenerationNOMS 2024-2024 IEEE Network Operations and Management Symposium10.1109/NOMS59830.2024.10575636(1-9)Online publication date: 6-May-2024
  • Show More Cited By

Index Terms

  1. HyperTester: high-performance network testing driven by programmable switches

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      CoNEXT '19: Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies
      December 2019
      395 pages
      ISBN:9781450369985
      DOI:10.1145/3359989
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 03 December 2019

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. P4
      2. network testing
      3. programmable switches

      Qualifiers

      • Research-article

      Funding Sources

      • National Science Foundation of China
      • National Key R&D Program of China

      Conference

      CoNEXT '19
      Sponsor:

      Acceptance Rates

      Overall Acceptance Rate 198 of 789 submissions, 25%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)55
      • Downloads (Last 6 weeks)6
      Reflects downloads up to 14 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)DEMO: P4 Replay (P4R): Reproducing Packet Traces and Stateful Connections at Line-Rate on Your P4-capable HardwareProceedings of the ACM SIGCOMM 2024 Conference: Posters and Demos10.1145/3672202.3673743(122-124)Online publication date: 4-Aug-2024
      • (2024)Toward Scalable and Low-Cost Traffic Testing for Evaluating DDoS Defense SolutionsIEEE/ACM Transactions on Networking10.1109/TNET.2023.328144932:1(191-206)Online publication date: Feb-2024
      • (2024)PIPO-TG: Parameterizable High-Performance Traffic GenerationNOMS 2024-2024 IEEE Network Operations and Management Symposium10.1109/NOMS59830.2024.10575636(1-9)Online publication date: 6-May-2024
      • (2023)Enhancing Network Visibility and Security with Advanced Port Scanning TechniquesSensors10.3390/s2317754123:17(7541)Online publication date: 30-Aug-2023
      • (2023)PTA: Finding Hard-to-Find Data Plane BugsIEEE/ACM Transactions on Networking10.1109/TNET.2022.321406231:3(1324-1337)Online publication date: Jun-2023
      • (2023)Automatic Performance-Optimal Offloading of Network Functions on Programmable SwitchesIEEE Transactions on Cloud Computing10.1109/TCC.2022.314981711:2(1591-1607)Online publication date: 1-Apr-2023
      • (2023)P4TG: 1 Tb/s Traffic Generation for Ethernet/IP NetworksIEEE Access10.1109/ACCESS.2023.324626211(17525-17535)Online publication date: 2023
      • (2022)CoFilter: High-Performance Switch-Accelerated Stateful Packet Filter for Bare-Metal ServersIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2021.313657533:9(2249-2262)Online publication date: 1-Sep-2022
      • (2022)TurboNet: Faithfully Emulating Networks With Programmable SwitchesIEEE/ACM Transactions on Networking10.1109/TNET.2022.314212630:3(1395-1409)Online publication date: Jun-2022
      • (2021)VisionProceedings of the 8th ACM Conference on Information-Centric Networking10.1145/3460417.3482973(13-19)Online publication date: 22-Sep-2021
      • Show More Cited By

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media