Bo Wang
ABSTRACT
Quantile tracking is an essential component of network measurement, where the tracked quantiles of the key performance metrics allow operators to better understand network performance. Given the high network speed and huge volume of traffic, the line-rate packet-processing performance and network visibility of programmable switches make it a trend to track quantiles in the programmable data plane. However, due to the rigorous resource constraints of programmable switches, quantile tracking is required to be both memory and computation efficient to be deployed in the data plane. In this paper, we present EasyQuantile, an efficient quantile tracking approach that has small constant memory usage and involves only hardware-friendly computations. EasyQuantile adopts an adjustable incremental update approach and calculates a pre-specified quantile with high accuracy entirely in the data plane. We implement EasyQuantile on Intel Tofino switches with small resource usage. Trace-driven experiments show that EasyQuantile achieves higher accuracy and lower complexities compared with state-of-the-art approaches.
- Pankaj K Agarwal, Graham Cormode, Zengfeng Huang, Jeff M Phillips, Zhewei Wei, and Ke Yi. 2013. Mergeable summaries. ACM Trans. Database Syst. 38, 4 (2013), 1–28.Google Scholar
Digital Library
- V Altukhov and E Chemeritskiy. 2014. On real-time delay monitoring in software-defined networks. In Proc. of IEEE MoNeTeC. IEEE, 1–6.Google ScholarCross Ref
- Aboubacar Amiri and Baba Thiam. 2014. A smoothing stochastic algorithm for quantile estimation. Statistics & Probability Letters 93 (2014), 116–125.Google ScholarCross Ref
- Ran Ben Basat, Sivaramakrishnan Ramanathan, Yuliang Li, Gianni Antichi, Minian Yu, and Michael Mitzenmacher. 2020. PINT: Probabilistic in-band network telemetry. In Proc. of ACM SIGCOMM. 662–680.Google ScholarDigital Library
- Pat Bosshart, Dan Daly, Glen Gibb, Martin Izzard, Nick McKeown, Jennifer Rexford, Cole Schlesinger, Dan Talayco, Amin Vahdat, George Varghese, 2014. P4: Programming protocol-independent packet processors. SIGCOMM Comput. Commun. Rev. 44, 3 (2014), 87–95.Google ScholarDigital Library
- CAIDA. 2014. IPv4 Routed /24 DNS Names Dataset.https://www.caida.org/data/active/ipv4_dnsnames_dataset.xml.Google Scholar
- CAIDA. 2019. The CAIDA UCSD Anonymized Internet Traces.https://www.caida.org/catalog/datasets/passive_dataset.Google Scholar
- Jin Cao, Li Erran Li, Aiyou Chen, and Tian Bu. 2010. Tracking quantiles of network data streams with dynamic operations. In Proc. of IEEE INFOCOM. IEEE, 1–5.Google ScholarCross Ref
- Fei Chen, Diane Lambert, and José C Pinheiro. 2000. Incremental quantile estimation for massive tracking. In Proc. of ACM SIGKDD. 516–522.Google ScholarDigital Library
- Peiqing Chen, Yuhan Wu, Tong Yang, Junchen Jiang, and Zaoxing Liu. 2021. Precise error estimation for sketch-based flow measurement. In Proc. of ACM IMC. 113–121.Google ScholarDigital Library
- Xiaoqi Chen, Shir Landau Feibish, Yaron Koral, Jennifer Rexford, Ori Rottenstreich, Steven A Monetti, and Tzuu-Yi Wang. 2019. Fine-grained queue measurement in the data plane. In Proc. of ACM CoNEXT. 15–29.Google ScholarDigital Library
- Xiang Chen, Qun Huang, Dong Zhang, Haifeng Zhou, and Chunming Wu. 2020. Approsync: approximate state synchronization for programmable networks. In Proc. of IEEE ICNP. IEEE, 1–12.Google ScholarCross Ref
- Ulrich Fiedler and Bernhard Plattner. 2003. Using latency quantiles to engineer qos guarantees for web services. In Proc. of ACM IWQoS. Springer, 345–362.Google Scholar
- Michael Greenwald and Sanjeev Khanna. 2001. Space-efficient online computation of quantile summaries. ACM SIGMOD Record 30, 2, 58–66.Google ScholarDigital Library
- Hugo Lewi Hammer, Anis Yazidi, and Håvard Rue. 2019. A new quantile tracking algorithm using a generalized exponentially weighted average of observations. Appl. Intell. 49 (2019), 1406–1420.Google ScholarDigital Library
- Zaobo He, Zhipeng Cai, Siyao Cheng, and Xiaoming Wang. 2015. Approximate aggregation for tracking quantiles and range countings in wireless sensor networks. Theoretical Computer Science 607 (2015), 381–390.Google ScholarDigital Library
- Nikita Ivkin, Edo Liberty, Kevin Lang, Zohar Karnin, and Vladimir Braverman. 2022. Streaming quantiles algorithms with small space and update time. Sensors 22, 24 (2022), 9612.Google ScholarCross Ref
- Nikita Ivkin, Zhuolong Yu, Vladimir Braverman, and Xin Jin. 2019. Qpipe: Quantiles sketch fully in the data plane. In Proc. of ACM CoNEXT. 285–291.Google ScholarDigital Library
- Raj Jain and Imrich Chlamtac. 1985. The P2 algorithm for dynamic calculation of quantiles and histograms without storing observations. Commun. ACM 28, 10 (1985), 1076–1085.Google ScholarDigital Library
- Zohar Karnin, Kevin Lang, and Edo Liberty. 2016. Optimal quantile approximation in streams. In Proc. of IEEE FOCS. IEEE, 71–78.Google ScholarCross Ref
- Yuliang Li, Rui Miao, Hongqiang Harry Liu, Yan Zhuang, Fei Feng, Lingbo Tang, Zheng Cao, Ming Zhang, Frank Kelly, Mohammad Alizadeh, 2019. HPCC: High precision congestion control. In Proc. of ACM SIGCOMM. 44–58.Google ScholarDigital Library
- Lingxia Liao, Victor CM Leung, and Min Chen. 2018. An efficient and accurate link latency monitoring method for low-latency software-defined networks. IEEE Trans. Instrum. Meas. 68, 2 (2018), 377–391.Google ScholarCross Ref
- Qiang Ma, Shanmugavelayutham Muthukrishnan, and Mark Sandler. 2013. Frugal streaming for estimating quantiles. Space-Efficient Data Structures, Streams, and Algorithms (2013), 77–96.Google Scholar
- Gurmeet Singh Manku, Sridhar Rajagopalan, and Bruce G Lindsay. 1999. Random sampling techniques for space efficient online computation of order statistics of large datasets. ACM SIGMOD Record 28, 2, 251–262.Google ScholarDigital Library
- Charles Masson, Jee E Rim, and Homin K Lee. 2019. DDSketch: A fast and fully-mergeable quantile sketch with relative-error guarantees. arXiv preprint arXiv:1908.10693 (2019).Google Scholar
- Byung-Hoon Park, George Ostrouchov, Nagiza F Samatova, and Al Geist. 2004. Reservoir-based random sampling with replacement from data stream. In SIAM. SIAM, 492–496.Google Scholar
- Sandhya Rathee, Shubham Tiwari, K Haribabu, and Ashutosh Bhatia. 2022. qMon: A method to monitor queueing delay in OpenFlow networks. Journal of Communications and Networks 24, 4 (2022), 463–474.Google ScholarCross Ref
- Herbert Robbins and Sutton Monro. 1951. A stochastic approximation method. The Annals of Mathematical Statistics (1951), 400–407.Google Scholar
- Rana Shahout, Roy Friedman, and Ran Ben Basat. 2022. SQUAD: Combining sketching and sampling is better than either for per-item quantile estimation. arXiv preprint arXiv:2201.01958 (2022).Google Scholar
- Siyuan Sheng, Qun Huang, and Patrick PC Lee. 2021. DeltaINT: Toward general in-band network telemetry with extremely low bandwidth overhead. In Proc. of IEEE ICNP. IEEE, 1–11.Google ScholarCross Ref
- Debanshu Sinha, K Haribabu, and Sundar Balasubramaniam. 2015. Real-time monitoring of network latency in software defined networks. In Proc. of IEEE ANTS. IEEE, 1–3.Google ScholarCross Ref
- Luke Tierney. 1983. A space-efficient recursive procedure for estimating a quantile of an unknown distribution. SIAM J. Sci. Statist. Comput. 4, 4 (1983), 706–711.Google ScholarDigital Library
- Nitya Tiwari and Prem C Pandey. 2019. A technique with low memory and computational requirements for dynamic tracking of quantiles. J. Signal Process. Syst. 91 (2019), 411–422.Google ScholarDigital Library
- [34] Tofino2. 2023. https://www.intel.com.Google Scholar
- Milan Vojnovic, J-Y Le Boudec, and Catherine Boutremans. 2000. Global fairness of additive-increase and multiplicative-decrease with heterogeneous round-trip times. In Proc. of IEEE INFOCOM, Vol. 3. IEEE, 1303–1312.Google ScholarCross Ref
- Weitao Wang, Xinyu Crystal Wu, Praveen Tammana, Ang Chen, and TS Eugene Ng. 2022. Closed-loop network performance monitoring and diagnosis with { SpiderMon}. In Proc. of USENIX NSDI. USENIX Association, 267–285.Google Scholar
- Anis Yazidi and Hugo Hammer. 2017. Multiplicative update methods for incremental quantile estimation. IEEE Trans. Cybern. 49, 3 (2017), 746–756.Google ScholarCross Ref
- Xinchang Zhang, Yinglong Wang, Jianwei Zhang, Lu Wang, and Yanling Zhao. 2019. RINGLM: A link-level packet loss monitoring solution for software-defined networks. IEEE J. Sel. Areas Commun. 37, 8 (2019), 1703–1720.Google ScholarDigital Library
- Hao Zheng, Chen Tian, Tong Yang, Huiping Lin, Chang Liu, Zhaochen Zhang, Wanchun Dou, and Guihai Chen. 2022. FlyMon: enabling on-the-fly task reconfiguration for network measurement. In Proc. of ACM SIGCOMM. 486–502.Google ScholarDigital Library
Index Terms
- EasyQuantile: Efficient Quantile Tracking in the Data Plane
Recommendations
Tracking of multiple quantiles in dynamically varying data streams
AbstractIn this paper, we consider the problem of tracking multiple quantiles of dynamically varying data stream distributions. The method is based on making incremental updates of the quantile estimates every time a new sample is received. The method is ...
A new quantile tracking algorithm using a generalized exponentially weighted average of observations
The Exponentially Weighted Average (EWA) of observations is known to be a state-of-art estimator for tracking expectations of dynamically varying data stream distributions. However, how to devise an EWA estimator to track quantiles of data stream ...
HyperParser: A High-Performance Parser Architecture for Next Generation Programmable Switch and SmartNIC
APNet '21: Proceedings of the 5th Asia-Pacific Workshop on NetworkingProgrammable switches and SmartNICs motivate the programmable network. ASIC is adopted in programmable switches to achieve high throughput, and FPGA-based SmartNIC is becoming increasingly popular. The programmable parser is a key element in ...
Comments