research-article

FCM-sketch: generic network measurements with data plane support

Authors:

Pravein Govindan Kannan,

Bryan Kian Hsiang Low,

Mun Choon ChanAuthors Info & Claims

CoNEXT '20: Proceedings of the 16th International Conference on emerging Networking EXperiments and Technologies

Pages 78 - 92

https://doi.org/10.1145/3386367.3432729

Published: 24 November 2020 Publication History

Abstract

Sketches have successfully provided accurate and fine-grained measurements (e.g., flow size and heavy hitters) which are imperative for network management. In particular, Count-Min (CM) sketch is widely utilized in many applications due to its simple design and ease of implementation. There have been many efforts to build monitoring frameworks based on Count-Min sketch. However, these frameworks either support very specific measurement tasks or they cannot be implemented on high-speed programmable hardware (PISA).

In this work, we propose FCM, a framework that is designed to support generic network measurement with high accuracy. Our key contribution is FCM-Sketch, a data structure that has a lightweight implementation on the emerging PISA programmable switches. FCM-Sketch can also be used as a substitute for CM-Sketch in applications that use CM-Sketch. We have implemented FCM-Sketch on a commodity programmable switch (Barefoot Tofino) using the P4 language. Our evaluation shows that FCM-Sketch can reduce the errors in many measurement tasks by 50% to 80% compared to CM-Sketch and other state-of-the-art approaches.

Supplementary Material

MP4 File (3386367.3432729.mp4)

FCM-Sketch

Download
237.04 MB

References

[1]

Barefoot tofino 2. https://www.barefootnetworks.com/press-releases/barefoot-networks-unveils-tofino-2-the-next-generation-of-the-worlds-first-fully-p4-programmable-network-switch-asics/.

[2]

The caida ucsd anonymized internet traces - 20190117. http://www.caida.org/data/passive/passive_dataset.xml.

[3]

Netflow. https://en.wikipedia.org/wiki/NetFlow.

[4]

P4 Language Consortium. 2018. Baseline switch.p4. https://github.com/p4lang/switch/blob/\master/p4src/switch.p4.

[5]

Portable Switch Architecture. https://p4.org/p4-spec/docs/PSA-v1.0.0.pdf.

[6]

Sflow. https://en.wikipedia.org/wiki/SFlow.

[7]

Wedge 100bf-32x. https://www.edge-core.com/productsInfo.php?\cls=1&cls2=180&cls3=181&id=335.

[8]

The world's fastest and most programmable networks. https://www.barefootnetworks.com/resources/worlds-fastest-most-programmable-networks/.

[9]

Pyramidsketch source code. https://github.com/zhouyangpkuer/Pyramid_Sketch_Framework, 2017.

[10]

Elasticsketch source code. https://github.com/BlockLiu/ElasticSketchCode, 2018.

[11]

Fcm-sketch source code. https://github.com/fcm-project, 2020.

[12]

R. B. Basat, X. Chen, G. Einziger, and O. Rottenstreich. Designing heavy-hitter detection algorithms for programmable switches. IEEE/ACM Transactions on Networking, 2020.

[13]

I. Basicevic, S. Ocovaj, and M. Popovic. The value of flow size distribution in entropy-based detection of dos attacks. Security and Communication Networks, 2016.

Digital Library

[14]

T. Benson, A. Akella, and D. A. Maltz. Network traffic characteristics of data centers in the wild. In ACM IMC, 2010.

Digital Library

[15]

P. Bereziński, B. Jasiul, and M. Szpyrka. An entropy-based network anomaly detection method. Entropy, 2015.

[16]

P. Bosshart, D. Daly, G. Gibb, M. Izzard, N. McKeown, J. Rexford, C. Schlesinger, D. Talayco, A. Vahdat, G. Varghese, et al. P4: Programming protocol-independent packet processors. ACM SIGCOMM Computer Communication Review, 2014.

Digital Library

[17]

V. Braverman and R. Ostrovsky. Generalizing the layering method of indyk and woodruff: Recursive sketches for frequency-based vectors on streams. In Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques. Springer, 2013.

[18]

Cavium. Xpliant ethernet switch product family, 2018.

[19]

M. Chen, S. Chen, and Z. Cai. Counter tree: A scalable counter architecture for per-flow traffic measurement. IEEE/ACM Transactions on Networking (TON), 2017.

[20]

X. Chen, S. L. Feibish, Y. Koral, J. Rexford, and O. Rottenstreich. Catching the microburst culprits with snappy. In Afternoon Workshop on Self-Driving Networks, 2018.

Digital Library

[21]

X. Chen, S. L. Feibish, Y. Koral, J. Rexford, and T.-Y. Wang. Fine-grained queue measurement in the data plane. In ACM CoNEXT, 2019.

Digital Library

[22]

G. Cormode and S. Muthukrishnan. An improved data stream summary: the count-min sketch and its applications. Journal of Algorithms, 2005.

[23]

J. David and C. Thomas. Ddos attack detection using fast entropy approach on flow-based network traffic. Procedia Computer Science, 2015.

[24]

D. Ding, M. Savi, G. Antichi, and D. Siracusa. Incremental deployment of programmable switches for network-wide heavy-hitter detection. In IEEE Conference on Network Softwarization (NetSoft), 2019.

[25]

G. Einziger, O. Eytan, R. Friedman, and B. Manes. Adaptive software cache management. In 19th International Middleware Conference, 2018.

Digital Library

[26]

C. Estan and G. Varghese. New directions in traffic measurement and accounting: Focusing on the elephants, ignoring the mice. ACM Transactions on Computer Systems (TOCS), 2003.

[27]

P. Flajolet, É. Fusy, O. Gandouet, and F. Meunier. Hyperloglog: the analysis of a near-optimal cardinality estimation algorithm. In Analysis of Algorithms (AofA), 2007.

[28]

A. Goyal and H. Daumé III. Approximate scalable bounded space sketch for large data nlp. In EMNLP, 2011.

[29]

R. Harrison, Q. Cai, A. Gupta, and J. Rexford. Network-wide heavy hitter detection with commodity switches. In ACM SOSR, 2018.

Digital Library

[30]

C. Henke, C. Schmoll, and T. Zseby. Empirical evaluation of hash functions for multipoint measurements. ACM SIGCOMM Computer Communication Review, 2008.

Digital Library

[31]

Q. Huang, X. Jin, P. P. Lee, R. Li, L. Tang, Y.-C. Chen, and G. Zhang. Sketchvisor: Robust network measurement for software packet processing. In ACM SIGCOMM, 2017.

Digital Library

[32]

Q. Huang, P. P. Lee, and Y. Bao. Sketchlearn: relieving user burdens in approximate measurement with automated statistical inference. In ACM SIGCOMM, 2018.

Digital Library

[33]

N. Ivkin, Z. Yu, V. Braverman, and X. Jin. Qpipe: Quantiles sketch fully in the data plane. In ACM CoNEXT, 2019.

Digital Library

[34]

X. Jin, X. Li, H. Zhang, R. Soulé, J. Lee, N. Foster, C. Kim, and I. Stoica. Netcache: Balancing key-value stores with fast in-network caching. In ACM SOSP, 2017.

Digital Library

[35]

N. Katta, M. Hira, C. Kim, A. Sivaraman, and J. Rexford. Hula: Scalable load balancing using programmable data planes. In ACM SOSR, 2016.

Digital Library

[36]

I. Katzela and M. Schwartz. Schemes for fault identification in communication networks. IEEE/ACM Transactions on Networking, 1995.

[37]

D. Kim, Z. Liu, Y. Zhu, C. Kim, J. Lee, V. Sekar, and S. Seshan. Tea: Enabling state-intensive network functions on programmable switches. In ACM SIGCOMM, 2020.

Digital Library

[38]

A. Kumar, M. Sung, J. J. Xu, and J. Wang. Data streaming algorithms for efficient and accurate estimation of flow size distribution. In ACM SIGMETRICS Performance Evaluation Review, 2004.

[39]

Y.-K. Lai, K.-Y. Shih, P.-Y. Huang, H.-P. Lee, Y.-J. Lin, T.-L. Liu, and J. H. Chen. Sketch-based entropy estimation for network traffic analysis using programmable data plane asics. In 2019 ACM/IEEE Symposium on Architectures for Networking and Communications Systems (ANCS). IEEE, 2019.

[40]

A. Lall, V. Sekar, M. Ogihara, J. Xu, and H. Zhang. Data streaming algorithms for estimating entropy of network traffic. ACM SIGMETRICS Performance Evaluation Review, 2006.

[41]

Â. C. Lapolli, J. A. Marques, and L. P. Gaspary. Offloading real-time ddos attack detection to programmable data planes. In IFIP/IEEE Symposium on Integrated Network and Service Management (IM), 2019.

[42]

Z. Liu, Z. Bai, Z. Liu, X. Li, C. Kim, V. Braverman, X. Jin, and I. Stoica. Distcache: Provable load balancing for large-scale storage systems with distributed caching. In USENIX FAST, 2019.

[43]

Z. Liu, R. Ben-Basat, G. Einziger, Y. Kassner, V. Braverman, R. Friedman, and V. Sekar. Nitrosketch: Robust and general sketch-based monitoring in software switches. In ACM SIGCOMM, 2019.

Digital Library

[44]

Z. Liu, A. Manousis, G. Vorsanger, V. Sekar, and V. Braverman. One sketch to rule them all: Rethinking network flow monitoring with univmon. In ACM SIGCOMM, 2016.

Digital Library

[45]

Y. Lu, A. Montanari, B. Prabhakar, S. Dharmapurikar, and A. Kabbani. Counter braids: a novel counter architecture for per-flow measurement. ACM SIGMETRICS Performance Evaluation Review, 2008.

[46]

J. McCauley, A. Panda, A. Krishnamurthy, and S. Shenker. Thoughts on load distribution and the role of programmable switches. ACM SIGCOMM Computer Communication Review, 2019.

[47]

R. Miao, H. Zeng, C. Kim, J. Lee, and M. Yu. Silkroad: Making stateful layer-4 load balancing fast and cheap using switching asics. In ACM SIGCOMM, 2017.

Digital Library

[48]

O. Papapetrou, M. Garofalakis, and A. Deligiannakis. Sketch-based querying of distributed sliding-window data streams. VLDB, 2012.

Digital Library

[49]

D. M. Powers. Applications and explanations of zipf's law. In New methods in language processing and computational natural language learning, 1998.

[50]

A. Roy, H. Zeng, J. Bagga, G. Porter, and A. C. Snoeren. Inside the social network's (datacenter) network. In ACM SIGCOMM, 2015.

Digital Library

[51]

P. Roy, A. Khan, and G. Alonso. Augmented sketch: Faster and more accurate stream processing. In ACM SIGMOD, 2016.

Digital Library

[52]

N. K. Sharma, A. Kaufmann, T. Anderson, A. Krishnamurthy, J. Nelson, and S. Peter. Evaluating the power of flexible packet processing for network resource allocation. In USENIX NSDI, 2017.

Digital Library

[53]

N. K. Sharma, M. Liu, K. Atreya, and A. Krishnamurthy. Approximating fair queueing on reconfigurable switches. In USENIX NSDI, 2018.

[54]

V. Sivaraman, S. Narayana, O. Rottenstreich, S. Muthukrishnan, and J. Rexford. Heavy-hitter detection entirely in the data plane. In ACM SOSR, 2017.

Digital Library

[55]

L. Tang, Q. Huang, and P. P. Lee. Spreadsketch: Toward invertible and network-wide detection of superspreaders. In IEEE INFOCOM, 2020.

Digital Library

[56]

B. Turkovic, F. Kuipers, N. van Adrichem, and K. Langendoen. Fast network congestion detection and avoidance using p4. In 2018 Workshop on Networking for Emerging Applications and Technologies, 2018.

Digital Library

[57]

V. Ðukić, S. A. Jyothi, B. Karlaš, M. Owaida, C. Zhang, and A. Singla. Is advance knowledge of flow sizes a plausible assumption? In USENIX NSDI, 2019.

[58]

K.-Y. Whang, B. T. Vander-Zanden, and H. M. Taylor. A linear-time probabilistic counting algorithm for database applications. ACM Transactions on Database Systems (TODS), 1990.

[59]

T. Yang, J. Jiang, P. Liu, Q. Huang, J. Gong, Y. Zhou, R. Miao, X. Li, and S. Uhlig. Elastic sketch: Adaptive and fast network-wide measurements. In ACM SIGCOMM, 2018.

Digital Library

[60]

T. Yang, Y. Zhou, H. Jin, S. Chen, and X. Li. Pyramid sketch: A sketch framework for frequency estimation of data streams. VLDB, 2017.

Digital Library

[61]

M. Yu, L. Jose, and R. Miao. Software defined traffic measurement with opensketch. In USENIX NSDI, 2013.

Digital Library

[62]

Y. Zhou, T. Yang, J. Jiang, B. Cui, M. Yu, X. Li, and S. Uhlig. Cold filter: A meta-framework for faster and more accurate stream processing. In ACM SIGMOD, 2018.

Digital Library

[63]

Y. Zhu, H. Eran, D. Firestone, C. Guo, M. Lipshteyn, Y. Liron, J. Padhye, S. Raindel, M. H. Yahia, and M. Zhang. Congestion control for large-scale rdma deployments. ACM SIGCOMM Computer Communication Review, 2015.

Digital Library

Cited By

Chen YWu HRen X(2025)Efficient and Secure Traffic Scheduling Based on Private SketchMathematics10.3390/math1302028813:2(288)Online publication date: 17-Jan-2025
https://doi.org/10.3390/math13020288
Gao GQian ZHuang HSun YDu Y(2025)TailoredSketch: A Fast and Adaptive Sketch for Efficient Per-Flow Size MeasurementIEEE Transactions on Network Science and Engineering10.1109/TNSE.2024.350390412:1(505-517)Online publication date: Jan-2025
https://doi.org/10.1109/TNSE.2024.3503904
Song XZheng JQian HZhao SZhang HPan XChen G(2025)In Search of a Memory-Efficient Framework for Online Cardinality EstimationIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.348657137:1(392-407)Online publication date: Jan-2025
https://doi.org/10.1109/TKDE.2024.3486571
Show More Cited By

Index Terms

FCM-sketch: generic network measurements with data plane support
1. Networks
  1. Network performance evaluation
    1. Network measurement
  2. Network services
    1. Programmable networks

Recommendations

BeauCoup: Answering Many Network Traffic Queries, One Memory Update at a Time
SIGCOMM '20: Proceedings of the Annual conference of the ACM Special Interest Group on Data Communication on the applications, technologies, architectures, and protocols for computer communication

Network administrators constantly monitor network traffic for congestion and attacks. They need to perform a large number of measurements on the traffic simultaneously, to detect different types of anomalies such as heavy hitters or super-spreaders. ...
Elastic sketch: adaptive and fast network-wide measurements
SIGCOMM '18: Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication

When network is undergoing problems such as congestion, scan attack, DDoS attack, etc., measurements are much more important than usual. In this case, traffic characteristics including available bandwidth, packet rate, and flow size distribution vary ...
Precise Time-synchronization in the Data-Plane using Programmable Switching ASICs
SOSR '19: Proceedings of the 2019 ACM Symposium on SDN Research

Current implementations of time synchronization protocols (e.g. PTP) in standard industry-grade switches handle the protocol stack in the slow-path (control-plane). With new use cases of in-network computing using programmable switching ASICs, global ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CoNEXT '20: Proceedings of the 16th International Conference on emerging Networking EXperiments and Technologies

November 2020

585 pages

ISBN:9781450379489

DOI:10.1145/3386367

Program Chairs:
Dongsu Han
KAIST, South Korea
,
Anja Feldmann
MPI, Germany

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCOMM: ACM Special Interest Group on Data Communication

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 November 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Singapore Ministry of Education

Conference

CoNEXT '20

Sponsor:

SIGCOMM

CoNEXT '20: The 16th International Conference on emerging Networking EXperiments and Technologies

December 1 - 4, 2020

Barcelona, Spain

Acceptance Rates

Overall Acceptance Rate 198 of 789 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

45
Total Citations
View Citations
1,098
Total Downloads

Downloads (Last 12 months)169
Downloads (Last 6 weeks)26

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Chen YWu HRen X(2025)Efficient and Secure Traffic Scheduling Based on Private SketchMathematics10.3390/math1302028813:2(288)Online publication date: 17-Jan-2025
https://doi.org/10.3390/math13020288
Gao GQian ZHuang HSun YDu Y(2025)TailoredSketch: A Fast and Adaptive Sketch for Efficient Per-Flow Size MeasurementIEEE Transactions on Network Science and Engineering10.1109/TNSE.2024.350390412:1(505-517)Online publication date: Jan-2025
https://doi.org/10.1109/TNSE.2024.3503904
Song XZheng JQian HZhao SZhang HPan XChen G(2025)In Search of a Memory-Efficient Framework for Online Cardinality EstimationIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.348657137:1(392-407)Online publication date: Jan-2025
https://doi.org/10.1109/TKDE.2024.3486571
Han YHuang HSun YLiu JChen S(2025)Expiration filter: Mining recent heavy flows in high-speed networksComputer Networks10.1016/j.comnet.2024.111010258(111010)Online publication date: Feb-2025
https://doi.org/10.1016/j.comnet.2024.111010
Xiao JSun RLiu J(2025)MLDDoS: a distributed denial of service attack detection method using multi-level sketchThe Journal of Supercomputing10.1007/s11227-025-06942-381:2Online publication date: 28-Jan-2025
https://doi.org/10.1007/s11227-025-06942-3
Abboud RFriedman R(2024)Practical Heavy-Hitter Detection Algorithms for Programmable Switches2024 IFIP Networking Conference (IFIP Networking)10.23919/IFIPNetworking62109.2024.10619799(377-385)Online publication date: 3-Jun-2024
https://doi.org/10.23919/IFIPNetworking62109.2024.10619799
Zheng LXiao QCai X(2024)A Universal Sketch for Estimating Heavy Hitters and Per-Element Frequency Moments in Data Streams with Bounded DeletionsProceedings of the ACM on Management of Data10.1145/36987992:6(1-28)Online publication date: 20-Dec-2024
https://dl.acm.org/doi/10.1145/3698799
Srivastava MSekar VSekar VYu MSeneviratne AVeitch D(2024)Poster: Circa: Re-imagining Network Telemetry from an Approximation-First PerspectiveProceedings of the ACM SIGCOMM 2024 Conference: Posters and Demos10.1145/3672202.3673742(57-59)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3672202.3673742
Chen XXiao QLiu HHuang QZhang DLiu XHu LZhou HWu CRen KSekar VYu MSeneviratne AVeitch D(2024)Eagle: Toward Scalable and Near-Optimal Network-Wide Sketch Deployment in Network MeasurementProceedings of the ACM SIGCOMM 2024 Conference10.1145/3651890.3672244(291-310)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3651890.3672244
Srivastava MHung SNamkung HLin KLiu ZSekar VVallina-Rodríguez NSuarez-Tángil GLevin DPelsser C(2024)Raising the Level of Abstraction for Sketch-Based Network Telemetry with SketchPlanProceedings of the 2024 ACM on Internet Measurement Conference10.1145/3646547.3689016(651-658)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3646547.3689016
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten