Abstract
Network management tasks require real-time visibility of current network status to perform the appropriate operations. However, the resource limitation of network devices and the real-time requirements make it difficult to provide accurate network measurement feedbacks. To reduce the error and inefficiencies caused by random operations in existing algorithms, we propose an efficient measurement architecture with the Deterministic Admission Policy (DAP). DAP provides accurate large-flow detection and high network measurement precision by making full use of the information belong to large flows and small flows, and dynamically filtrating small flows as the network status evolves. To make the algorithm easy to implement on hardware, we propose d-Length DAP by replacing the global optimality with local optimality. Experimental results show that our algorithm can reduce the measurement error by 3 to 25 times compared to other algorithms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Basat, R.B., Einziger, G., Friedman, R., Kassner, Y.: Randomized admission policy for efficient top-k and frequency estimation. In: IEEE INFOCOM Conference on Computer Communications, pp. 1–9. IEEE (2017)
Yang, T., et al.: Elastic sketch: adaptive and fast network-wide measurements. In: Proceedings of the Conference of the ACM Special Interest Group on Data Communication, pp. 561–575. ACM (2018)
Huang, Q., Lee, P.P.C., Bao, Y.: Sketchlearn: relieving user burdens in approximate measurement with automated statistical inference. In: Proceedings of the Conference of the ACM Special Interest Group on Data Communication, pp. 576–590. ACM (2018)
Huang, Q.: Sketchvisor: robust network measurement for software packet processing. In: Proceedings of the Conference of the ACM Special Interest Group on Data Communication, pp. 113–126. ACM (2017)
Liu, Z., Manousis, A., Vorsanger, G., Sekar, V., Braverman, V.: One sketch to rule them all: rethinking network flow monitoring with UnivMon. In: Proceedings of the ACM SIGCOMM Conference, pp. 101–114. ACM (2016)
Yu, M., Jose, L.,Miao, R.: Software defined traffic measurement with OpenSketch. In: Presented as part of the 10th USENIX Symposium on Networked Systems Design and Implementation (NSDI 2013), pp. 29–42 (2013)
Zhou, Y., Zhou, Y., Chen, S., Zhang, Y.: Highly compact virtual active counters for per-flow traffic measurement. In: IEEE INFOCOM Conference on Computer Communications, pp. 1–9. IEEE (2018)
Assaf, E., Basat, R.B., Einziger, G., Friedman, R.: Pay for a sliding bloom filter and get counting, distinct elements, and entropy for free. In: IEEE INFOCOM Conference on Computer Communications, pp. 2204–2212. IEEE (2018)
Xiwen, Y., Hongli, X., Yao, D., Wang, H., Huang, L.: CountMax: a lightweight and cooperative sketch measurement for software-defined networks. IEEE/ACM Trans. Netw. (TON) 26(6), 2774–2786 (2018)
Basat, R.B., Einziger, G., Friedman, R., Kassner, Y.: Optimal elephant flow detection. In: IEEE INFOCOM Conference on Computer Communications, pp. 1–9. IEEE (2017)
Basat, R.B., Einziger, G., Friedman, R., Kassner, Y.: Heavy hitters in streams and sliding windows. In: IEEE INFOCOM - The 35th Annual IEEE International Conference on Computer Communications, pp. 1–9. IEEE (2016)
Nyang, D.H., Shin, D.O.: Recyclable counter with confinement for real-time per-flow measurement. IEEE/ACM Trans. Netw. (TON) 24(5), 3191–3203 (2016)
Demaine, E.D., López-Ortiz, A., Munro, J.I.: Frequency estimation of internet packet streams with limited space. In: Möhring, R., Raman, R. (eds.) ESA 2002. LNCS, vol. 2461, pp. 348–360. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-45749-6_33
Manku, G.S., Motwani, R.: Approximate frequency counts over data streams. In: VLDB 2002: Proceedings of the 28th International Conference on Very Large Databases, pp. 346–357. Elsevier (2002)
Metwally, A., Agrawal, D., El Abbadi, A.: Efficient computation of frequent and top-k elements in data streams. In: Eiter, T., Libkin, L. (eds.) ICDT 2005. LNCS, vol. 3363, pp. 398–412. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-30570-5_27
Cormode, G., Muthukrishnan, S.: An improved data stream summary: the count-min sketch and its applications. J. Algorithms 55(1), 58–75 (2005)
Einziger, G., Friedman, R.: Tinyset–an access efficient self adjusting bloom filter construction. IEEE/ACM Trans. Netw. 25(4), 2295–2307 (2017)
Einziger, G., Friedman, R.: Counting with tinytable: every bit countscounting with tinytable: every bit counts! IEEE Access (2019)
Hash table. https://en.wikipedia.org/wiki/Hash_table
The CAIDA UCSD anonymized internet traces 2015 - February 19th. http://www.caida.org/data/passive/passive_dataset.xml
Acknowledgments
This research is sponsored by National Key R&D Program of China (2017YFB0902600); State Grid Corporation of China Project (SGJS0000DKJS1700840) Research and Application of Key Technology for Intelligent Dispatching and Security Early-warning of Large Power Grid.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Du, H., Wang, R., Shen, Z., Jia, Z. (2020). Accurate Network Flow Measurement with Deterministic Admission Policy. In: Wen, S., Zomaya, A., Yang, L.T. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2019. Lecture Notes in Computer Science(), vol 11945. Springer, Cham. https://doi.org/10.1007/978-3-030-38961-1_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-38961-1_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-38960-4
Online ISBN: 978-3-030-38961-1
eBook Packages: Computer ScienceComputer Science (R0)