Abstract
The sketch structure is widely applied in the network measurement field due to its limited memory usage and simple operation. However, When the less memory space the system occupied, the accuracy decreases. However, as the flow rate rapidly increases, the on-chip memory will become a system bottleneck. HybridSketch provides a methods to save memory and maintain the accuracy of the measurement system. We apply analysis and new algorithms to make it more efficient. We analyze the error bound of the system and observed that the sketch part of the system will lose the precision of the information along with less memory inevitably. So we propose the data augmentation algorithm based on our analysis. We apply it and propose the optimized HybridSketch. We evaluate the performance and present a comparison with the origin algorithm. The results show that optimized HybridSketch provides an 80% precision rate compared to the original one which occupied 10\( \times \) the memory size.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Huang, Q., Lee, P. P., Bao, Y.: Sketchlearn: relieving user burdens in approximate measurement with automated statistical inference. In Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication (SIGCOMM ’18). ACM, New York, NY, USA, pp. 576–590. ACM (2018)
Huang, Q., et al.: SketchVisor: robust network measurement for software packet processing. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication (SIGCOMM ’17). ACM, New York, NY, USA, pp. 113–126. ACM (2017)
Yang, T., et al.: Elastic sketch: adaptive and fast network-wide measurements. In Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication (SIGCOMM ’18). ACM, New York, NY, USA, pp. 561–575. ACM (2018)
Roy, P., Khan, A., Alonso, G.: Augmented sketch: faster and more accurate stream processing. In Proceedings of the 2016 International Conference on Management of Data (SIGMOD ’16). ACM, New York, NY, USA, 1449–1463. ACM (2016)
Sivaraman, V., Narayana, S., Rottenstreich, O., Muthukrishnan, S., Rexford, J.: Heavy-hitter detection entirely in the data plane. In Proceedings of the Symposium on SDN Research (SOSR ’17). ACM, New York, NY, USA, 164–176. ACM (2017)
Liu, Z., et al.: Nitrosketch: robust and general sketch-based monitoring in software switches. In Proceedings of the ACM Special Interest Group on Data Communication (SIGCOMM ’19). Association for Computing Machinery, New York, NY, USA, pp. 334–350. ACM (2019)
Li, Y., Miao, R., Kim, C., Yu, M.: Flowradar: a better netflow for data centers. In Proceedings of the 13th Usenix Conference on Networked Systems Design and Implementation (NSDI’16). USENIX Association, Berkeley, CA, USA, pp. 311–324. ACM (2016.)
Cormode, G., Muthukrishnan, S.: An improved data stream summary: the count-min sketch and its applications. J. Algorithms. 55(1), 58–75 (2005)
Zhang, Y., Zhu, H., Bao, N., Zhang, L.: Comparative analysis of different sketch methods in practical use. In 2018 Sixth International Conference on Advanced Cloud and Big Data (CBD), Lanzhou, pp. 124–129 (2018)
Yang, T., Gao, S., Sun, Z., Wang, Y., Shen, Y., Li, X.: Diamond sketch: accurate per-flow measurement for big streaming data. IEEE Trans. Parallel Distrib. Syst. 30(12), 2650–2662 (2019)
Liu, Z., Manousis, A., Vorsanger, G., Sekar, V., Braverman, V.: One sketch to rule them all: rethinking network flow monitoring with univmon. In: Proceedings of the2016 ACM SIGCOMM Conference (SIGCOMM ’16). ACM, New York, NY, USA, pp. 101–114. ACM (2016)
Yang, T., et al.: HeavyKeeper: an accurate algorithm for finding top-k elephant flows. IEEE/ACM Trans. Netw. 27(5), 1845–1858 (2019)
Caida Anonymized Internet Traces 2018 Dataset, (2019). http://www.caida.org/data/passive/passive_dataset.xml
Canini, M., Fay, D., Miller, D.J., Moore, A.W., Bolla, R.: Per flow packet sampling for high-speed network monitoring: first international communication systems and networks and workshops. Bangalore 2009, 1–10 (2009)
Kandula, S., Mahajan, R.: Sampling biases in network path measurements and what to do about it. In: Proceedings of the 9th ACM SIGCOMM conference on Internet measurement (IMC ’09). ACM, New York, NY, USA, pp. 156–169. ACM (2009)
Ben Basat, R., Einziger, G., Friedman, R., Luizelli, M.C., Waisbard, E.: Constant time updates in hierarchical heavy hitters. In: Proceedings of the Conference of the ACM Special Interest Group on Data Communication (SIGCOMM ’17). ACM, New York, NY, USA, pp. 127–140. ACM (2017)
Metwally, A., Agrawal, D., El Abbadi, A.: Efficient computation of frequent and top-k elements in data streams. In: Eiter, T., Libkin, L. (eds.) ICDT 2005. LNCS, vol. 3363, pp. 398–412. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-30570-5_27
Manku, G.S., Motwani, R.: Approximate frequency counts over data streams. In: Proceedings of the 28th international conference on Very Large Data Bases (VLDB ’02), VLDB Endowment, pp. 346–357 (2002)
Kumar, A., Sung, M., Xu, J., Wang, J.: Data streaming algorithms for efficient and accurate estimation of flow size distribution. ACM SIGMETRICS Perform. Eval. Rev. 32(1), 177–188 (2004)
Harvey, N.J., Nelson, J., Onak, K.: Sketching and streaming entropy via approximation theory. In: 2008 49th Annual IEEE Symposium on Foundations of Computer Science, pp. 489–498. IEEE (2008)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhao, X., Wen, M., Tang, M., Huang, Q., zhang, C. (2020). Optimized HybridSketch: More Efficient with Analysis and Algorithm. In: Qiu, M. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2020. Lecture Notes in Computer Science(), vol 12452. Springer, Cham. https://doi.org/10.1007/978-3-030-60245-1_42
Download citation
DOI: https://doi.org/10.1007/978-3-030-60245-1_42
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-60244-4
Online ISBN: 978-3-030-60245-1
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)