Abstract
Network measurement is critical for many network functions such as detecting network anomalies, accounting, detecting elephant flow and congestion control. Recently, sketch based solutions are widely used for network measurement because of two benefits: high computation efficiency and acceptable error rate. However, there is usually a tradeoff between accuracy and memory cost. To make a reasonable tradeoff, we propose a novel sketch, namely the HBL (Heavy-Buffer-Light) sketch in this paper. The architecture of HBL sketch is three-tier consisting of heavy part, buffer layer and light part, which can be viewed as an improved version of Elastic sketch which is the state-of-the-art in network measurement. Compared to the Elastic sketch and other typical work, HBL sketch can reduce the average relative error rate by 55%–93% with the same memory capacity limitations.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
The caida anonymized internet traces. http://www.caida.org/data/overview./
Cisco netflow. http://www.cisco.com
AlGhadhban, A., Shihada, B.: Flight: a fast and lightweight elephant-flow detection mechanism. In: 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS), pp. 1537–1538. IEEE (2018)
Alipourfard, O., Moshref, M., Zhou, Y., Yang, T., Yu, M.: A comparison of performance and accuracy of measurement algorithms in software. In: Proceedings of the Symposium on SDN Research, p. 18. ACM (2018)
Ben Basat, R., Einziger, G., Friedman, R., Luizelli, M.C., Waisbard, E.: Constant time updates in hierarchical heavy hitters. In: Proceedings of the Conference of the ACM Special Interest Group on Data Communication, pp. 127–140. ACM (2017)
Bloom, B.H.: Space/time trade-offs in hash coding with allowable errors. Commun. ACM 13(7), 422–426 (1970)
Brauckhoff, D., Tellenbach, B., Wagner, A., May, M., Lakhina, A.: Impact of packet sampling on anomaly detection metrics. In: Proceedings of the 6th ACM SIGCOMM Conference on Internet Measurement, pp. 159–164. ACM (2006)
Charikar, M., Chen, K., Farach-Colton, M.: Finding frequent items in data streams. In: Widmayer, P., Eidenbenz, S., Triguero, F., Morales, R., Conejo, R., Hennessy, M. (eds.) ICALP 2002. LNCS, vol. 2380, pp. 693–703. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-45465-9_59
Cormode, G., Muthukrishnan, S.: An improved data stream summary: the count-min sketch and its applications. J. Algorithms 55(1), 58–75 (2005)
Črepinšek, M., Liu, S.H., Mernik, M.: Exploration and exploitation in evolutionary algorithms: a survey. ACM Comput. Surv. 45(3), 35–68 (2013)
Estan, C., Varghese, G.: New directions in traffic measurement and accounting. ACM SIGCOMM Comput. Commun. Rev. 32, 323–336 (2002)
Flajolet, P., Martin, G.N.: Probabilistic counting algorithms for data base applications. J. Comput. Syst. Sci. 31(2), 182–209 (1985)
Gong, J., et al.: HeavyKeeper: an accurate algorithm for finding top-k elephant flows. In: 2018 USENIX Annual Technical Conference (USENIX ATC 2018), pp. 909–921 (2018)
Huang, Q., et al.: SketchVisor: robust network measurement for software packet processing. In: Proceedings of the Conference of the ACM Special Interest Group on Data Communication, pp. 113–126. ACM (2017)
Li, Y., Miao, R., Kim, C., Yu, M.: FlowRadar: a better NetFlow for data centers. In: 13th USENIX Symposium on Networked Systems Design and Implementation (NSDI 2016), pp. 311–324 (2016)
Liu, Z., Manousis, A., Vorsanger, G., Sekar, V., Braverman, V.: One sketch to rule them all: rethinking network flow monitoring with univmon. In: Proceedings of the 2016 ACM SIGCOMM Conference, pp. 101–114. ACM (2016)
Liu, Z., Gao, D., Liu, Y., Zhang, H., Foh, C.H.: An adaptive approach for elephant flow detection with the rapidly changing traffic in data center network. Int. J. Network Manage. 27(6), e1987 (2017)
Mai, J., Chuah, C.N., Sridharan, A., Ye, T., Zang, H.: Is sampled data sufficient for anomaly detection? In: Proceedings of the 6th ACM SIGCOMM Conference on Internet Measurement, pp. 165–176. ACM (2006)
Poupart, P., et al.: Online flow size prediction for improved network routing. In: 2016 IEEE 24th International Conference on Network Protocols (ICNP), pp. 1–6. IEEE (2016)
Przybylski, S., Horowitz, M., Hennessy, J.: Characteristics of performance-optimal multi-level cache hierarchies. In: The 16th Annual International Symposium on Computer Architecture, pp. 114–121. IEEE (1989)
Sivaraman, V., Narayana, S., Rottenstreich, O., Muthukrishnan, S., Rexford, J.: Heavy-hitter detection entirely in the data plane. In: Proceedings of the Symposium on SDN Research, pp. 164–176. ACM (2017)
Wang, M., Li, B., Li, Z.: sFlow: towards resource-efficient and agile service federation in service overlay networks. In: Proceedings of the 24th International Conference on Distributed Computing Systems, pp. 628–635. IEEE (2004)
Wellem, T., Lai, Y.K., Chung, W.Y.: A software defined sketch system for traffic monitoring. In: Proceedings of the Eleventh ACM/IEEE Symposium on Architectures for Networking and Communications Systems, pp. 197–198. IEEE Computer Society (2015)
Wellem, T., Lai, Y.K., Huang, C.Y., Chung, W.Y.: A hardware-accelerated infrastructure for flexible sketch-based network traffic monitoring. In: 2016 IEEE 17th International Conference on High Performance Switching and Routing (HPSR), pp. 162–167. IEEE (2016)
Yang, T., et al.: Elastic sketch: adaptive and fast network-wide measurements. In: Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication, pp. 561–575. ACM (2018)
Yang, T., et al.: Sf-sketch: a fast, accurate, and memory efficient data structure to store frequencies of data items. In: 2017 IEEE 33rd International Conference on Data Engineering (ICDE), pp. 103–106. IEEE (2017)
Yang, T., et al.: Empowering sketches with machine learning for network measurements. In: Proceedings of the 2018 Workshop on Network Meets AI & ML, pp. 15–20. ACM (2018)
Zhou, A., Zhu, H., Liu, L., Zhu, C.: Identification of heavy hitters for network data streams with probabilistic sketch. In: 2018 IEEE 3rd International Conference on Cloud Computing and Big Data Analysis (ICCCBDA), pp. 451–456. IEEE (2018)
Zhou, Y., Jin, H., Liu, P., Zhang, H., Yang, T., Li, X.: Accurate per-flow measurement with bloom sketch. In: IEEE INFOCOM 2018-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), pp. 1–2. IEEE (2018)
Zhou, Y., Liu, P., Jin, H., Yang, T., Dang, S., Li, X.: One memory access sketch: a more accurate and faster sketch for per-flow measurement. In: GLOBECOM 2017–2017 IEEE Global Communications Conference, pp. 1–6. IEEE (2017)
Acknowledgments
This work was supported by the State Key Program of National Natural Science of China (Grant No. 61432002), NSFC Grant Nos. 61772112, U1836214, U1701263, 61672379, and 61751203, and the Science Innovation Foundation of Dalian under Grant 2019J12GX037.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhao, K., Wang, J., Qi, H., Xie, X., Zhou, X., Li, K. (2020). HBL-Sketch: A New Three-Tier Sketch for Accurate Network Measurement. In: Wen, S., Zomaya, A., Yang, L. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2019. Lecture Notes in Computer Science(), vol 11944. Springer, Cham. https://doi.org/10.1007/978-3-030-38991-8_4
Download citation
DOI: https://doi.org/10.1007/978-3-030-38991-8_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-38990-1
Online ISBN: 978-3-030-38991-8
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)