Abstract
Heavy hitters can provide an important indicator for detecting abnormal network events. Most of existing algorithms for heavy hitter identification are implemented to deal with static datasets generated within a fixed time frame, lacking the ability to handle the latest arrivals of data streams adaptively. Considering the rigid demand for accurate and fast detection of outlier events in some networks like Smart Grids, these existing algorithms are not suitable to be deployed straightforward. To this end, this paper presents a new algorithm called D2Sketch for efficient heavy hitter identification over an adaptive sliding window for flexible dataset input. D2Sketch provides a novel framework that combines the Count-Min Sketch to get the connection degree of each host, with the stream-summary structure of Space Saving algorithm to get a more accurate list of Top-K heavy hitters. Moreover, it can adjust its measurement window to the most recent datasets automatically. Extensive experimental results show that the D2Sketch algorithm outperforms the related algorithm in terms of false positive rate, ordering deviation and estimate error.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Goel, S.: Anonymity vs. security: the right balance for the smart grid. Commun. Assoc. Inf. Syst. 36(1) (2015). Article 2
Zhao, Q., Kumar, A., Xu, J.: Joint data streaming and sampling techniques for detection of super sources and destinations. In: IMC. ACM Press, Berkeley, pp. 77–90 (2005)
Kompella, R.R., Singh, S., Varghese, G.: On scalable attack detection in the network. In: Proceedings of the 4th ACM SIGCOMM Conference on Internet Measurement, pp. 187–200 (2004)
Venkataraman, S., Song, D., Gibbons, P.B., Blum, A.: New streaming algorithms for fast detection of superspreaders. In: Proceedings of the 12th ISOC Symposium on Network and Distributed Systems Security (SNDSS), pp. 149–166 (2005)
Estan, C., Varghese, G., Fisk, M.: Bitmap algorithms for counting active flows on high speed links. In: ACM SIGCOMM Internet Measurement Workshop (2003)
Wang, P., Guan, X., Gong, W., Towsley., D.F.: A new virtual indexing method for measuring host connection degrees. In: INFOCOM 2011, pp. 156–160 (2011)
Metwally, A., Agrawal, D., Abbadi, A.E.: Efficient computation of frequent and Top-k elements in data streams. In: Proceedings of 10th International Conference on Database Theory (ICDT 2005), pp. 398–412 (2005)
Liu, J., Xiao, Y., Li, S., et al.: Cyber security and privacy issues in smart grids. IEEE Commun. Surv. Tutorials 14(4), 981–997 (2012)
Marques, C., Ribeiro, M., Duque, C., Ribeiro, P., Da Silva, E.A.B.: A controlled filtering method for estimating harmonics of off-nominal frequencies. IEEE Trans. Smart Grid 3(1), 38–49 (2012)
Roesch, M.: Snort–lightweight intrusion detection for networks. In: Proceedings of the USENIX LISA Conference on System Administration 1999, Seattle, WA, pp. 229–238 (1999)
Plonka, D.: Flowscan: a network traffic flow reporting and visualization tool. In: Proceedings of USENIX LISA 2000, New Orleans, LA, pp. 305–317 (2000)
Cormode, G., Muthukrishnan, S.: An improved data stream summary: the count-min sketch and its applications. In: Farach-Colton, M. (ed.) LATIN 2004. LNCS, vol. 2976, pp. 29–38. Springer, Heidelberg (2004). doi:10.1007/978-3-540-24698-5_7
Homem, N., Carvalho, J.P.: Finding top-k elements in a time-sliding window. Evolving Syst. 2(1), 51–70 (2011)
Zhang, Z., Wang, B., Lan, J.: Identifying elephant flows in internet backbone traffic with bloom filters and LRU. Comput. Commun. 61, 70–78 (2015)
Cormode, G., Hadjieleftheriou, M.: Methods for finding frequent items in data streams. VLDB J. 19(1), 3–20 (2010)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Tang, H., Wu, Y., Li, T., Shi, H., Ge, J. (2017). D2Sketch: Supporting Efficient Identification of Heavy Hitters Over Sliding Windows. In: Hu, J., Leung, V., Yang, K., Zhang, Y., Gao, J., Yang, S. (eds) Smart Grid Inspired Future Technologies. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 175. Springer, Cham. https://doi.org/10.1007/978-3-319-47729-9_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-47729-9_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-47728-2
Online ISBN: 978-3-319-47729-9
eBook Packages: Computer ScienceComputer Science (R0)