Skip to main content
SpringerLink
Log in

An Adaptive Method for Identifying Super Nodes from Network-wide View

  • Research
  • Published:
Journal of Network and Systems Management Aims and scope Submit manuscript

Abstract

Super nodes with large cardinalities remain serious threats to production networks. Super node identification is significant for network security and management, including network attacks detection such as DDoS attacks, spam emails, etc. Since the cardinality distribution exhibits dynamic change, most existing approaches are not able to adaptively allocate the memory size for nodes with small and large cardinalities in order to balance accuracy and memory usage in cardinality estimation. Moreover, there are not capable of simultaneously measuring multiple kinds of cardinalities and efficiently recover super nodes due to high calculation and memory cost by constructing data structures only once. To solve these problems, we present a data streaming approach for identifying super nodes based on novel summary data structures. The main idea of our approach is to design a changeable and reversible data structure, which increase its size according to the dynamic cardinality distribution, collect the information associated with cardinalities in network-wide view, and reconstruct super sources and destinations by simple inverse computation based on the aggregated data structure. We perform theoretical analysis and conduct extensive experiments on real network traffic. The experimental results show that the proposed approach can identify up to 96% super nodes with the low memory and computation requirement in comparison with state-of-the-art approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Zuo, Y., Wu, Y., Min, G., Cui, L.: Learning-based network path planning for traffic engineering. Futur. Gener. Comput. Syst. 92, 59–67 (2019). https://doi.org/10.1016/j.future.2018.09.043

    Article  Google Scholar 

  2. Xie, K., Li, X., Wang, X., Cao, J., Xie, G., Wen, J., Zhang, D., Qin, Z.: On-line anomaly detection with high accuracy. IEEE/ACM Trans. Netw. 26(3), 1222–1235 (2018). https://doi.org/10.1109/TNET.2018.2819507

    Article  Google Scholar 

  3. Khan, S., Gani, A., Wahab, A., Shiraz, M., Ahmad, I.: Network forensics: review, taxonomy, and open challenges. J. Netw. Comput. Appl. 66, 214–235 (2016). https://doi.org/10.1016/j.jnca.2016.03.005

    Article  Google Scholar 

  4. Ma, C., Chen, S., Zhang, Y., Xiao, Q., Odegbile, O.O.: Super spreader identification using geometric-min filter. IEEE/ACM Trans. Netw. 30(1), 299–312 (2022). https://doi.org/10.1109/TNET.2021.3108033

    Article  Google Scholar 

  5. Jing, X., Han, H., Yan, Z., Pedrycz, W.: SuperSketch: a multi-dimensional reversible data structure for super host identification. IEEE Trans. Dependable Secur. Comput. 19(4), 2741–2754 (2021). https://doi.org/10.1109/TDSC.2021.3072295

    Article  Google Scholar 

  6. Zheng, L., Liu, D., Liu, W., Liu, Z., Li, Z., Wu, T.: A data streaming algorithm for detection of superpoints with small memory consumption. IEEE Commun. Lett. 21(5), 1067–1070 (2017). https://doi.org/10.1109/LCOMM.2017.2665490

    Article  Google Scholar 

  7. Tang, F., Kawamoto, Y., Kato, N., Yano, K., Suzuki, Y.: Probe delay based adaptive port scanning for IoT devices with private IP address behind NAT. IEEE Netw. 34(2), 195–201 (2020). https://doi.org/10.1109/MNET.001.1900264

    Article  Google Scholar 

  8. Singh, A., Awasthi, A.K., Singh, K., Srivastava, P.: Modeling and analysis of worm propagation in wireless sensor networks. Wirel. Pers. Commun. 98(3), 2535–2551 (2018). https://doi.org/10.1007/s11277-017-4988-3

    Article  Google Scholar 

  9. Faris, H., Ala’M, A., Heidari, A., Aljarah, I., Mafarja, M., Hassonah, M., Fujita, H.: An intelligent system for spam detection and identification of the most relevant features based on evolutionary random weight networks. Inf. Fus. 48, 67–83 (2019). https://doi.org/10.1016/j.inffus.2018.08.002

    Article  Google Scholar 

  10. Xu, Z., Wang, X., Zhang, Y.: Towards persistent detection of DDoS attacks in NDN: A sketch-based approach. IEEE Trans. Dependable Secur. Comput. (2022). https://doi.org/10.1109/TDSC.2022.3196187

    Article  Google Scholar 

  11. Jia, P., Wang, P., Zhang, Y., Zhang, X., Tao, J., Ding, J., Guan, X., Towsley, D.: Accurately estimating user cardinalities and detecting super spreaders over time. IEEE Trans. Knowl. Data Eng. 34(1), 92–106 (2022). https://doi.org/10.1109/TKDE.2020.2975625

    Article  Google Scholar 

  12. Du, Y., Huang, H., Sun, Y., Chen, S., Gao, G., Wang, X., Xu, S. 2022 Short-term memory sampling for spread measurement in high-speed networks. In: Proceedings of the conference on computer communications, London, United Kingdom, pp. 470–479. https://doi.org/10.1109/INFOCOM48880.2022.9796702

  13. Wang, H., Ma, C., Chen, S., Wang, Y.: Fast and accurate cardinality estimation by self-morphing bitmaps. IEEE/ACM Trans. Netw. (2022). https://doi.org/10.1109/TNET.2022.3147204

    Article  Google Scholar 

  14. Xiao, Q., Chen, S., Zhou, Y., Luo, J.: Estimating cardinality for arbitrarily large data stream with improved memory efficiency. IEEE/ACM Trans. Netw. 28(2), 433–446 (2020). https://doi.org/10.1109/TNET.2020.2970860

    Article  Google Scholar 

  15. Bruschi, V., Pontarelli, S., Tollet, J., Barach, D., Bianchi, G.: FlowFight: High performance–low memory top-k spreader detection. Comput. Netw. 196, 108239 (2021). https://doi.org/10.1016/j.comnet.2021.108239

    Article  Google Scholar 

  16. Zhou, A., Qian, J.: An efficient method for detecting supernodes using reversible summary data structures in the distributed monitoring systems. Secur. Commun. Netw. (2022). https://doi.org/10.1155/2022/3271433

    Article  Google Scholar 

  17. Ma, C., Wang, H., Odegbile, O.O., Chen, S.: Virtual filter for non-duplicate sampling. In: Proceedings of the 29th international conference on network protocols, Dallas, TX, USA, 2021, pp. 1–11. https://doi.org/10.1109/ICNP52444.2021.9651974

  18. Yang, T., Zhang, H., Li, J., Gong, J., Uhlig, S., Chen, S., Li, X.: HeavyKeeper: An accurate algorithm for finding top-k elephant flows. IEEE/ACM Trans. Netw. 27(5), 1845–1858 (2019). https://doi.org/10.1109/TNET.2019.2933868

    Article  Google Scholar 

  19. Liu, L., Ding, T., Feng, H., Yan, Z., Lu, X.: Tree sketch: an accurate and memory-efficient sketch for network-wide measurement. Comput. Commun. (2022). https://doi.org/10.1016/j.comcom.2022.07.009

    Article  Google Scholar 

  20. Umer, M., Sher, M., Bi, Y.: Flow-based intrusion detection: Techniques and challenges. Comput. Secur. 70, 238–254 (2017). https://doi.org/10.1016/j.cose.2017.05.009

    Article  Google Scholar 

  21. Du, Y., Huang, H., Sun, Y., Chen, S., Gao, G. 2021 Self-adaptive sampling for network traffic measurement. In: Proceedings of the Conference on Computer Communications, Vancouver, BC, Canada, pp. 1–10. https://doi.org/10.1109/INFOCOM42981.2021.9488425

  22. Venkataraman, S., Song, D., Gibbons, P., Blum, A.: New streaming algorithms for fast detection of superspreaders. In: Proceedings of the network and distributed system security symposium, San Diego, California, USA, 2005, pp. 149–166.

  23. Cao, J., Jin, Y., Chen, A., Bu, T., Zhang, Z.: Identifying high cardinality internet hosts. In: Proceedings of the conference on computer communications, Rio de Janeiro, Brazil, 2009, pp. 810–818. https://doi.org/10.1109/INFCOM.2009.5061990

  24. Huang, H., Sun, Y., Ma, C., Chen, S., Du, Y., Wang, H., Xiao, Q.: Spread estimation with non-duplicate sampling in high-speed networks. IEEE/ACM Trans. Netw. 29(5), 2073–2086 (2021). https://doi.org/10.1109/TNET.2021.3078725

    Article  Google Scholar 

  25. Ma, C., Wang, H., Odegbile, O.O., Chen, S., Melissourgos, D.: Virtual filter for non-duplicate sampling with network applications. IEEE/ACM Trans. Netw. (2022). https://doi.org/10.1109/TNET.2022.3182694

    Article  Google Scholar 

  26. Han, H., Yan, Z., Jing, X., Pedrycz, W.: Applications of sketches in network traffic measurement: a survey. Inf. Fus. 82, 58–85 (2022). https://doi.org/10.1016/j.inffus.2021.12.007

    Article  Google Scholar 

  27. Zhao, Q., Kumar, A., Xu, J. 2005 Joint data streaming and sampling techniques for detection of super sources and destinations. In: Proceedings of the 5th ACM SIGCOMM conference on Internet measurement, Berkeley, CA, USA, 2005, pp. 77–90. https://doi.org/10.5555/1251086.1251093

  28. Yoon, M., Li, T., Chen, S., Peir, J.: Fit a spread estimator in small memory. In: Proceedings of the conference on computer communications, Rio de Janeiro, Brazil, 2009, pp. 504–512. https://doi.org/10.1109/INFCOM.2009.5061956

  29. Yoon, M., Li, T., Chen, S., Peir, J.: Fit a compact spread estimator in small high-speed memory. IEEE/ACM Trans. Netw. 19(5), 1253–1264 (2011). https://doi.org/10.1109/TNET.2010.2080285

    Article  Google Scholar 

  30. Zhao, Q., Xu, J., Kumar, A.: Detection of super sources and destinations in high-speed networks: algorithms, analysis and evaluation. IEEE J. Sel. Areas Commun. 24(10), 1840–1852 (2006). https://doi.org/10.1109/JSAC.2006.877139

    Article  Google Scholar 

  31. Wang, P., Guan, X., Towsley, D., Tao, J.: Virtual indexing based methods for estimating node connection degrees. Comput. Netw. 56(12), 2773–2787 (2012). https://doi.org/10.1016/j.comnet.2012.03.025

    Article  Google Scholar 

  32. Schweller, R., Li, Z., Chen, Y., Gao, Y., Gupta, A., Zhang, Y., Dinda, P., Kao, M., Memik, G.: Reversible sketches: enabling monitoring and analysis over high-speed data streams. IEEE/ACM Trans. Netw. 15(5), 1059–1072 (2007). https://doi.org/10.1109/TNET.2007.896150

    Article  Google Scholar 

  33. Wang, P., Guan, X., Qin, T., Huang, Q.: A data streaming method for monitoring host connection degrees of high-speed links. IEEE Trans. Inf. Foren. Secur. 6(3), 1086–1098 (2011). https://doi.org/10.1109/TIFS.2011.2123094

    Article  Google Scholar 

  34. Liu, W., Qu, W., Gong, J., Li, K.: Detection of superpoints using a vector bloom filter. IEEE Trans. Inf. Foren. Secur. 11(3), 514–527 (2016). https://doi.org/10.1109/TIFS.2015.2503269

    Article  Google Scholar 

  35. Wang, J., Liu, W., Zheng, L., Li, Z., Liu, Z.: A novel algorithm for detecting superpoints based on reversible virtual bitmaps. J. Inf. Secur. Appl. 49, 102403 (2019). https://doi.org/10.1016/j.jisa.2019.102403

    Article  Google Scholar 

  36. Liu, Y., Chen, W., Guan, Y.: Identifying high-cardinality hosts from network-wide traffic measurements. IEEE Trans. Dependable Secur. Comput. 13(5), 547–558 (2016). https://doi.org/10.1109/TDSC.2015.2423675

    Article  Google Scholar 

  37. Tang, L., Huang, Q., Lee, P.: SpreadSketch: Toward invertible and network-wide detection of superspreaders. In: Proceedings of the conference on computer communications, Toronto, ON, Canada, 2020, pp. 1608–1617. https://doi.org/10.1109/INFOCOM41043.2020.9155541

  38. Xiao, Q., Qiao, Y., Zhen, M., Chen, S.: Estimating the persistent spreads in high-speed networks. In: Proceedings of the 22nd international conference on network protocols, Raleigh, NC, USA, 2014, pp. 131–142. https://doi.org/10.1109/ICNP.2014.33

  39. Zhou, Y., Zhou, Y., Chen, M., Chen, S.: Persistent spread measurement for big network data based on register intersection. Proc. ACM Meas. Anal. Comput. Syst. 1(1), 1–29 (2017). https://doi.org/10.1145/3084452

    Article  Google Scholar 

  40. Huang, H., Sun, Y., Chen, S., Tang, S., Han, K., Yuan, J., Yang, W.: You can drop but you can’t hide: K-persistent spread estimation in high-speed networks. In: Proceedings of the conference on computer communications, Honolulu, HI, USA, 2018, pp. 1889–97. https://doi.org/10.1109/INFOCOM.2018.8485998

  41. Huang, H., Sun, Y., Ma, C., Chen, S., Zhou, Y., Yang, W., Tang, S., Xu, H., Qiao, Y.: An efficient k-persistent spread estimator for traffic measurement in high-speed networks. IEEE/ACM Trans. Netw. 28(4), 1463–1476 (2020). https://doi.org/10.1109/TNET.2020.2982003

    Article  Google Scholar 

  42. Jing, X., Yan, Z., Han, H., Pedrycz, W.: ExtendedSketch: Fusing network traffic for super host identification with a memory efficient sketch. IEEE Trans. Dependable Secur. Comput. 19(6), 3913–3924 (2022). https://doi.org/10.1109/TDSC.2021.3111328

    Article  Google Scholar 

  43. Zhang, J., Cui, J., Zhong, H., Chen, Z., Liu, L.: PA-CRT: Chinese remainder theorem based conditional privacy-preserving authentication scheme in vehicular ad-hoc networks. IEEE Trans. Dependable Secur. Comput. 18(2), 722–735 (2021). https://doi.org/10.1109/TDSC.2019.2904274

    Article  Google Scholar 

  44. WIDE. MAWI working group traffic archive. Accessed on Aug. 2022. http://mawi.wide.ad.jp/mawi/.

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China, under Grant No. 61802274, the Open Project Foundation of Key Laboratory of Computer Network and Information Integration (Southeast University), Ministry of Education, China, under Grant No. K93-9-2017-01, the Natural Science Foundation of the Jiangsu Higher Education Institutions of China, under Grant No. 22KJB520036.

Author information

Authors and Affiliations

  1. School of Information Engineering, Taizhou University, Taizhou, China

    Aiping Zhou & Jin Qian

  2. Key Laboratory of Computer Network and Information Integration (Southeast University), Ministry of Education, Nanjing, China

    Aiping Zhou

Authors
  1. Aiping Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jin Qian
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

AZ: methodology and writing of manuscript; JQ: experiment and revising of manuscript.

Corresponding author

Correspondence to Aiping Zhou.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, A., Qian, J. An Adaptive Method for Identifying Super Nodes from Network-wide View. J Netw Syst Manage 31, 51 (2023). https://doi.org/10.1007/s10922-023-09745-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10922-023-09745-0

Keywords

Search

Navigation