Skip to main content
Log in

Improved clustering algorithm based on high-speed network data stream

  • Focus
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

With the continuous development of network technology, the attack has become the biggest threat to the stable operation of the network. Intrusion detection technology is a proactive safety protection measure which provides real-time monitoring of internal attacks, external attacks, and misuse. Traditional intrusion detection system is short of adaptability due to the complication and scale of the network. The main problem is that the real-time performance of the network is poor and the reliability is not high. This paper designs the intrusion detection mechanism combined with data stream clustering algorithm and intrusion detection system to solve the problem in processing a large amount of high-speed data streams. The performance of processing data streams is improved through the clustering algorithm based on density and the sliding window and the experiments show that the intrusion detection efficiency is higher than DenStream algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • Barbara D, Daniel J Couto et al (2001) ADAM: a testbed for exploring the use of data mining in intrusion detection. Acm Sigmod Rec 30(4):15–24

    Article  Google Scholar 

  • Bu S, Yu R, Liu P et al (2011) Distributed combined authentication and intrusion detection with data fusion in high-security mobile Ad Hoc networks. IEEE Trans Veh Technol 60(3):1025–1036

    Article  Google Scholar 

  • Cai Y, Xie K, Ma X (2004) An improved DBSCAN algorithm which is insensitive to input parameters. Acta Sci Nat Univ Pekin 40(3):480–486

    Google Scholar 

  • Chen Z, Zhang H, Hatcher W, Nguyen J Yu W (2016) A streaming-based network monitoring and threat detection system. In: IEEE 14th international conference on software engineering research, management and applications (SERA), pp 31–37

  • Cho Y, Moon S (2015) Recommender system using periodicity analysis via mining sequential patterns with time-series and FRAT analysis. J Converg (JoC) 6(1):9–17

    Google Scholar 

  • Dash T (2017) A study on intrusion detection using neural networks trained with evolutionary algorithms. Soft Comput 21(10):2687–2700

    Article  Google Scholar 

  • Gao H, Zhu D, Wang X (2011) A parallel clustering ensemble algorithm for intrusion detection system. Ninth Int Symp Distrib Comput Appl Bus Eng Sci 2011:450–453

    Google Scholar 

  • Gaur M, Pant B (2015) Trusted and secure clustering in mobile pervasive environment. Hum–cent Comput Inform Sci (HCIS) 5:32

    Google Scholar 

  • Gu B, Sheng VS, Tay K, Romano W, Li S (2014) Incremental support vector learning for ordinal regression. IEEE T Neur Net Lear 26(7):1403–1416

    Article  MathSciNet  Google Scholar 

  • Gu B, Sheng VS, Wang Z, Ho D, Osman S, Li S (2015) Incremental learning for \(\nu \)-support vector regression. Neural Netw 67:140–150

    Article  Google Scholar 

  • Keegan N, Ji S, Chaudhary A, Concolato C, Yu B, Jeong D (2016) A survey of cloud-based network intrusion detection analysis. Hum–cent Comput Inform Sci (HCIS) 6:19

    Google Scholar 

  • Kranen P, Assent I, Baldauf C, Seidl T (2009) Self-adaptive anytime stream clustering. In: The 9th IEEE international conference on data mininig (ICDM), pp 249–258

  • Kranen P, Assent I, Baldauf C, Seidl T (2011) The clu-stree: Indexing micro-clusters for anytime stream mining. Knowl Inform Syst 29(2):249–272

    Article  Google Scholar 

  • Lee W, Stolfo S, Mok K (1998) Mining audit data to build intrusion detection models. In: International conference on knowledge discovery & data mining, pp 66–72

  • Liang C, Zhang Y, Shi P et al (2012) Learning very fast decision tree from uncertain data streams with positive and unlabeled samples. Inform Sci 213(23):50–67

    Article  MathSciNet  Google Scholar 

  • Liu X, Xu H, Dong Y, Wang Y, Qian J (2005) Discovering frequent pattern in the data stream. Comput Res Dev 2005(12):2192–2198

    Article  Google Scholar 

  • Rajput R, Mishra A, Kumar S (2014) Optimize intrusion prevention and minimization of threats for stream data classification. Fourth Int Conf Commun Syst Netw Technol 2014:408–413

    Google Scholar 

  • Wang D, Sun J (2010) Research on data mining technology. Stat Decis 2010(7):161–162

    Google Scholar 

  • Wang W, Guyet T, Quiniou R et al (2014) Autonomic intrusion detection: adaptively detecting anomalies over unlabeled audit data streams in computer networks. Knowl-Based Syst 70:103–117

    Article  Google Scholar 

  • Xia L, Jing J (2009) SA-DBSCAN: a self-adaptive density-based clustering algorithm. J Grad Sc Chin Acad Sci 26(4):530–538

    MathSciNet  Google Scholar 

  • Xu K, Kliger M, Hero A III (2014) Adaptive evolutionary clustering. Data Min Knowl Discov 28(2):304–336

    Article  MathSciNet  MATH  Google Scholar 

  • Yang D, Rundensteiner E, Ward M (2012) Shared execution strategy for neighbor-based pattern mining requests over streaming windows. ACM Trans Database Syst 37(1):5

    Article  Google Scholar 

  • Yin C, Ma L, Feng L (2016) A feature selection method for improved clonal algorithm towards intrusion detection. Int J Pattern Recog Artif Intell 30(5):1–14

    Article  Google Scholar 

  • Yin C, Zhang S, Xi J (2016) An improved anonymity model for big data security based on clustering algorithm. Concurr Comput Pract Exp 2016:1–13

    Google Scholar 

  • Yin C, Feng L, Ma L, Kim J, Wang J (2016) An effective feature selection and data-stream classification model HDP. J Internet Technol 17(4):695–702

    Google Scholar 

  • Yu Y, Guo S, Huang H (2007) Abnormal intrusion detection based on data stream. Comput Sci 34(5):66–71

    Article  Google Scholar 

  • Zhen J, Hu M, Yun X (2006) Large scale network anomaly discovery based on data flow method. J Commun 27(2):1–8

    Google Scholar 

  • Zhu C, Dun X, Zhu L (2011) A study on the application of data stream clustering mining through a sliding and damped window to intrusion detection. Fourth Int Conf Inform Comput 2011:22–26

    Google Scholar 

Download references

Acknowledgements

This work was funded by the National Natural Science Foundation of China (61373134, 61402234). It was also supported by the Priority Academic Program Development of Jiangsu Higher Education Institutions (PAPD), Jiangsu Key Laboratory of Meteorological Observation and Information Processing (KDXS1105) and Jiangsu Collaborative Innovation Center on Atmospheric Environment and Equipment Technology (CICAEET). Prof. Ruxia Sun is the corresponding author. We declare that we do not have any conflicts of interest to this work.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Ruxia Sun.

Ethics declarations

Conflict of interest:

Chunyong Yin declares that he has no conflict of interest. Lian Xia declares that she has no conflict of interest. Sun Zhang declares that he has no conflict of interest. Ruxia Sun declares that she has no conflict of interest. Jin Wang declares that he has no conflict of interest.

Ethical approval:

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Communicated by J. Park.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yin, C., Xia, L., Zhang, S. et al. Improved clustering algorithm based on high-speed network data stream. Soft Comput 22, 4185–4195 (2018). https://doi.org/10.1007/s00500-017-2708-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-017-2708-2

Keywords

Navigation