Skip to main content
SpringerLink
Log in

A real-time network security visualization system based on incremental learning (ChinaVis 2018)

  • Regular Paper
  • Published:
Journal of Visualization Aims and scope Submit manuscript

Abstract

The real-time analysis of network data is of great significance to network security. Visualization technology and machine learning can assist in network data analysis from different aspects. However, there is little research regarding combining these two methods to process real-time network data. This paper proposes a novel real-time network security system. Combining unsupervised learning and visualization technology, it can identify network behavior patterns and provide a visualization module to adjust models interactively. The system is primarily divided into three parts. In the feature extraction part, we train a deep auto-encoder to compress the feature dimension. In the behavior pattern recognition part, normal and abnormal pattern SOINNs are trained incrementally. In visualization part, analysts can use multiple views to judge recognition results rapidly and adjust models so that the identification accuracy can be increased. We use the data in VAST Challenge 2013 to show that our system can identify network behavior patterns in real time and find the correlations between them.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  • Ali SHA, Ozawa S, Ban T, Nakazato J, Shimamura J (2016) A neural network model for detecting ddos attacks using darknet traffic features. In: Neural networks (IJCNN), 2016 international joint conference on, pp. 2979–2985. IEEE

  • Boschetti A, Salgarelli L, Muelder C, Ma K-L (2011) TVi: a visual querying system for network monitoring and anomaly detection. In: Proceedings of the 8th international symposium on visualization for cyber security, p 1. ACM

  • Bruns-Smith D, Baskaran MM, Ezick J, Henretty T, Lethin R (2016) Cyber security through multidimensional data decompositions. In: Cybersecurity symposium (CYBERSEC), 2016, pp. 59–67. IEEE

  • Buczak AL, Guven E (2016) A survey of data mining and machine learning methods for cyber security intrusion detection. IEEE Commun Surv Tutor 18(2):1153–1176

    Article  Google Scholar 

  • Chen W, Kong F, Mei F, Yuan G, Li B (2017) A novel unsupervised anomaly detection approach for intrusion detection system. In: Big data security on cloud (BigDataSecurity), IEEE international conference on high performance and smart computing (HPSC), and IEEE international conference on intelligent data and security (IDS), 2017 IEEE 3rd international conference on, pp 69–73. IEEE

  • Furao S, Hasegawa O (2006) An incremental network for on-line unsupervised classification and topology learning. Neural Netw 19(1):90–106

    Article  MATH  Google Scholar 

  • Hajar AAS, Fukase K, Ozawa S (2013) A neural network model for large-scale stream data learning using locally sensitive hashing. In: International conference on neural information processing. Springer, Berlin, pp 369–376

  • Hao L, Healey CG, Hutchinson SE (2015) Ensemble visualization for cyber situation awareness of network security data. In: Visualization for cyber security (VizSec), 2015 IEEE symposium on, pp 1–8. IEEE

  • Hinton G E, Salakhutdinov R R (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507

    Article  MathSciNet  MATH  Google Scholar 

  • Huang S-Y, Yu F, Tsaih R-H, Huang Y (2015) Network-traffic anomaly detection with incremental majority learning. In: Neural networks (IJCNN), 2015 international joint conference on, pp. 1–8. IEEE

  • Leban G, Zupan B, Vidmar G, Bratko I (2006) Vizrank: data visualization guided by machine learning. Data Min Knowl Discov 13(2):119–136

    Article  MathSciNet  Google Scholar 

  • Shiravi H, Shiravi A, Ghorbani AA (2012) A survey of visualization systems for network security. IEEE Trans Vis Comput Graph 18(8):1313–1329

    Article  Google Scholar 

  • Sommer R, Paxson V (2010) Outside the closed world: on using machine learning for network intrusion detection. In: Security and privacy (SP), 2010 IEEE symposium on, pp 305–316. IEEE

  • Sultana A, Jabbar M (2016) Intelligent network intrusion detection system using data mining techniques. In: Applied and theoretical computing and communication technology (iCATccT), 2016 2nd international conference on, pp 329–333. IEEE

  • Talbot J, Lee B, Kapoor A, Tan DS (2009) Ensemblematrix: interactive visualization to support machine learning with multiple classifiers. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp 1283–1292. ACM

  • Theron R, Magán-Carrión R, Camacho J, Fernndez GM (2017) Network-wide intrusion detection supported by multivariate analysis and interactive visualization. In: Visualization for cyber security (VizSec), 2017 IEEE symposium on, pp 1–8. IEEE

  • Vast challenge 2013 homepage. http://www.vacommunity.org/VAST+Challenge+2013 (2013)

  • Webb GI, Boughton JR, Wang Z (2005) Not so naive bayes: aggregating one-dependence estimators. Mach Learn 58(1):5–24

    Article  MATH  Google Scholar 

  • Yousefi-Azar M, Varadharajan V, Hamey L, Tupakula U (2017) Autoencoder-based feature learning for cyber security applications. In: Neural networks (IJCNN), 2017 international joint conference on, pp 3854–3861. IEEE

  • Zhang S, Fung C, Huang S, Luan Z, Qian D (2017) Psom: periodic self-organizing maps for unsupervised anomaly detection in periodic time series. In: Quality of service (IWQoS), 2017 IEEE/ACM 25th international symposium on, pp 1–6. IEEE

  • Zhao S, Chandrashekar M, Lee Y, Medhi D (2015) Real-time network anomaly detection system using machine learning. In: Design of reliable communication networks (DRCN), 2015 11th international conference on the, pp 267–270. IEEE

Download references

Acknowledgements

Authors thank Prof. Xiaoru Yuan, Peking university, and unknown reviewers for instruction. This work was supported by National Key Research and Development Program of China (Grant No. 2017YFB0701900), National Nature Science Foundation of China (Grant No. 61100053) and CCF-Venustech Hongyan Research Initiative (2016-013).

Author information

Authors and Affiliations

  1. BASICS, Department of Computer Science and Engineering, School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Shanghai, China

    Xin Fan, Chenlu Li & Xiaoju Dong

Authors
  1. Xin Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chenlu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaoju Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaoju Dong.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 10396 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fan, X., Li, C. & Dong, X. A real-time network security visualization system based on incremental learning (ChinaVis 2018). J Vis 22, 215–229 (2019). https://doi.org/10.1007/s12650-018-0525-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12650-018-0525-z

Keywords

Search

Navigation