Abstract
To perceive the overall cyber security situation of the target network, cyberspace situational awareness platforms collect billions of alerts from IDS, IPS, firewalls, probes, and third-party systems in real-time. It is urgent to aggregate these multi-source, massive, real-time, heterogeneous, dynamic, strong timeliness alerts to help analysts find valuable clues as quickly as possible. In this paper, we propose a novel real-time alert aggregation approach. It is based on a dynamic attack granularity graph model combined with a dynamic threshold update algorithm, which not only can effectively solve the redundancy problem of massive multi-source data, but also can extract valuable alert aggregation from the data flood. To evaluate our approach, we conduct experiments using the public datasets Suricata (81600 alerts) and a real 24-hour online dataset (1204753 alerts). We use evaluation metrics including Aggregation Rate(AR), Simplicity Metric(SM) and Time Delay(TD), and select three common alert aggregation algorithms to perform a comparative test in a simulated real-time situation. The experiment shows that our approach achieves more than 98% aggregation rate, reduces data complexity by more than 82%, and has stronger robustness.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Albasheer, H., et al.: Cyber-attack prediction based on network intrusion detection systems for alert correlation techniques: a survey. Sensors 22(4), 1494 (2022). https://doi.org/10.3390/s22041494
Valdes, A., Skinner, K.: Probabilistic alert correlation. In: Lee, W., Mé, L., Wespi, A. (eds.) RAID 2001. LNCS, vol. 2212, pp. 54–68. Springer, Heidelberg (2001). https://doi.org/10.1007/3-540-45474-8_4
de Alvarenga, S.C., Barbon, S., Miani, R.S., Cukier, M., Zarpelão, B.B.: Process mining and hierarchical clustering to help intrusion alert visualization. Comput. Secur. 73, 474–491 (2018). https://doi.org/10.1016/j.cose.2017.11.021
Carbone, P., Katsifodimos, A., Ewen, S., Markl, V., Haridi, S., Tzoumas, K.: Apache flink: stream and batch processing in a single engine. IEEE Data Eng. Bull. 38(4), 28–38 (2015). http://sites.computer.org/debull/A15dec/p28.pdf
Cheung, S., Lindqvist, U., Fong, M.W.: Modeling multistep cyber attacks for scenario recognition. In: 3rd DARPA Information Survivability Conference and Exposition (DISCEX-III 2003), Washington, DC, USA, 22–24 April 2003, pp. 284–292. IEEE Computer Society (2003). https://doi.org/10.1109/DISCEX.2003.1194892
Cuppens, F.: Managing alerts in a multi-intrusion detection environment. In: 17th Annual Computer Security Applications Conference (ACSAC 2001), New Orleans, Louisiana, USA, 11–14 December 2001, pp. 22–31. IEEE Computer Society (2001). https://doi.org/10.1109/ACSAC.2001.991518
Debar, H., Wespi, A.: Aggregation and correlation of intrusion-detection alerts. In: Lee, W., Mé, L., Wespi, A. (eds.) RAID 2001. LNCS, vol. 2212, pp. 85–103. Springer, Heidelberg (2001). https://doi.org/10.1007/3-540-45474-8_6
Fatma, H., Mohamed, L.: A two-stage technique to improve intrusion detection systems based on data mining algorithms. In: 2013 5th International Conference on Modeling, Simulation and Applied Optimization (ICMSAO), pp. 1–6 (2013). https://doi.org/10.1109/ICMSAO.2013.6552542
Husák, M., Cermák, M., Lastovicka, M., Vykopal, J.: Exchanging security events: which and how many alerts can we aggregate? In: 2017 IFIP/IEEE Symposium on Integrated Network and Service Management (IM), Lisbon, Portugal, 8–12 May 2017, pp. 604–607. IEEE (2017). https://doi.org/10.23919/INM.2017.7987340
Wang, J.-X., Wang, Z.-Y., Dai, K.: A PCA-LVQ model for intrusion alert analysis. In: Mehrotra, S., Zeng, D.D., Chen, H., Thuraisingham, B., Wang, F.-Y. (eds.) ISI 2006. LNCS, vol. 3975, pp. 715–716. Springer, Heidelberg (2006). https://doi.org/10.1007/11760146_102
Julisch, K.: Mining alarm clusters to improve alarm handling efficiency. In: Seventeenth Annual Computer Security Applications Conference, pp. 12–21 (2001)
Julisch, K.: Info, claims: clustering intrusion detection alarms to support root cause analysis. ACM Trans. Inf. Syst. Secur. 6, 443–471 (2003). https://doi.org/10.1145/950191.950192
Kumar, M., Siddique, S., Noor, H.: Feature-based alert correlation in security systems using self organizing maps. In: Dasarathy, B.V. (ed.) Data Mining, Intrusion Detection, Information Security and Assurance, and Data Networks Security, Orlando, Florida, USA, 13 April 2009. SPIE Proceedings, vol. 7344, p. 734404. SPIE (2009). https://doi.org/10.1117/12.820000
Man, D., Yang, W., Wang, W., Xuan, S.: An alert aggregation algorithm based on iterative self-organization. Procedia Eng. 29, 3033–3038 (2012). https://doi.org/10.1016/j.proeng.2012.01.435. https://www.sciencedirect.com/science/article/pii/S1877705812004456. 2012 International Workshop on Information and Electronics Engineering
Mohamed, A.B., Idris, N.B., Shanmugum, B.: Alert correlation using a novel clustering approach. 2212(12747443), 720–725 (2012). https://doi.org/10.1109/CSNT.2012.212
Nandimath, J., Banerjee, E., Patil, A., Kakade, P., Vaidya, S.: Big data analysis using apache hadoop. In: IEEE 14th International Conference on Information Reuse & Integration, IRI 2013, San Francisco, CA, USA, 14–16 August 2013, pp. 700–703. IEEE Computer Society (2013). https://doi.org/10.1109/IRI.2013.6642536
Noac’h, P.L., Costan, A., Bougé, L.: A performance evaluation of Apache Kafka in support of big data streaming applications. In: Nie, J., et al. (eds.) 2017 IEEE International Conference on Big Data (IEEE BigData 2017), Boston, MA, USA, 11–14 December 2017, pp. 4803–4806. IEEE Computer Society (2017). https://doi.org/10.1109/BigData.2017.8258548
Raftopoulos, E., Dimitropoulos, X.A.: IDS alert correlation in the wild with edge. IEEE J. Sel. Areas Commun. 32(10), 1933–1946 (2014). https://doi.org/10.1109/JSAC.2014.2358834
Benferhat, S., Boudjelida, A., Tabia, K., Drias, H.: An intrusion detection and alert correlation approach based on revising probabilistic classifiers using expert knowledge. Appl. Intell. 38(15), 520–540 (2013). https://doi.org/10.1007/s10489-012-0383-7
Suricata: Suricata open source IDS (2020). https://suricata-ids.org/
Tan, T.K., Darken, C.J.: Learning and prediction of relational time series. Comput. Math. Organ. Theory 21(2), 210–241 (2015). https://doi.org/10.1007/s10588-015-9182-0
Templeton, S.J., Levitt, K.E.: A requires/provides model for computer attacks. In: Zurko, M.E., Greenwald, S.J. (eds.) Proceedings of the 2000 Workshop on New Security Paradigms, Ballycotton, Co., Cork, Ireland, 18–21 September 2000, pp. 31–38. ACM (2000). https://doi.org/10.1145/366173.366187
Tjhai, G.C., Furnell, S., Papadaki, M., Clarke, N.L.: A preliminary two-stage alarm correlation and filtering system using SOM neural network and k-means algorithm. Comput. Secur. 29(6), 712–723 (2010)
Zhang, Y., Huang, S., Wang, Y.: IDS alert classification model construction using decision support techniques. In: 2012 International Conference on Computer Science and Electronics Engineering, vol. 1, pp. 301–305 (2012). https://doi.org/10.1109/ICCSEE.2012.242
Zhao, N., et al.: Understanding and handling alert storm for online service systems. In: Rothermel, G., Bae, D. (eds.) ICSE-SEIP 2020: 42nd International Conference on Software Engineering, Software Engineering in Practice, Seoul, South Korea, 27 June–19 July 2020, pp. 162–171. ACM (2020). https://doi.org/10.1145/3377813.3381363
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, H. et al. (2023). Real-Time Aggregation for Massive Alerts Based on Dynamic Attack Granularity Graph. In: Yung, M., Chen, C., Meng, W. (eds) Science of Cyber Security . SciSec 2023. Lecture Notes in Computer Science, vol 14299. Springer, Cham. https://doi.org/10.1007/978-3-031-45933-7_14
Download citation
DOI: https://doi.org/10.1007/978-3-031-45933-7_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-45932-0
Online ISBN: 978-3-031-45933-7
eBook Packages: Computer ScienceComputer Science (R0)