Abstract
The slow adoption rate of machine learning-based methods for novel attack detection by Security Operation Centers (SOC) analysts can be partly explained by their lack of data science expertise and the insufficient explainability of the results provided by these approaches. In this paper, we present an anomaly-based detection method that fuses events coming from heterogeneous sources into sets describing the same phenomenons and relies on a deep auto-encoder model to highlight anomalies and their context. To implicate security analysts and benefit from their expertise, we focus on limiting the need of data science knowledge during the configuration phase. Results on a lab environment, monitored using off-the-shelf tools, show good detection performances on several attack scenarios (F1 score \({\approx }0.9\)), and eases the investigation of anomalies by quickly finding similar anomalies through clustering.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Timestamp excepted.
References
Bertero, C., Roy, M., Sauvanaud, C., Trédan, G.: Experience report: log mining using natural language processing and application to anomaly detection. In: 2017 IEEE 28th International Symposium on Software Reliability Engineering (ISSRE), pp. 351–360. IEEE (2017)
Brogi, G., Tong, V.V.T.: TerminAPTor: highlighting advanced persistent threats through information flow tracking. In: 2016 8th IFIP International Conference on New Technologies, Mobility and Security (NTMS), pp. 1–5. IEEE (2016)
de Masson d’Autume, C., Ruder, S., Kong, L., Yogatama, D.: Episodic memory in lifelong language learning. CoRR abs/1906.01076 (2019)
Debnath, B., et al.: LogLens: a real-time log analysis system. In: 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS), pp. 1052–1062. IEEE (2018)
Dey, A., Totel, E., Navers, S.: Heterogeneous security events prioritization using auto-encoders. In: Garcia-Alfaro, J., Leneutre, J., Cuppens, N., Yaich, R. (eds.) CRiSIS 2020. LNCS, vol. 12528, pp. 164–180. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-68887-5_10
Ding, Z., Fei, M.: An anomaly detection approach based on isolation forest algorithm for streaming data using sliding window. IFAC Proceedings Volumes 46, 12–17 (2013)
Du, M., Li, F., Zheng, G., Srikumar, V.: DeepLog: anomaly detection and diagnosis from system logs through deep learning. In: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pp. 1285–1298. ACM (2017)
Elastic. Elastic common schema. https://github.com/elastic/ecs. Accessed 25 Mar 2021
Ester, M., Kriegel, H.-P., Sander, J., Xu, X., et al.: A density-based algorithm for discovering clusters in large spatial databases with noise. In: KDD, vol. 96, pp. 226–231 (1996)
Hassan, W.U., Bates, A., Marino, D.: Tactical provenance analysis for endpoint detection and response systems. In: 2020 IEEE Symposium on Security and Privacy (SP), pp. 1172–1189. IEEE (2020)
He, S., Zhu, J., He, P., Lyu, M.R.: Experience report: system log analysis for anomaly detection. In: 2016 IEEE 27th International Symposium on Software Reliability Engineering (ISSRE), pp. 207–218. IEEE (2016)
Kirkpatrick, J., et al.: Overcoming catastrophic forgetting in neural networks. Proc. Natl. Acad. Sci. 114, 3521–3526 (2017)
Leichtnam, L., Totel, E., Prigent, N., Mé, L.: Forensic analysis of network attacks: restructuring security events as graphs and identifying strongly connected sub-graphs. In: 2020 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW), pp. 565–573. IEEE (2020)
Liu, F., Wen, Y., Zhang, D., Jiang, X., Xing, X., Meng, D.: Log2vec: a heterogeneous graph embedding based approach for detecting cyber threats within enterprise. In: Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, pp. 1777–1794 (2019)
Liu, F.T., Ting, K.M., Zhou, Z.-H.: Isolation forest. In: 2008 Eighth IEEE International Conference on Data Mining, pp. 413–422. IEEE (2008)
Milajerdi, S.M., Gjomemo, R., Eshete, B., Sekar, R., Venkatakrishnan, V.: HOLMES: real-time apt detection through correlation of suspicious information flows. In: 2019 IEEE Symposium on Security and Privacy (SP), pp. 1137–1152. IEEE (2019)
Mirsky, Y., Doitshman, T., Elovici, Y., Shabtai, A.: Kitsune: an ensemble of autoencoders for online network intrusion detection. arXiv preprint arXiv:1802.09089 (2018)
MITRE. Att&ck data sources. https://github.com/mitre-attack/attack-datasources. Accessed 16 Mar 2021
Pascoal, C., De Oliveira, M.R., Valadas, R., Filzmoser, P., Salvador, P., Pacheco, A.: Robust feature selection and robust PCA for internet traffic anomaly detection. In: 2012 Proceedings IEEE INFOCOM, pp. 1755–1763. IEEE (2012)
Paxson, V.: Bro: a system for detecting network intruders in real-time. Comput. Netw. 31, 2435–2463 (1999)
Pei, K., et al.: HERCULE: attack story reconstruction via community discovery on correlated log graph. In: Proceedings of the Annual Computer Security Applications Conference (ACSAC). ACM (2016)
Rolnick, D., Ahuja, A., Schwarz, J., Lillicrap, T.P., Wayne, G.: Experience replay for continual learning. CoRR abs/1811.11682 (2018)
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning internal representations by error propagation. Technical report, California Univ San Diego La Jolla Inst for Cognitive Science (1985)
Sadoddin, R., Ghorbani, A.A.: An incremental frequent structure mining framework for real-time alert correlation. Comput. Secur. 28, 153–173 (2009)
Sprechmann, P., et al.: Memory-based parameter adaptation (2018)
Valeur, F.: Real-time intrusion detection alert correlation. Citeseer (2006)
Veeramachaneni, K., Arnaldo, I., Korrapati, V., Bassias, C., Li, K.: Ai2: training a big data machine to defend. In: 2016 IEEE 2nd International Conference on Big Data Security on Cloud (BigDataSecurity), IEEE International Conference on High Performance and Smart Computing (HPSC), and IEEE International Conference on Intelligent Data and Security (IDS), pp. 49–54. IEEE (2016)
Viinikka, J., Debar, H., Mé, L., Lehikoinen, A., Tarvainen, M.: Processing intrusion detection alert aggregates with time series modeling. Inf. Fusion 10, 312–324 (2009)
Xosanavongsa, C., Totel, E., Bettan, O.: Discovering correlations: a formal definition of causal dependency among heterogeneous events. In: 2019 IEEE European Symposium on Security and Privacy (EuroS P), pp. 340–355 (2019)
Zhou, C., Paffenroth, R.C.: Anomaly detection with robust deep autoencoders. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 665–674. ACM (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 IFIP International Federation for Information Processing
About this paper
Cite this paper
Dey, A., Totel, E., Costé, B. (2022). DAEMON: Dynamic Auto-encoders for Contextualised Anomaly Detection Applied to Security MONitoring. In: Meng, W., Fischer-Hübner, S., Jensen, C.D. (eds) ICT Systems Security and Privacy Protection. SEC 2022. IFIP Advances in Information and Communication Technology, vol 648. Springer, Cham. https://doi.org/10.1007/978-3-031-06975-8_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-06975-8_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-06974-1
Online ISBN: 978-3-031-06975-8
eBook Packages: Computer ScienceComputer Science (R0)