Abstract
Anomaly analysis plays a significant role in building a secure and reliable system. Raw system logs contain important system information, such as execution paths and execution time. People often use system logs for fault diagnosis and root cause localization. However, due to the complexity of raw system logs, these tasks can be arduous and ineffective. To solve this problem, we propose ETGC (Event Topology Graph Construction), a method for mining event topology graph of the normal execution status of systems. ETGC mines the dependency relationship between events and generates the event topology graph based on the maximum spanning tree. We evaluate the proposed method on data sets of real systems to demonstrate the effectiveness of our approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Guo, S., Liu, Z., Chen, W., Li, T.: Event extraction from streaming system logs. Inf. Sci. Appl. 2018, 465–474 (2019)
He, S., Zhu, J., He, P., Lyu, M.R.: Experience report: system log analysis for anomaly detection. In: 2016 IEEE 27th International Symposium on Software Reliability Engineering (ISSRE), pp. 207–218, October 2016
Jia, T., Yang, L., Chen, P., Li, Y., Meng, F., Xu, J.: LogSed: anomaly diagnosis through mining time-weighted control flow graph in logs. In: 2017 IEEE 10th International Conference on Cloud Computing (CLOUD), pp. 447–455 (2017)
Li, T., Liu, Z., Zhou, Q.: Application-driven big data mining. ZTE Technol. J. 22(2), 49–52 (2016)
Lin, Q., Zhang, H., Lou, J., Zhang, Y., Chen, X.: Log clustering based problem identification for online service systems. In: 2016 IEEE/ACM 38th International Conference on Software Engineering Companion (ICSE-C), pp. 102–111 (2016)
Lou, J.G., Fu, Q., Yang, S., Xu, Y., Li, J.: Mining invariants from console logs for system problem detection. In: USENIX Annual Technical Conference, pp. 1–14 (2010)
McDonald, R., Pereira, F., Ribarov, K., Hajič, J.: Non-projective dependency parsing using spanning tree algorithms. In: Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing, pp. 523–530 (2005)
Nagaraj, K., Killian, C., Neville, J.: Structured comparative analysis of systems logs to diagnose performance problems. In: Proceedings of the 9th USENIX Conference on Networked Systems Design and Implementation, p. 26 (2012)
Tak, B.C., Tao, S., Yang, L., Zhu, C., Ruan, Y.: Logan: problem diagnosis in the cloud using log-based reference models. In: 2016 IEEE International Conference on Cloud Engineering (IC2E), pp. 62–67, April 2016
Li, T., Ma, S.: Mining temporal patterns without predefined time windows. In: Fourth IEEE International Conference on Data Mining (ICDM 2004), pp. 451–454, November 2004
Xu, W., Huang, L., Fox, A., Patterson, D., Jordan, M.I.: Detecting large-scale system problems by mining console logs. In: Proceedings of the ACM SIGOPS 22Nd Symposium on Operating Systems Principles, pp. 117–132 (2009)
Yu, X., Joshi, P., Xu, J., Jin, G., Zhang, H., Jiang, G.: Cloudseer: workflow monitoring of cloud infrastructures via interleaved logs. ACM SIGPLAN Not. 51(4), 489–502 (2016)
Acknowledgments
This work is supported in part by Nanjing University of Posts and Telecommunications under Grant No. NY215045 and NY219084, and Shanghai Sailing Program under Grant No. 18YF1423300.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Ling, H., Han, J., Pang, J., Liu, J., Xu, J., Liu, Z. (2019). Behavior Flow Graph Construction from System Logs for Anomaly Analysis. In: Liu, F., Xu, J., Xu, S., Yung, M. (eds) Science of Cyber Security. SciSec 2019. Lecture Notes in Computer Science(), vol 11933. Springer, Cham. https://doi.org/10.1007/978-3-030-34637-9_16
Download citation
DOI: https://doi.org/10.1007/978-3-030-34637-9_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-34636-2
Online ISBN: 978-3-030-34637-9
eBook Packages: Computer ScienceComputer Science (R0)