Fangrong Wang
ABSTRACT
Logs record system events and status, which help developers and system administrators diagnose run time errors, monitor running status and mine operation patterns [13, 23]. However, logs are complex and weakly linked, making it difficult to diagnose the causes of failures. While recent studies on log knowledge extraction focus on lifting entities from log messages for enriching a background knowledge graph (BKG), they do not involve knowledge reasoning for inferring implicit relations nor guarantee that the knowledge learned from log streams is consistent with the background knowledge. In this preliminary research paper, we present a log extraction approach to log knowledge graph (KG) construction. It includes a novel strategy that utilizes inference rules from a background knowledge graph to learn new triples and validate triples. Also, it implements a local to global strategy to perform reasoning on temporary log instance graphs (LIGs) then on the extended BKG, which significantly reduces query space. Finally we demonstrate the applicability of this approach by a use case in the context of root cause analysis.
- Naser Ahmadi, Viet Phi Huynh, Vamsi Meduri, Stefano Ortona, and Paolo Papotti. 2020. Mining Expressive Rules in Knowledge Graphs. Journal of Data and Information Quality 12, 2 (2020). https://doi.org/10.1145/3371315Google Scholar
Digital Library
- Álvaro Brandón, Marc Solé, Alberto Huélamo, David Solans, María S. Pérez, and Victor Muntés-Mulero. 2020. Graph-based root cause analysis for service-oriented and microservice architectures. Journal of Systems and Software 159 (2020). https://doi.org/10.1016/j.jss.2019.110432Google ScholarDigital Library
- Harith A. Dawood. 2014. Graph theory and cyber security. Proceedings - 3rd International Conference on Advanced Computer Science Applications and Technologies, ACSAT 2014(2014), 90–96. https://doi.org/10.1109/ACSAT.2014.23Google ScholarDigital Library
- Youcef Djenouri, Asma Belhadi, and Philippe Fournier-Viger. 2018. Extracting useful knowledge from event logs: A frequent itemset mining approach. Knowledge-Based Systems 139 (2018), 132–148. https://doi.org/10.1016/j.knosys.2017.10.016Google ScholarDigital Library
- Hristo Djidjev, Gary Sandine, Curtis B. Storlie, and Scott Vander Wiel. 2011. Graph Based Statistical Analysis of Network Traffic. Mlg ’11 (2011), 8. https://www.cs.purdue.edu/mlg2011/papers/paper_10.pdfGoogle Scholar
- Min Du, Feifei Li, Guineng Zheng, and Vivek Srikumar. 2017. DeepLog: Anomaly detection and diagnosis from system logs through deep learning. Proceedings of the ACM Conference on Computer and Communications Security (2017), 1285–1298. https://doi.org/10.1145/3133956.3134015Google ScholarDigital Library
- Andreas Ekelhart, Fajar J. Ekaputra, and Elmar Kiesling. 2021. The SLOGERT Framework for Automated Log Knowledge Graph Construction. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 12731 LNCS (2021), 631–646. https://doi.org/10.1007/978-3-030-77385-4_38Google ScholarDigital Library
- Andreas Ekelhart, Elmar Kiesling, and Kabul Kurniawan. 2018. Taming the logs – Vocabularies for semantic security analysis. Procedia Computer Science 137 (2018), 109–119. https://doi.org/10.1016/j.procs.2018.09.011Google ScholarCross Ref
- Alexandre P. Francisco, Ricardo Baeza-Yates, and Arlindo L. Oliveira. 2012. Mining query log graphs towards a query folksonomy. Concurrency and Computation: Practice and Experience 24, 17(2012), 2179–2192. https://doi.org/10.1002/cpe.1773Google ScholarDigital Library
- Luis Galárraga, Christina Teflioudi, Katja Hose, and Fabian M. Suchanek. 2015. Fast rule mining in ontological knowledge bases with AMIE+. VLDB Journal 24, 6 (2015), 707–730. https://doi.org/10.1007/s00778-015-0394-1Google ScholarDigital Library
- Luis Antonio Galárraga, Christina Teflioudi, Katja Hose, and Fabian Suchanek. 2013. AMIE: Association Rule Mining under Incomplete Evidence in Ontological Knowledge Bases. (2013), 413–422. https://doi.org/10.1145/2488388.2488425Google ScholarDigital Library
- Víctor Gutiérrez-Basulto, Jean Christoph Jung, Carsten Lutz, and Lutz Schröder. 2017. Probabilistic description logics for subjective uncertainty. Journal of Artificial Intelligence Research 58(1) (2017), 1–66.Google Scholar
- Pinjia He, Jieming Zhu, Shilin He, Jian Li, and Michael R. Lyu. 2016. An evaluation study on log parsing and its use in log mining. Proceedings - 46th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2016(2016), 654–661. https://doi.org/10.1109/DSN.2016.66Google ScholarCross Ref
- Aidan Hogan. 2020. Shape Constraints and Expressions. In The Web of Data. Springer, 449–513.Google Scholar
- Elmar Kiesling, Andreas Ekelhart, Kabul Kurniawan, and Fajar Ekaputra. 2019. The SEPSES Knowledge Graph: An Integrated Resource for Cybersecurity. Vol. 11779 LNCS. Springer International Publishing. 198–214 pages. https://doi.org/10.1007/978-3-030-30796-7_13Google ScholarDigital Library
- Jonathan Lajus, Luis Galárraga, and Fabian Suchanek. 2020. Fast and Exact Rule Mining with AMIE 3. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 12123 LNCS (2020), 36–52. https://doi.org/10.1007/978-3-030-49461-2_3Google ScholarDigital Library
- Xue Li, Alan Bundy, and Eugene Philalithis. 2021. Signature Entrenchment and Conceptual Changes in Automated Theory Repair. In The Ninth Annual Conference on Advances in Cognitive Systems. Cognitive Systems Foundation.Google Scholar
- Xue Li, Alan Bundy, and Alan Smaill. 2018. ABC repair system for datalog-like theories. IC3K 2018 - Proceedings of the 10th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management 2 (2018), 335–342. https://doi.org/10.5220/0006959703350342Google Scholar
- Jian Guang Lou, Qiang Fu, Yi Wang, and Jiang Li. 2010. Mining dependency in distributed systems through unstructured logs analysis. Operating Systems Review (ACM) 44, 1 (2010), 91–96. https://doi.org/10.1145/1740390.1740411Google ScholarDigital Library
- Ravi Lourdusamy and Stanislaus Abraham. 2020. A Survey on Methods of Ontology Learning from Text. May (2020), 113–123. https://doi.org/10.1007/978-3-030-38501-9_11Google Scholar
- Thomas Lukasiewicz. 2008. Expressive probabilistic description logics. Artificial Intelligence 172(6-7) (2008), 852–883.Google Scholar
- Weibin Meng, Ying Liu, Yichen Zhu, Shenglin Zhang, Dan Pei, Yuqing Liu, Yihao Chen, Ruizhi Zhang, Shimin Tao, Pei Sun, and Rong Zhou. 2019. Loganomaly: Unsupervised detection of sequential and quantitative anomalies in unstructured logs. IJCAI International Joint Conference on Artificial Intelligence 2019-Augus(2019), 4739–4745. https://doi.org/10.24963/ijcai.2019/658Google ScholarCross Ref
- Adam Oliner, Archana Ganapathi, and Wei Xu. 2012. Advances and challenges in log analysis. Commun. ACM 55, 2 (feb 2012), 55–61. https://doi.org/10.1145/2076450.2076466Google ScholarDigital Library
- Stefano Ortona, Venkata Vamsikrishna Meduri, and Paolo Papotti. 2018. Robust discovery of positive and negative rules in knowledge bases. Proceedings - IEEE 34th International Conference on Data Engineering, ICDE 2018 (2018), 1180–1191. https://doi.org/10.1109/ICDE.2018.00108Google ScholarCross Ref
- J.Z. Pan, D. Calvanese, T. Eiter, I. Horrocks, M. Kifer, F. Lin, and Y. Zhao. 2017. Reasoning Web: Logical Foundation of Knowledge Graph Construction and Querying Answering. Springer.Google ScholarCross Ref
- J.Z. Pan, G. Vetere, J.M. Gomez-Perez, and H. Wu. 2016. Exploiting Linked Data and Knowledge Graphs for Large Organisations. Springer.Google Scholar
- Jeff Z. Pan and Ian Horrocks. 2002. Reasoning in the SHOQ(Dn) Description Logic. In Proceedings of the 2002 International Workshop on Description Logics (DL2002), Ian Horrocks and Sergio Tessaris (Eds.).Google Scholar
- Jeff Z. Pan and Ian Horrocks. 2003. Web Ontology Reasoning with Datatype Groups. In Proc. of the International Semantic Web Conference. 47–63.Google Scholar
- Jeff Z. Pan, Giorgos B. Stamou, Vassilis Tzouvaras, and Ian Horrocks. 2005. f-SWRL: A Fuzzy Extension of SWRL. In Artificial Neural Networks: Formal Models and Their Applications - ICANN. 829–834.Google Scholar
- Jeff Z. Pan, Giorgos Stoilos, Giorgos Stamou, Vassilis Tzouvaras, and Ian Horrocks. 2006. f-SWRL: A Fuzzy Extension of SWRL. Journal of Data Semantic(2006), 28–46.Google Scholar
- Antonio Pecchia, Ingo Weber, Marcello Cinque, and Yu Ma. 2020. Discovering process models for the analysis of application failures under uncertainty of event logs. Knowledge-Based Systems 189 (2020). https://doi.org/10.1016/j.knosys.2019.105054Google ScholarDigital Library
- Kexin Pei, Zhongshu Gu, Brendan Saltaformaggio, Shiqing Ma, Fei Wang, Zhiwei Zhang, Luo Si, Xiangyu Zhang, and Dongyan Xu. 2016. HERCULE: Attack story reconstruction via community discovery on correlated log graph. ACM International Conference Proceeding Series 5-9-Decemb, 3(2016), 583–595. https://doi.org/10.1145/2991079.2991122Google ScholarDigital Library
- Aditya Pingle, Aritran Piplai, Sudip Mittal, Anupam Joshi, James Holt, and Richard Zak. 2019. Relext: Relation extraction using deep learning approaches for cybersecurity knowledge graph improvement. Proceedings of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2019 (2019), 879–886. https://doi.org/10.1145/3341161.3343519 arxiv:1905.02497Google ScholarDigital Library
- Guilin Qi, Qiu Ji, Jeff Z. Pan, and Jianfeng Du. 2011. Extending Description Logics with Uncertainty Reasoning in Possibilistic Logic. International Journal of Intelligent Systems 26(4) (2011).Google ScholarDigital Library
- Guilin Qi, Jeff Z. Pan, and Qiu Ji. 2007. A Possibilistic Extension of Description Logics. In Proc. of 2007 International Workshop on Description Logics (DL2007).Google Scholar
- Wilhelm Schickard-lnstitute, Ulrich Gntzer, Wilhelm Schickard-lnstitute, Daimlerchrysler Ag, and F T Ad. [n. d.]. Algorithms for Association Rule Mining - A General Survey and Comparison. 2, 1 ([n. d.]), 58–64.Google Scholar
- Murat Sensoy, Achille Fokoue, Jeff Z. Pan, Timothy J. Norman, Yuqing Tang, Nir Oren, and Katia P. Sycara. 2013. Reasoning about uncertain information and conflict resolution through trust revision. In Proc. of the International conference on Autonomous Agents and Multi-Agent Systems, AAMAS, Maria L. Gini, Onn Shehory, Takayuki Ito, and Catholijn M. Jonker (Eds.). 837–844.Google Scholar
- Evren Sirin, Bijan Parsia, Bernardo Cuenca Grau, Aditya Kalyanpur, and Yarden Katz. 2007. Pellet: A practical OWL-DL reasoner. Web Semantics 5, 2 (2007), 51–53. https://doi.org/10.1016/j.websem.2007.03.004Google ScholarDigital Library
- Giorgos Stoilos, Giorgos B. Stamou, and Jeff Z. Pan. 2006. Handling Imprecise Knowledge with Fuzzy Description Logic. In the Proc. of the 2006 International Workshop on Description Logics (DL2006).Google Scholar
- Hudan Studiawan, Christian Payne, and Ferdous Sohel. 2017. Graph clustering and anomaly detection of access control log for forensic purposes. Digital Investigation 21(2017), 76–87. https://doi.org/10.1016/j.diin.2017.05.001Google ScholarCross Ref
- Jan Svacina, Jackson Raffety, Connor Woodahl, Brooklynn Stone, Tomas Cerny, Miroslav Bures, Dongwan Shin, Karel Frajtak, and Pavel Tisnovsky. 2020. On Vulnerability and Security Log analysis: A Systematic Literature Review on Recent Trends. ACM International Conference Proceeding Series (2020), 175–180. https://doi.org/10.1145/3400286.3418261Google ScholarDigital Library
- Ding Yuan, Haohui Mai, Weiwei Xiong, Lin Tan, Yuanyuan Zhou, and Shankar Pasupathy. 2010. SherLog: Error diagnosis by connecting clues from run-time logs. International Conference on Architectural Support for Programming Languages and Operating Systems - ASPLOS(2010), 143–154. https://doi.org/10.1145/1736020.1736038Google ScholarDigital Library
- Jieming Zhu, Shilin He, Jinyang Liu, Pinjia He, Qi Xie, Zibin Zheng, and Michael R. Lyu. 2019. Tools and Benchmarks for Automated Log Parsing. Proceedings - 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice, ICSE-SEIP 2019 (2019), 121–130. https://doi.org/10.1109/ICSE-SEIP.2019.00021 arxiv:1811.03509Google ScholarDigital Library
- De Qing Zou, Hao Qin, and Hai Jin. 2016. UiLog: Improving Log-Based Fault Diagnosis by Log Analysis. Journal of Computer Science and Technology 31, 5 (2016), 1038–1052. https://doi.org/10.1007/s11390-016-1678-7Google ScholarCross Ref
Index Terms
- LEKG: A System for Constructing Knowledge Graphs from Log Extraction
Index terms have been assigned to the content through auto-classification.
Recommendations
Robust log-based anomaly detection on unstable log data
ESEC/FSE 2019: Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software EngineeringLogs are widely used by large and complex software-intensive systems for troubleshooting. There have been a lot of studies on log-based anomaly detection. To detect the anomalies, the existing methods mainly construct a detection model using log event ...
A Survey on Automated Log Analysis for Reliability Engineering
Invited TutorialLogs are semi-structured text generated by logging statements in software source code. In recent decades, software logs have become imperative in the reliability assurance mechanism of many software systems, because they are often the only data ...
Constructing Graphs with No Immersion of Large Complete Graphs
In 1989, Lescure and Meyniel proved, for d=5,6, that every d-chromatic graph contains an immersion of Kd, and in 2003 Abu-Khzam and Langston conjectured that this holds for all d. In 2010, DeVos, Kawarabayashi, Mohar, and Okamura proved this conjecture ...
Comments