Abstract
Enterprise systems are inherently complex and maintaining their full, up-to-date overview poses a serious challenge to the enterprise architects’ teams. This problem encourages the search for automated means of discovering knowledge about such systems. An important aspect of this knowledge is understanding the data that are processed by applications and their relationships. In our previous work, we used application logs of an enterprise system to derive knowledge about the interactions taking place between applications. In this paper, we further explore logs to discover correspondence between data processed by different applications. Our contribution is the following: we propose a method for discovering relationships between data using log analysis, we validate our method against a real-life system running at Nordea Bank, we provide detailed insights into a real-life dataset, we analyze the influence of log quality on the results provided by our method, and we provide recommendations for developers on logging practices that can support the log analysis.
Supported by the Gdańsk University of Technology and Nordea Bank.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Gephi, the open graph viz platform. https://gephi.org. Accessed 30 Oct 2023
Acmeair: A nodejs implementation of the acme air sample application with extended logging., https://github.com/lkorzeni11/acmeair-nodejs. Accessed 24 Jul 2023. commitId: 59e8545c1e5264107e60706a360e0c8133aa8f9e
Barua, D., Rumpa, N.T., Hossen, S., Ali, M.M.: Ontology based log analysis of web servers using process mining techniques, pp. 341 – 344 (2019). https://doi.org/10.1109/ICECE.2018.8636791
Bilke, A., Naumann, F.: Schema matching using duplicates, pp. 69 – 80 (2005). https://doi.org/10.1109/ICDE.2005.126
Fu, Q., et al.: Where do developers log? an empirical study on logging practices in industry, pp. 24 – 33 (2014). https://doi.org/10.1145/2591062.2591175
He, P., Zhu, J., Zheng, Z., Lyu, M.R.: Drain: an online log parsing approach with fixed depth tree, pp. 33–40 (2017). https://doi.org/10.1109/ICWS.2017.13
Hulsebos, M., et al.: Sherlock: a deep learning approach to semantic data type detection, pp. 1500–1508 (2019). https://doi.org/10.1145/3292500.3330993
Korzeniowski, L., Goczyła, K.: Discovering interactions between applications with log analysis. In: Maria Ganzha, Leszek Maciaszek, M.P.D.S. (ed.) Proceedings of the 17th Conference on Computer Science and Intelligence Systems. ACSIS, vol. 30, p. 861 – 869 (2022). https://doi.org/10.15439/2022F172
Korzeniowski, L., Goczyła, K.: Discovering relationships between data in enterprise system using log analysis. In: Maria Ganzha, Leszek Maciaszek, M.P.D.S. (ed.) Proceedings of the 18th Conference on Computer Science and Intelligence Systems. ACSIS, vol. 35, pp. 141–150 (2023). https://doi.org/10.15439/2023F4617
Korzeniowski, L., Goczyla, K.: Landscape of automated log analysis: a systematic literature review and mapping study. IEEE Access 10, 21892–21913 (2022). https://doi.org/10.1109/ACCESS.2022.3152549
Manning, C.D., Schütze, H., Weikurn, G.: Foundations of statistical natural language processing. SIGMOD Record 31(3), 37–38 (2002). https://doi.org/10.1145/601858.601867
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space (2013)
Pennington, J., Socher, R., Manning, C.D.: Glove: global vectors for word representation, pp. 1532 – 1543 (2014). https://doi.org/10.3115/v1/d14-1162
Piai, F., Atzeni, P., Merialdo, P., Srivastava, D.: Fine-grained semantic type discovery for heterogeneous sources using clustering. VLDB Journal 32(2), 305–324 (2023). https://doi.org/10.1007/s00778-022-00743-3
Rahm, E., Peukert, E.: Holistic schema matching. In: Sakr, S., Zomaya, A. (eds.) Encyclopedia of Big Data Technologies, 1st edn. Springer, Cham (2019). https://doi.org/10.1007/978-3-319-77525-8_12
Rahm, E., Peukert, E.: Large-scale schema matching. In: Sakr, S., Zomaya, A. (eds.) Encyclopedia of Big Data Technologies, 1st edn. Springer, Cham (2019). https://doi.org/10.1007/978-3-319-77525-8_100191
Shvaiko, P., Euzenat, J.: A survey of schema-based matching approaches. In: Spaccapietra, S. (ed.) Journal on Data Semantics IV. LNCS, vol. 3730, pp. 146–171. Springer, Heidelberg (2005). https://doi.org/10.1007/11603412_5
Vaarandi, R., Pihelgas, M.: Logcluster - a data clustering and pattern mining algorithm for event logs, pp. 1–7 (2015). https://doi.org/10.1109/CNSM.2015.7367331
Xue, X., Zhu, H.: Matching knowledge graphs with compact niching evolutionary algorithm. Expert Syst. Appl. 203 (2022). https://doi.org/10.1016/j.eswa.2022.117371
Zhang, D., Suhara, Y., Li, J., Hulsebos, M., Demiralp, a., Tan, W.C.: Sato: Contextual semantic type detection in tables. Proc. VLDB Endowment 13(11), 1835 – 1848 (2020). https://doi.org/10.14778/3407790.3407793
Acknowledgment
This paper was written in cooperation with the Nordea Bank which provided the log dataset and an overview of the systems that were subject to this study.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Korzeniowski, Ł., Goczyła, K. (2024). Exploring Relationships Between Data in Enterprise Information Systems by Analysis of Log Contents. In: Jarzębowicz, A., Luković, I., Przybyłek, A., Staroń, M., Ahmad, M.O., Ochodek, M. (eds) Software, System, and Service Engineering. KKIO 2023. Lecture Notes in Business Information Processing, vol 499. Springer, Cham. https://doi.org/10.1007/978-3-031-51075-5_5
Download citation
DOI: https://doi.org/10.1007/978-3-031-51075-5_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-51074-8
Online ISBN: 978-3-031-51075-5
eBook Packages: Computer ScienceComputer Science (R0)