Abstract
Identification of execution anomalies is very important for the maintenance and performance refinement of computer systems. For this purpose we can use system logs. These logs contain vast amounts of data, hence there is a great demand for techniques targeted at log analysis. The paper presents our experience with monitoring event and performance logs related to data repository operation. Having collected representative data from the monitored systems we have developed original algorithms of log analysis and problem predictions, they are based on various data mining approaches. These algorithms have been included in the implemented tools: LogMiner, FEETS, ODM. Practical significance of the developed approaches has been illustrated with some examples of exploring data repository logs. To improve the accuracy of problem diagnostics we have developed supplementary log database which can be filled in by system administrators and users.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chandola, V., Baerjee, A., Kumar, V.: Anomaly detection, a survey. ACM Comput. Surv. 41(3), 1–58 (2009)
Chen, C., Singh, N., Yajnik, M.: Log analytics for dependable enterprise telephony. In: Proceedings of 9th European Dependable Computing Conference, IEEE Computer Society, pp. 94–101 (2012)
Cinque, M., et al.: A logging approach for effective dependability evaluation of computer systems. In: Proceedings of 2nd IEEE International Conference on Dependability, pp. 105–110 (2009)
Naggapan, M., Vouk, M.A.: Abstracting log lines to log event types for mining software system logs. In: Proceedings of Mining Software Repositories, pp. 114–117 (2010)
Oliner, A., Stearley, J.: What supercomputers say: A study of five system logs. In: Proceedings of the IEEE/IFIP International Conference on Dependable Systems and Networks (2007)
Salfiner, F., Lenk, M., Malek, M.: A survey of failure prediction methods. ACM Comput. Surv. 42(3), 10.1–10.42 (2010)
Yu, L., Zheng, Z., Lan, Z.: Practical online failure prediction for blue gene/p: period-based vs. event-driven. In: Proceedings of the IEEE/IFIP International Conference on Dependable Systems and Networks Workshops, pp. 259–264 (2011)
Sosnowski, J., Gawkowski, P., Cabaj, K.: Exploring the space of system monitoring. In: Bembenik, R., et al. (eds.) Intelligent Tools for Building a Scientific Information Platform: Advanced Architectures and Solutions. Studies in Computational Intelligence, vol. 467, pp. 501–517 (2013). ISBN 978-3-642-35646-9
Sosnowski, J., Gawkowski, P., Cabaj, K.: Event and performance logs in system management and evaluation. In: Jałowiecki, P., Orłowski, A. (eds.) Information Systems in Management XIV, Security and Effectiveness of ICT Systems, pp. 83–93. WULS Press, Warsaw (2011). ISBN 978-83-7583-371-3
Sosnowski, J., Kubacki, M., Krawczyk, H.: Monitoring event logs within a cluster system. In: Zamojski, W. et al. (eds.) Complex Systems and Dependability. Advances in Intelligent and Soft Computing, vol. 170, pp. 259–271. Springer, Berlin (2012)
Kubacki, M., Sosnowski, J.: Creating a knowledge data base on system dependability and resilience. Control Cybern. 42(1), 287–307 (2013)
Berry, M.W., Kogan, J.: Text Mining Applications and Theory. Wiley, Chichester (2010)
Han, J., Kamber, M., Pei, J.: Data Mining Concepts and Techniques. Elsevier, Amsterdam (2012)
Hershey, P., Silio, C.B.: Systems engineering approach for event monitoring and analysis in high speed enterprise communication systems. In: IEEE International Systems Conference SysCon (2009)
Kufel, L.: Security event monitoring in a distributed systems environment. IEEE Secur. Priv. 11(1), 36–42 (2013)
Cinque, M., Cotroneo, D., Pecchia, A.: Event logs for the analysis of software failures, a rule based approach. IEEE Trans. Softw. Eng. 39(8), 806–821 (2013)
Fu, X., Rebn, R., Jianfeng, Z., Wei, Z., Zhen, J., Gang, L.: LogMaster: mining event correlations in logs of large-scale cluster systems. In: Proceedings of IEEE Symposium on Reliable Distributed Systems, pp. 71–80 (2012)
Vaarandi, R.: A data clustering algorithm for mining patterns from event logs. In: Proceedings of 3rd IEEE Workshop on IP operations and Management, pp. 119–126 (2003)
Acknowledgment
This work is supported by the National Centre for Research and Development (NCBiR) under Grant No. SP/I/1/77065/10.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Sosnowski, J., Gawkowski, P., Cabaj, K., Kubacki, M. (2014). Analyzing Logs of the University Data Repository. In: Bembenik, R., Skonieczny, Ł., Rybiński, H., Kryszkiewicz, M., Niezgódka, M. (eds) Intelligent Tools for Building a Scientific Information Platform: From Research to Implementation. Studies in Computational Intelligence, vol 541. Springer, Cham. https://doi.org/10.1007/978-3-319-04714-0_9
Download citation
DOI: https://doi.org/10.1007/978-3-319-04714-0_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-04713-3
Online ISBN: 978-3-319-04714-0
eBook Packages: EngineeringEngineering (R0)