Jamal El abdelkhalki
Mohamed Ben ahmed
ABSTRACT
analyzing the log file for software or device provides a focal point for making incremental improvements; it is the performed step to start the incident analysis. Although, log messages format or contents may not always be fully documented, and described in many different formats. It makes the log analysis task more difficult, affects the correction deadline of incidents and therefore involves a high financial risk. In this paper, we survey the log file analysis and the existing systems elaborated to resolve current issue. Then, we propose a methodology to support the log analysis in the complex environment related to big data issues. Finally, we illustrate our proposal on the file log of the Tivoli Storage Manager (TSM) and provide a discussion of the result clusters.
- J. Valdman, "Log file analysis," Dep. Comput. Sci. Eng. FAV UWB Tech Rep DCSETR-2001-04, 2001.Google Scholar
- "Advances and challenges in log analysis."Google Scholar
- M. Charrad, M. B. Ahmed, and Y. Lechevallier, "Extraction des connaissances à partir des fichiers logs," Atelier Fouille Web EGC2006, vol. 768, 2005.Google Scholar
- J.-S. Vayre, "Les big data et la relation client," in 12ème Journées Normandes de Recherches sur la Consommation: Société et Consommation, 2013, pp. 1--20.Google Scholar
- "What is Log Analysis? - Definition from Techopedia," Techopedia.com. {Online}. Available: https://www.techopedia.com/definition/31756/log-analysis. {Accessed: 20-Aug-2017}.Google Scholar
- K. E. Nawyn, "A security analysis of system event logging with syslog," Inst. No Part Inf. Secur. Read. Room, 2003.Google Scholar
- D. Casey, "Turning log files into a security asset," Netw. Secur., vol. 2008, no. 2, pp. 4--7, 2008. Google ScholarDigital Library
- S. Narkhede and T. Baraskar, "HMR Log Analyzer: Analyze Web Application Logs Over Hadoop MapReduce," Int. J. UbiComp, vol. 4, no. 3, pp. 41--51, Jul. 2013.Google Scholar
- H. Yu and D. Wang, "Mass log data processing and mining based on Hadoop and cloud computing," in 2012 7th International Conference on Computer Science Education (ICCSE), 2012, pp. 197--202.Google Scholar
- "SAFAL: A MapReduce Spatio-temporal Analyzer for UNAVCO FTP Logs." {Online}. Available: https://www.computer.org/csdl/proceedings/cse/2013/5096/00/5096b083-abs.html. {Accessed: 28-Jul-2017}.Google Scholar
- R. S. J. de Baker and P. S. Inventado, "Chapter X: Educational Data Mining and Learning Analytics."Google Scholar
- L. Wang et al., "G-Hadoop: MapReduce across distributed data centers for data-intensive computing," Future Gener. Comput. Syst., vol. 29, no. 3, pp. 739--750, Mar. 2013. Google ScholarDigital Library
- K. Nagaraj, C. Killian, and J. Neville, "Structured comparative analysis of systems logs to diagnose performance problems," in Proceedings of the 9th USENIX conference on Networked Systems Design and Implementation, 2012, pp. 26--26. Google ScholarDigital Library
- P. J. Rousseeuw and L. Kaufman, Finding Groups in Data. Wiley Online Library, 1990.Google Scholar
- K. Alsabti, S. Ranka, and V. Singh, "An efficient k-means clustering algorithm," 1997.Google Scholar
- G. Nathiya, S. C. Punitha, and M. Punithavalli, "An analytical study on behavior of clusters using k means, em and k* means algorithm," ArXiv Prepr. ArXiv10041743, 2010.Google Scholar
- "IBM Knowledgecenter - Tivoli Storage Manager server and client messages format." {Online}. Available: https://www.ibm.com/support/knowledgecenter/en/SSGSG7_7.1.4/msgs.tsm/msgs_msgsformat.html. {Accessed: 28-Jul-2017}.Google Scholar
- "Weka 3 - Data Mining with Open Source Machine Learning Software in Java." {Online}. Available: http://www.cs.waikato.ac.nz/ml/weka/. {Accessed: 20-Aug-2017}.Google Scholar
Recommendations
Incident prediction through logging management and machine learning
SCA '19: Proceedings of the 4th International Conference on Smart City ApplicationsAnalyzing the log file for software or device provides a focal point for making incremental improvements; it is the performed step to start the incident analysis. Although, log messages format or contents may not always be fully documented, and ...
The design and implementation of a log-structured file system
This paper presents a new technique for disk storage management called a log-structured file system. A log-structured file system writes all modifications to disk sequentially in a log-like structure, thereby speeding up both file writing and crash ...
Comments