Abstract
Web server in the Internet is vulnerable to be attacked. Analyzing on the web logs is one efficient method to figure out intrusion. Using unsupervised algorithm for anomaly detection is suitable for the big data situation. Therefore, the research designs a framework using unsupervised classifiers for anomaly detection in the web log. In this paper, we concentrate on the statistic features and the character features of the web logs. Using the features, we transform the web logs to vectors. We apply a suitable normalized method for our unsupervised classifiers. The principal component analysis (PCA) and the AutoEncoder (AE) are the theoretical basis for the classifiers. As we know, this paper is the first research applying PCA and AE to the web log anomaly detection combining statistic features and character features. In the simulation, we find the statistic features are efficient for the PCA. When we use the AE, character features are better. Compared with other methods, results show that our model performs better.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Gollmann, D.: Computer Security, 2nd edn. Wiley, Hoboken (2006)
Zolotukhin, M., Hamalainen, T., Kokkonen, T.: Analysis of HTTP requests for anomaly detection of web attacks. In: IEEE, International Conference on Dependable, Autonomic and Secure Computing, pp. 406–411. IEEE (2014)
Garcia, K.A., Monroy, R., Trejo, L.A.: Analyzing log files for postmortem intrusion detection. IEEE Trans. Syst. Man Cybern. Part C 42(6), 1690–1704 (2012)
Dogbe, E., Millham, R., Singh, P.: A combined approach to prevent SQL Injection Attacks. In: Science and Information Conference, pp. 406–410. IEEE (2013)
Joshi, A., Geetha, V.: SQL Injection detection using machine learning. In: International Conference on Control, Instrumentation, Communication and Computational Technologies, pp. 1111–1115. IEEE (2014)
Algiriyage, N., Jayasena, S., Dias, G.: Identification and characterization of crawlers through analysis of web logs. IEEE International Conference on Industrial and Information Systems, pp. 150–155. IEEE (2014)
Liu, F.T., Ting, K.M., Zhou, Z.-H.: On detecting clustered anomalies using SCiForest. In: Balcázar, J.L., Bonchi, F., Gionis, A., Sebag, M. (eds.) ECML PKDD 2010. LNCS, vol. 6322, pp. 274–290. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15883-4_18
Aryal, S., Ting, K.M., Wells, J.R., Washio, T.: Improving iForest with relative mass. In: Tseng, V.S., Ho, T.B., Zhou, Z.-H., Chen, A.L.P., Kao, H.-Y. (eds.) PAKDD 2014. LNCS, vol. 8444, pp. 510–521. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-06605-9_42
Williamson, R., Smola, A., Shawe-Taylor, J.: Support vector method for novelty detection. In: International Conference on Neural Information Processing Systems, pp. 582–588. MIT Press (1999)
Lin, Q., Zhang, H., Lou, J.G., Zhang, Y., Chen, X.: Log clustering based problem identification for online service systems. In: Proceedings of the 38th International Conference on Software Engineering (2016)
Dauxois, J., Pousse, A., Romain, Y.: Asymptotic theory for the principal component analysis of a vector random function: some applications to statistical inference. J. Multivar. Anal. 12(1), 136–154 (1982)
Hirsimaki, T., Pylkkonen, J., Kurimo, M.: Importance of high-order N-gram models in morph-based speech recognition. IEEE Trans. Audio Speech Lang. Process. 17(4), 724–732 (2009)
Acknowledgments
This research is funded by The National Natural Science Fund (61471055).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Jin, L., Wang, X.J., Zhang, Y., Yao, L. (2019). Anomaly Detection in the Web Logs Using Unsupervised Algorithm. In: Tang, Y., Zu, Q., RodrÃguez GarcÃa, J. (eds) Human Centered Computing. HCC 2018. Lecture Notes in Computer Science(), vol 11354. Springer, Cham. https://doi.org/10.1007/978-3-030-15127-0_40
Download citation
DOI: https://doi.org/10.1007/978-3-030-15127-0_40
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-15126-3
Online ISBN: 978-3-030-15127-0
eBook Packages: Computer ScienceComputer Science (R0)