Abstract
Cloud computing has emerged as a new paradigm that offers on-demand availability and flexible pricing models. However, cloud applications are being transformed into large scale systems where managing and monitoring cloud resources becomes a challenging task. System administrators are in need of automated tools to effectively detect abnormal system behaviour and ensure the Service Level Agreement (SLA) between the service user and the service provider. In this work, we propose a framework for online anomaly detection based on cloud application metrics. We utilize Recurrent Neural Networks for learning normal sequence representations and predict future events. Then, we use the predicted sequence as the representative sequence of normal events and based on the Dynamic Time Warping algorithm we classify future time series as normal or abnormal. Furthermore, to create a real world scenario and validate the proposed method, we used Yahoo! Cloud Serving Benchmark as a state-of-the-art benchmark tool for cloud data serving systems. Our experimental analysis shows the ability of the proposed approach to detect abnormal behaviours of NoSQL systems on-the-fly with minimum instrumentation.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Gudivada, V.N., Rao, D., Raghavan, V.V.: NoSQL systems for big data management. In: 2014 IEEE World Congress on Services, pp. 190–197. IEEE (2014)
Bhattacharyya, A., Jandaghi, S.A.J., Sotiriadis, S., Amza, C.: Semantic aware online detection of resource anomalies on the cloud. In: 2016 IEEE International Conference on Cloud Computing Technology and Science (CloudCom), pp. 134–143. IEEE (2016)
Gupta, B., Badve, O.P.: Taxonomy of DoS and DDoS attacks and desirable defense mechanism in a cloud computing environment. Neural Comput. Appl. 28(12), 3655–3682 (2017). https://doi.org/10.1007/s00521-016-2317-5
Chouliaras, S., Sotiriadis, S.: Real-time anomaly detection of NoSQL systems based on resource usage monitoring. IEEE Trans. Ind. Inf. 16(9), 6042–6049 (2019)
Cooper, B.F., Silberstein, A., Tam, E., Ramakrishnan, R., Sears, R.: Benchmarking cloud serving systems with YCSB. In: Proceedings of the 1st ACM Symposium on Cloud Computing, pp. 143–154. ACM (2010)
Jain, A.K., Mao, J., Mohiuddin, K.M.: Artificial neural networks: a tutorial. Computer 29(3), 31–44 (1996)
Williams, R.J., Zipser, D.: A learning algorithm for continually running fully recurrent neural networks. Neural Comput. 1(2), 270–280 (1989)
Raschka, S.: Python Machine Learning. Packt Publishing Ltd., Birmingham (2015)
Berndt, D.J., Clifford, J.: Using dynamic time warping to find patterns in time series. In: KDD Workshop, vol. 10, no. 16, pp. 359–370. Seattle, WA (1994)
Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Dhingra, M., Lakshmi, J., Nandy, S.: Resource usage monitoring in clouds. In: 2012 ACM/IEEE 13th International Conference on Grid Computing. IEEE (2012)
Mosberger, D., Jin, D.: httperf–a tool for measuring web server performance. ACM SIGMETRICS Perform. Eval. Rev. 26(3), 31–37 (1998)
Chouliaras, S., Sotiriadis, S.: Detecting performance degradation in cloud systems using LSTM autoencoders. In: Barolli, L., Woungang, I., Enokido, T. (eds.) Advanced Information Networking and Applications. AINA 2021. Lecture Notes in Networks and Systems, vol. 226, pp. 472–481. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-75075-6_38
Zhao, S., Chen, K., Zheng, W.: Defend against denial of service attack with VMM. In: 2009 Eighth International Conference on Grid and Cooperative Computing, pp. 91–96. IEEE (2009)
Bertero, C., Roy, M., Sauvanaud, C., Trédan, G.: Experience report: log mining using natural language processing and application to anomaly detection. In: 2017 IEEE 28th International Symposium on Software Reliability Engineering (ISSRE), pp. 351–360. IEEE (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Chouliaras, S., Sotiriadis, S. (2022). Inferring Anomalies from Cloud Metrics Using Recurrent Neural Networks. In: Barolli, L., Chen, HC., Enokido, T. (eds) Advances in Networked-Based Information Systems. NBiS 2021. Lecture Notes in Networks and Systems, vol 313. Springer, Cham. https://doi.org/10.1007/978-3-030-84913-9_14
Download citation
DOI: https://doi.org/10.1007/978-3-030-84913-9_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-84912-2
Online ISBN: 978-3-030-84913-9
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)