Skip to main content
SpringerLink
Log in

A two-stage virtual machine abnormal behavior-based anomaly detection mechanism

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Virtual machine abnormal behavior detection is an effective way to help cloud platform administrators monitor the running status of cloud platform to improve the reliability of cloud platform, which has become one of the research hotspots in the field of cloud computing. Aiming at the problems of high computational complexity and high false alarm rate in the existing virtual machine anomaly monitoring mechanism of cloud platform, this paper proposed a two-stage virtual machine abnormal behavior-based detection mechanism. Firstly, a workload-based incremental clustering algorithm is used to monitor and analyze both the virtual machine workload information and performance index information. Then, an online anomaly detection mechanism based on the incremental local outlier factor algorithm is designed to enhance detection efficiency. By applying this two-phase detection mechanism, it can significantly reduce the computational complexity and meet the needs of real-time performance. The experimental results are verified on the mainstream Openstack cloud platform.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Chiba, Z., Abghour, N., Moussaid, K., et al.: A survey of intrusion detection systems for cloud computing environment. In: Presented at the 2016 International Conference on Engineering and MIS (ICEMIS) (2016)

  2. Razaque, A., Amsaad, F., Hariri, S., et al.: Enhanced Grey risk assessment model for support of cloud service provider. IEEE Access 99, 1 (2020)

    Google Scholar 

  3. Jian, Z., Zheng, L., Gong, L., et al.: A survey on security of cloud environment: threats, solutions, and innovation. In: 2018 IEEE Third International Conference on Data Science in Cyberspace (DSC). IEEE Computer Society (2018)

  4. Daniya, T., Suresh Kumar, K., Santhosh Kumar, B., Chandra Sekhar, K.: A survey on anomaly based intrusion detection system. Mater. Today Proc. (2021). https://doi.org/10.1016/j.matpr.2021.03.353

    Article  Google Scholar 

  5. Fargo, F., Franza, O., Tunc, C., et al.: Autonomic resource management for power, performance, and security in cloud environment. In: 2019 IEEE/ACS 16th International Conference on Computer Systems and Applications (AICCSA). IEEE (2019)

  6. Doshi, R., Kute, V.: A review paper on security concerns in cloud computing and proposed security models. In: 2020 International Conference on Emerging Trends in Information Technology and Engineering (IC-ETITE). (2020)

  7. Pan, Z., Hariri, S., Pacheco, J.: Context aware intrusion detection for building automation systems. Comput. Secur. 85(Aug.), 181–201 (2019)

    Article  Google Scholar 

  8. Kumbhare, N., Marathe, A., Akoglu, A., et al.: A value-oriented job scheduling approach for power-constrained and oversubscribed HPC systems. IEEE Trans. Parallel Distrib. Syst. 31(6), 1419–1433 (2020)

    Article  Google Scholar 

  9. Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection: a survey. ACM Comput. Surv. 41(3), Article No. 15 (2009)

  10. Gu, B., Sheng, V.S.: A robust regularization path algorithm for \(\nu \)-support vector classification. IEEE Trans. Neural Netw. Learn. Syst. 28(5), 1241–1248 (2017)

    Article  Google Scholar 

  11. Kumar, M., Mathur, R.: Unsupervised outlier detection technique for intrusion detection in cloud computing. In: IEEE International Conference for Convergence of Technology (I2CT). IEEE (2015)

  12. Song, C., Wang, Y., Zhang, S., et al.: A low cost and easy implement highway accident detection model based on Big Data. In: 2019 IEEE International Conferences on Ubiquitous Computing and Communications (IUCC) and Data Science and Computational Intelligence (DSCI) and Smart Computing, Networking and Services (Smart CNS). IEEE (2020)

  13. Yin, C., Zhang, S.: Parallel implementing improved k-means applied for image retrieval and anomaly detection. Multimed. Tools Appl. 76(16), 16911–16927 (2017)

    Article  Google Scholar 

  14. Adler, N., Yazhemsky, E.: Improving discrimination in data envelopment analysis: PCA-DEA or variable reduction. Eur. J. Oper. Res. 202(1), 273–284 (2010)

    Article  Google Scholar 

  15. Koziel, S., Pietrenkodabrowska, A.: Low-cost data-driven modelling of microwave components using domain confinement and PCA-based dimensionality reduction. IET Microw. Antennas Propag. 14(13), 1643–1650 (2020)

    Article  Google Scholar 

  16. Li, Z., Yan, X.: Fault-relevant optimal ensemble ICA model for non-Gaussian process monitoring. IEEE Trans. Control Syst. Technol. 28(6), 2581–2590 (2020)

    Article  Google Scholar 

  17. Advani, M.S., Saxe, A.M.: High-dimensional dynamics of generalization error in neural networks. Neural Netw. 132, 428–446 (2020)

    Article  Google Scholar 

  18. Stephanakis, I.M., Chochliouros, I.P., Sfakianakis, E., Shirazi, S.N.: Hybrid self-organizing feature map (SOM) for anomaly detection in cloud infrastructures using granular clustering based upon value-difference metrics. Inf. Sci. 494(C), 247–277 (2019)

    Article  Google Scholar 

  19. Gu, B., Sheng, V.S., Tay, K.Y., Romano, W., Li, S.: Incremental support vector learning for ordinal regression. IEEE Trans. Neural Netw. Learn. Syst. 26(7), 1403–1416 (2017)

    Article  MathSciNet  Google Scholar 

  20. Injadat, M., Salo, F., Nassif, A.B., et al.: Bayesian optimization with machine learning algorithms towards anomaly detection. In: IEEE Global Communications Conference. IEEE (2018)

  21. Chen, C.C., Fu, X., Chang, C.Y.: A terms mining and clustering technique for surveying network and content analysis of academic groups exploration. Clust. Comput. 20(1), 43–52 (2017)

    Article  Google Scholar 

  22. Ji, Y., Zhang, H., Zhang, Z., et al.: CNN-based encoder–decoder networks for salient object detection: a comprehensive review and recent advances. Inf. Sci. 546, 835–857 (2021)

    Article  MathSciNet  Google Scholar 

  23. Ullah, W., Ullah, A., Haq, I.U., et al.: CNN features with bi-directional LSTM for real-time anomaly detection in surveillance networks. Multimed. Tools Appl. (2020). https://doi.org/10.1007/s11042-020-09406-3

    Article  Google Scholar 

  24. Kc, K., Gu, X.: ELT: efficient log-based troubleshooting system for cloud computing infrastructures. In: Proceedings of the IEEE Symposium on Reliable Distributed Systems, pp. 11–20 (2011)

  25. Wang, C., Schwan, K., Wolf M.: EbAT: an entropy based online Anomaly Tester for data center management. In: 2009 IFIP/IEEE International Symposium on Integrated Network Management-Workshops. IEEE (2009)

  26. Wang, C., Talwar, V., Schwan, K., et al.: Online detection of utility cloud anomalies using metric distributions. In: IEEE/IFIP Network Operations and Management Symposium. IEEE (2010)

  27. Bhaduri, K., Das, K., Matthews, B.L.: Detecting abnormal machine characteristics in cloud infrastructures. In: 2011 IEEE 11th International Conference on Data Mining Workshops (ICDMW), Vancouver, BC, Canada. IEEE (2012)

  28. Lopez, A.D., et al.: Network traffic behavioral analytics for detection of DDoS attacks. SMU Data Sci. Rev. 2(1), 14 (2019)

    Google Scholar 

  29. Zhang, X., Meng, F., Xu, J.: PerfInsight: a robust clustering-based abnormal behavior detection system for large-scale cloud. In: 2018 IEEE 11th International Conference on Cloud Computing (CLOUD), pp. 896–899 (2018)

  30. Liu, J., Chen, S., Zhou, Z., et al.: An anomaly detection algorithm of cloud platform based on self-organizing maps. Math. Probl. Eng. 4, 1–9 (2016)

    Google Scholar 

  31. Liu, J., Zhang, H., Xu, G.: An anomaly detector deployment awareness detection framework based on multi-dimensional resources balancing in coud platform. IEEE Access 6, 44927–44933 (2018)

    Article  Google Scholar 

  32. Liu, J., Tang, S., Xu, G., et al.: A novel configuration tuning method based on feature selection for Hadoop MapReduce. IEEE Access 99, 1 (2020)

    Google Scholar 

  33. Alnafessa, H.A., Casale, G.: Artificial neural networks based techniques for anomaly detection in Apache Spark. Clust. Comput. 23(4), 1–16 (2020)

    Google Scholar 

  34. Jindal, A., Staab, P., Cardoso, J., et al.: Online memory leak detection in the cloud-based infrastructures. In: International Workshop on Artificial Intelligence for IT Operations (AIOPS) 2020 (2021)

  35. Luo, J., Tang, J., Xiao, X.: Abnormal gait behavior detection for elderly based on enhanced Wigner–Ville analysis and cloud incremental SVM learning. J. Sens. 2016, 1–18 (2016)

    Google Scholar 

  36. Breunig, M.M., Kriegel, H.P., Ng, R.T., Sander, J.: LOF: identifying density-based local outliers. In: Proceedings of the ACM SIGMOD International Conference on Management of Data, vol. 29(2), pp. 93–104 (2000)

  37. Guizani, N., Ghafoor, A.: A network function virtualization system for detecting malware in large IoT based networks. IEEE J. Sel. Areas Commun. 99, 1 (2020)

    Google Scholar 

  38. Alatawi, S., Alhasani, A., Alfaidi, S., et al.: A survey on cloud security issues and solution. In: 2020 International Conference on Computing and Information Technology (ICCIT-1441) (2020)

  39. Hussain, S.A., Fatima, M., Saeed, A., et al.: Multilevel classification of security concerns in cloud computing. Appl. Comput. Inform. 13(1), 57–65 (2017)

    Article  Google Scholar 

  40. Liu, J., Zheng, S., Xu, G., et al.: Cross-domain sentiment aware word embeddings for review sentiment analysis. Int. J. Mach. Learn. Cybern. 12(5), 1–12 (2020)

    Google Scholar 

  41. Coppolino, L., D’Antonio, S., Mazzeo, G., et al.: Cloud security: emerging threats and current solutions. Comput. Electr. Eng. 59, 126–140 (2017)

    Article  Google Scholar 

  42. Yazdanov, L., Fetzer, C.: VScaler: autonomic virtual machine scaling. In: IEEE Sixth International Conference on Cloud Computing. IEEE (2013)

  43. Chen, T., Bahsoon, R.: Self-adaptive and online QoS modeling for cloud-based software services. IEEE Trans. Softw. Eng. (2017). https://doi.org/10.1109/TSE.2016.2608826

    Article  Google Scholar 

  44. Hosseini, F.S., Yang, C.: Comprehensive evaluation of program reliability with ComFIDet: an integrated fault injection and detection framework for embedded systems. In: 2019 IEEE International Conference on Embedded Software and Systems (ICESS). IEEE (2019)

  45. Alexandersson, R., Karlsson, J.: Fault injection-based assessment of aspect-oriented implementation of fault tolerance. Lect. Notes Comput. Sci. 6351(4), 466–479 (2010)

    Article  Google Scholar 

  46. Chen, X., Lin, J., Ma, Y., et al.: Self-adaptive resource allocation for cloud-based software services based on progressive QoS prediction model. Sci. China Inf. Sci. 62(11), 1–3 (2019)

    Google Scholar 

  47. Sethi, K., Kumar, R., Prajapati, N., et al.: Deep reinforcement learning based intrusion detection system for cloud infrastructure. In: 2020 International Conference on COMmunication Systems and NETworkS (COMSNETS) (2020)

  48. Ferrari, P., Rinaldi, S., Sisinni, E., et al.: Performance evaluation of full-cloud and edge-cloud architectures for industrial IoT anomaly detection based on deep learning. In: 2019 II Workshop on Metrology for Industry 4.0 and IoT (MetroInd4.0 & IoT) (2019)

  49. Huang, J., Jiang, Z., Gong, L., et al.: Construction of hidden fault channel cloud test platform based on deep learning. In: 2020 12th International Conference on Measuring Technology and Mechatronics Automation (ICMTMA) (2020)

  50. Kimmel, J.C., Mcdole, A.D., Abdelsalam, M., et al.: Recurrent neural networks based online behavioural malware detection techniques for cloud infrastructure. IEEE Access 9, 68066–68080 (2021)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Information Science and Technology, Zhejiang Sci-Tech University, Hangzhou, 310018, China

    Hancui Zhang & Weida Zhou

Authors
  1. Hancui Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Weida Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hancui Zhang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, H., Zhou, W. A two-stage virtual machine abnormal behavior-based anomaly detection mechanism. Cluster Comput 25, 203–214 (2022). https://doi.org/10.1007/s10586-021-03385-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-021-03385-2

Keywords

Search

Navigation