Skip to main content
SpringerLink
Log in

Detecting Anomaly in Cloud Platforms Using a Wavelet-Based Framework

  • Conference paper
  • First Online:
Cloud Computing and Services Science (CLOSER 2016)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 740))

Included in the following conference series:

  • 746 Accesses

Abstract

Cloud computing enables the delivery of compute resources as services in an on-demand fashion. The reliability of these services is of significant importance to their consumers. The presence of anomaly in Cloud platforms can put their reliability into question, since an anomaly indicates deviation from normal behaviour. Monitoring enables efficient Cloud service provisioning management; however, most of the management efforts are focused on the performance of the services and little attention is paid to detecting anomalous behaviour from the gathered monitoring data. In addition, the existing solutions for detecting anomaly in Clouds lacks a multi-dimensional approach. In this chapter, we present a wavelet-based anomaly detection framework that is capable of analysing multiple monitored metrics simultaneously to detect anomalous behaviour. It operates in both frequency and time domains in analysing monitoring data that represents system behaviour. The framework is first trained using over seven days worth of historical monitoring data to identify healthy behaviour. Based on this training, anomalous behaviour can be detected as deviations from the healthy system. The effectiveness of the proposed framework was evaluated based on a Cloud service deployment use-case scenario that produced both healthy and anomalous behaviour.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Agarwal, S., Mozafari, B., Panda, A., Milner, H., Madden, S., Stoica, I.: BlinkDB: queries with bounded errors and bounded response times on very large data. In: Proceedings of the 8th ACM European Conference on Computer Systems, pp. 29–42. ACM (2013)

    Google Scholar 

  2. Agarwala, S., Alegre, F., Schwan, K., Mehalingham, J.: E2EProf: automated end-to-end performance management for enterprise systems. In: 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2007, pp. 749–758, June 2007

    Google Scholar 

  3. Albanese, D., Visintainer, R., Merler, S., Riccadonna, S., Jurman, G., Furlanello, C.: mlpy: machine learning Python (2012). http://mlpy.sourceforge.net/. Accessed 22 Feb 2016

  4. Apache Software Foundation. Apache JMeter (2016). http://jmeter.apache.org/. Accessed 06 Jan 2016

  5. Bahl, P., Chandra, R., Greenberg, A., Kandula, S., Maltz, D., Zhang, M.: Towards highly reliable enterprise network services via inference of multi-level dependencies. In: SIGCOMM. Association for Computing Machinery Inc., August 2007

    Google Scholar 

  6. Bakhtazad, A., Palazoglu, A., Romagnoli, J.A.: Detection and classification of abnormal process situations using multidimensional wavelet domain hidden Markov trees. Comput. Chem. Eng. 24(2), 769–775 (2000)

    Article  Google Scholar 

  7. Buzen, J.P., Shum, A.W.: MASF - multivariate adaptive statistical filtering. In: International CMG Conference, pp. 1–10 (1995)

    Google Scholar 

  8. Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection: a survey. ACM Comput. Surv. 41(3), 15:1–15:58 (2009)

    Article  Google Scholar 

  9. Doelitzscher, F., Knahl, M., Reich, C., Clarke, N.: Anomaly detection in IaaS clouds. In: 2013 IEEE 5th International Conference on Cloud Computing Technology and Science (CloudCom), pp. 387–394, December 2013

    Google Scholar 

  10. Emeakaroha, V.C., Brandic, I., Maurer, M., Dustdar, S.: Low level metrics to high level SLAs - LoM2HiS framework: bridging the gap between monitored metrics and SLA parameters in cloud environments. In: 2010 International Conference on High Performance Computing and Simulation (HPCS), pp. 48–54, July 2010

    Google Scholar 

  11. Emeakaroha, V.C., Netto, M.A.S., Calheiros, R.N., Brandic, I., Buyya, R., De Rose, C.A.F.: Towards autonomic detection of SLA violations in cloud infrastructures. Future Gener. Comput. Syst. 28(7), 1017–1029 (2012)

    Article  Google Scholar 

  12. Fatema, K., Emeakaroha, V.C., Healy, P.D., Morrison, J.P., Lynn, T.: A survey of cloud monitoring tools: taxanomy, capabilities and objectives. J. Parallel Distrib. Comput. 74, 2918–2933 (2014)

    Article  Google Scholar 

  13. Frigo, M.: A fast Fourier transform compiler. ACM Sigplan Not. 34, 169–180 (1999). ACM

    Article  Google Scholar 

  14. Gander, M., Felderer, M., Katt, B., Tolbaru, A., Breu, R., Moschitti, A.: Anomaly detection in the cloud: detecting security incidents via machine learning. In: Moschitti, A., Plank, B. (eds.) Trustworthy Eternal Systems via Evolving Software, Data and Knowledge, pp. 103–116. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  15. Guan, Q., Fu, S.: Adaptive anomaly identification by exploring metric subspace in cloud computing infrastructures. In: 2013 IEEE 32nd International Symposium on Reliable Distributed Systems (SRDS), pp. 205–214, September 2013

    Google Scholar 

  16. Guan, Q., Fu, S.: Wavelet-based multi-scale anomaly identification in cloud computing systems. In: 2013 IEEE Global Communications Conference (GLOBECOM), pp. 1379–1384, December 2013

    Google Scholar 

  17. Guan, Q., Fu, S., DeBardeleben, N., Blanchard, S.: Exploring time and frequency domains for accurate and automated anomaly detection in cloud computing systems. In: 2013 IEEE 19th Pacific Rim International Symposium on Dependable Computing (PRDC), pp. 196–205. IEEE (2013)

    Google Scholar 

  18. Gul, I., Hussain, M.: Distributed cloud intrusion detection model. Int. J. Adv. Sci. Technol. 34, 71–82 (2011)

    Google Scholar 

  19. Hodge, V.J., Austin, J.: A survey of outlier detection methodologies. Artif. Intell. Rev. 22(2), 85–126 (2004)

    Article  MATH  Google Scholar 

  20. Ibidunmoye, O., Hernández-Rodriguez, F., Elmroth, E.: Performance anomaly detection and bottleneck identification. ACM Comput. Surv. 48(1), 1–35 (2015)

    Article  Google Scholar 

  21. Lin, M., Yao, Z., Gao, F., Li, Y.: Toward anomaly detection in IaaS cloud computing platforms. Int. J. Secur. Appl. 9(12), 175–188 (2015)

    Google Scholar 

  22. Liu, A., Chen, J.X., Wechsler, H.: Real-time timing channel detection in an software-defined networking virtual environment. Intell. Inf. Manag. 7(06), 283 (2015)

    Google Scholar 

  23. Mi, H., Wang, H., Yin, G., Cai, H., Zhou, Q., Sun, T., Zhou, Y.: Magnifier: online detection of performance problems in large-scale cloud computing systems. In: 2011 IEEE International Conference on Services Computing (SCC), pp. 418–425, July 2011

    Google Scholar 

  24. Penn, B.S.: Using self-organizing maps to visualize high-dimensional data. Comput. Geosci. 31(5), 531–544 (2005)

    Article  MathSciNet  Google Scholar 

  25. Reynolds, P., Killian, C., Wiener, J.L., Mogul, J.C., Shah, M.A., Vahdat, A.: PIP: detecting the unexpected in distributed systems. In: Proceedings of the 3rd Conference on Networked Systems Design and Implementation, NSDI 2006, Berkeley, CA, USA, vol. 3. USENIX Association (2006)

    Google Scholar 

  26. Sha, W., Zhu, Y., Chen, M., Huang, T.: Statistical learning for anomaly detection in cloud server systems: a multi-order Markov chain framework. IEEE Trans. Cloud Comput. (2015). https://doi.org/10.1109/TCC.2015.2415813

  27. Song, X., Wu, M., Jermaine, C., Ranka, S.: Conditional anomaly detection. IEEE Trans. Knowl. Data Eng. 19(5), 631–645 (2007)

    Article  Google Scholar 

  28. Videla, A., Williams, J.J.W.: RabbitMQ in Action: Distributed Messaging for Everyone. Manning Publications Company, Grand Forks (2012)

    Google Scholar 

  29. Wang, C., Talwar, V., Schwan, K., Ranganathan, P.: Online detection of utility cloud anomalies using metric distributions. In: 2010 IEEE Network Operations and Management Symposium (NOMS), pp. 96–103, April 2010

    Google Scholar 

  30. Wang, C., Viswanathan, K., Choudur, L., Talwar, V., Satterfield, W., Schwan, K.: Statistical techniques for online anomaly detection in data centers. In: 2011 IFIP/IEEE International Symposium on Integrated Network Management (IM), pp. 385–392, May 2011

    Google Scholar 

  31. Zhang, Z., Wang, Y., Wang, K.: Fault diagnosis and prognosis using wavelet packet decomposition, Fourier transform and artificial neural network. J. Intell. Manuf. 24(6), 1213–1227 (2013)

    Article  Google Scholar 

Download references

Acknowledgements

The research work described in this paper was supported by the Irish Centre for Cloud Computing and Commerce, an Irish national Technology Centre funded by Enterprise Ireland and the Irish Industrial Development Authority.

Author information

Authors and Affiliations

  1. Irish Centre for Cloud Computing and Commerce, University College Cork, Cork, Ireland

    David O’Shea, Vincent C. Emeakaroha, Neil Cafferkey & John P. Morrison

  2. Irish Centre for Cloud Computing and Commerce, Dublin City University, Dublin 9, Ireland

    Theo Lynn

Authors
  1. David O’Shea
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vincent C. Emeakaroha
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Neil Cafferkey
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. John P. Morrison
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Theo Lynn
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vincent C. Emeakaroha .

Editor information

Editors and Affiliations

  1. Dublin City University, Dublin, Ireland

    Markus Helfert

  2. Columbia University, New York, New York, USA

    Donald Ferguson

  3. Escola d’Enginyeria, Bellaterra, Barcelona, Spain

    Victor Méndez Muñoz

  4. Departamento de Engenharia Informática, Universidade de Coimbra, Coimbra, Portugal

    Jorge Cardoso

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

O’Shea, D., Emeakaroha, V.C., Cafferkey, N., Morrison, J.P., Lynn, T. (2017). Detecting Anomaly in Cloud Platforms Using a Wavelet-Based Framework. In: Helfert, M., Ferguson, D., Méndez Muñoz, V., Cardoso, J. (eds) Cloud Computing and Services Science. CLOSER 2016. Communications in Computer and Information Science, vol 740. Springer, Cham. https://doi.org/10.1007/978-3-319-62594-2_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-62594-2_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-62593-5

  • Online ISBN: 978-3-319-62594-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation