Skip to main content

Advertisement

Efficient resource utilization using multi-step-ahead workload prediction technique in cloud

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

The demand of cloud-based services is growing rapidly due to the high scalability and cost-effective nature of cloud infrastructure. As a result, the size of the data center is increasing drastically, so is the cost of maintenance in terms of resource management and energy consumption. Hence, it is important to develop a proper resource management plan to maximize the profit by reducing the overhead of operational cost. In this paper, we propose a multi-step-ahead workload prediction approach using Machine learning techniques and allocate the resources based on this prediction in a way that allows the resources to be utilized more efficiently and thereby, reducing the data center’s overall energy consumption. We evaluate the effectiveness of our framework based on real workload trace of Bitbrains. Experimental results show that our framework outperforms other state-of-the-art approaches for predicting workload over a long-run and significantly improves resource utilization while enabling substantial energy savings.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Ahmed NK, Atiya AF, Gayar NE, El-Shishiny H (2010) An empirical comparison of machine learning models for time series forecasting. Econom Rev 29(5–6):594–621

    Article  MathSciNet  Google Scholar 

  2. Ajiro Y, Tanaka A (2007) Improving packing algorithms for server consolidation. In: International CMG Conference, vol 253, pp 399–406

  3. Barford P, Crovella M (1998) Generating representative web workloads for network and server performance evaluation. In: Proceedings of the 1998 ACM SIGMETRICS Joint International Conference on Measurement and Modeling of Computer Systems, pp 151–160

  4. Beloglazov A, Abawajy J, Buyya R (2012) Energy-aware resource allocation heuristics for efficient management of data centers for cloud computing. Future Gener Comput Syst 28(5):755–768

    Article  Google Scholar 

  5. Beloglazov A, Buyya R (2010) Energy efficient resource management in virtualized cloud data centers. In: 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing. IEEE, pp 826–831

  6. Beloglazov A, Buyya R (2012) Managing overloaded hosts for dynamic consolidation of virtual machines in cloud data centers under quality of service constraints. IEEE Trans Parallel Distrib Syst 24(7):1366–1379

    Article  Google Scholar 

  7. Benson T, Anand A, Akella A, Zhang M (2011) Microte: fine grained traffic engineering for data centers. In: Proceedings of the Seventh Conference on Emerging Networking Experiments and Technologies, pp 1–12

  8. Bey KB, Benhammadi F, Mokhtari A, Guessoum Z (2009) CPU load prediction model for distributed computing. In: 2009 Eighth International Symposium on Parallel and Distributed Computing. IEEE, pp 39–45

  9. Borodin A, Karp R, Tardos G (1990) On the power of randomization in online algorithms. In: Proceeding of the Twenty-Second Annual ACM Symposium on Theory of Computing

  10. Buyya R, Yeo CS, Venugopal S, Broberg J, Brandic I (2009) Cloud computing and emerging it platforms: vision, hype, and reality for delivering computing as the 5th utility. Future Gener Comput Syst 25(6):599–616

    Article  Google Scholar 

  11. Chase JS, Anderson DC, Thakar PN, Vahdat AM, Doyle RP (2001) Managing energy and server resources in hosting centers. ACM SIGOPS Oper Syst Rev 35(5):103–116

    Article  Google Scholar 

  12. Chen Y-L, Chang M-F, Chao-Wei Yu, Chen X-Z, Liang W-Y (2018) Learning-directed dynamic voltage and frequency scaling scheme with adjustable performance for single-core and multi-core embedded and mobile systems. Sensors 18(9):3068

    Article  Google Scholar 

  13. Chen Z, Zhu Y, Di Y, Feng S (2015) Self-adaptive prediction of cloud resource demands using ensemble model and subtractive-fuzzy clustering based fuzzy neural network. Comput Intell Neurosci 2015:919805

    Google Scholar 

  14. Chou J-S, Nguyen T-K (2018) Forward forecast of stock price using sliding-window metaheuristic-optimized machine-learning regression. IEEE Trans Ind Inf 14(7):3132–3142

    Article  Google Scholar 

  15. Cook G, Lee J, Tsai T, Kong A, Deans J, Johnson B, Jardim E (2017) Clicking clean: who is winning the race to build a green internet? Greenpeace Inc., Washington, DC

    Google Scholar 

  16. Dabbagh M, Hamdaoui B, Guizani M, Rayes A (2015) Energy-efficient resource allocation and provisioning framework for cloud data centers. IEEE Trans Netw Serv Manag 12(3):377–391

    Article  Google Scholar 

  17. Dinda PA (2006) Design, implementation, and performance of an extensible toolkit for resource prediction in distributed systems. IEEE Trans Parallel Distrib Syst 17(2):160–173

    Article  Google Scholar 

  18. Duan H, Chen C, Min G, Yu W (2017) Energy-aware scheduling of virtual machines in heterogeneous cloud computing systems. Future Gener Comput Syst 74:142–150

    Article  Google Scholar 

  19. Feitelson DG (2002) Workload modeling for performance evaluation. In: IFIP International Symposium on Computer Performance Modeling, Measurement and Evaluation. Springer, pp 114–141

  20. Garg SK, Yeo CS, Anandasivam A, Buyya R (2011) Environment-conscious scheduling of HPC applications on distributed cloud-oriented data centers. J Parallel Distrib Comput 71(6):732–749

    Article  Google Scholar 

  21. Hota HS, Handa R, Shrivas AK (2017) Time series data prediction using sliding window based RBF neural network. Int J Comput Intell Res 13(5):1145–1156

    Google Scholar 

  22. Iranfar A, Zapater M, Atienza D (2018) Machine learning-based quality-aware power and thermal management of multistream HEVC encoding on multicore servers. IEEE Trans Parallel Distrib Syst 29(10):2268–2281

    Article  Google Scholar 

  23. Islam S, Keung J, Lee K, Liu A (2012) Empirical prediction models for adaptive resource provisioning in the cloud. Future Gener Comput Syst 28(1):155–162

    Article  Google Scholar 

  24. Ismaeel S, Miri A (2015) Using ELM techniques to predict data centre VM requests. In: 2015 IEEE 2nd International Conference on Cyber Security and Cloud Computing. IEEE, pp 80–86

  25. Li H (2009) Workload dynamics on clusters and grids. J Supercomput 47(1):1–20

    Article  Google Scholar 

  26. Li M, Ganesan D, Shenoy P (2009) Presto: feedback-driven data management in sensor networks. IEEE/ACM Trans Netw 17(4):1256–1269

    Article  Google Scholar 

  27. Łuczak M (2016) Hierarchical clustering of time series data with parametric derivative dynamic time warping. Expert Syst Appl 62:116–130

    Article  Google Scholar 

  28. man Jr EGC, Garey MR, Johnson DS (1996) Approximation algorithms for bin packing: a survey. In: Approximation algorithms for NP-hard problems, pp 46–93

  29. Mastroianni C, Meo M, Papuzzo G (2013) Probabilistic consolidation of virtual machines in self-organizing cloud data centers. IEEE Trans Cloud Comput 1(2):215–228

    Article  Google Scholar 

  30. Nathuji R, Schwan K (2007) Virtualpower: coordinated power management in virtualized enterprise systems. ACM SIGOPS Oper Syst Rev 41(6):265–278

    Article  Google Scholar 

  31. Peng C, Li Y, Yu Y, Zhou Y, Du S (2018) Multi-step-ahead host load prediction with gru based encoder-decoder in cloud computing. In: 2018 10th International Conference on Knowledge and Smart Technology (KST). IEEE, pp 186–191

  32. Rodrigues PP, Gama J, Pedroso J (2008) Hierarchical clustering of time-series data streams. IEEE Trans Knowl Data Eng 20(5):615–627

    Article  Google Scholar 

  33. Rong H, Zhang H, Xiao S, Li C, Chunhua H (2016) Optimizing energy consumption for data centers. Renew Sustain Energy Rev 58:674–691

    Article  Google Scholar 

  34. Saini LM, Soni MK (2002) Artificial neural network-based peak load forecasting using conjugate gradient methods. IEEE Trans Power Syst 17(3):907–912

    Article  Google Scholar 

  35. Shen S, van Beek V, Iosup A (2015) Statistical characterization of business-critical workloads hosted in cloud datacenters. In 2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing. IEEE, pp 465–474

  36. Shojafar M, Cordeschi N, Amendola D, Baccarelli E (2015) Energy-saving adaptive computing and traffic engineering for real-time-service data centers. In: 2015 IEEE International Conference on Communication Workshop (ICCW). IEEE, pp 1800–1806

  37. Son J, Dastjerdi AV, Calheiros RN, Buyya R (2017) SLA-aware and energy-efficient dynamic overbooking in SDN-based cloud data centers. IEEE Trans Sustain Comput 2(2):76–89

    Article  Google Scholar 

  38. Song B, Yao Yu, Zhou Yu, Wang Z, Sidan D (2018) Host load prediction with long short-term memory in cloud computing. J Supercomput 74(12):6554–6568

    Article  Google Scholar 

  39. Subirats J, Guitart J (2015) Assessing and forecasting energy efficiency on cloud computing platforms. Future Gener Comput Syst 45:70–94

    Article  Google Scholar 

  40. The SPECpower Benchmark. https://www.spec.org/power_ssj2008/results/res2020q1/power_ssj2008-20200310-01018.html/

  41. Trace description. http://gwa.ewi.tudelft.nl/datasets/gwa-t-12-bitbrains/

  42. Tran N, Reed DA (2004) Automatic ARIMA time series modeling for adaptive I/O prefetching. IEEE Trans Parallel Distrib Syst 15(4):362–377

    Article  Google Scholar 

  43. Voorsluys W, Broberg J, Buyya R et al (2011) Introduction to cloud computing. In: Cloud computing: principles and paradigms, pp 1–44

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Sarbani Roy.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Banerjee, S., Roy, S. & Khatua, S. Efficient resource utilization using multi-step-ahead workload prediction technique in cloud. J Supercomput 77, 10636–10663 (2021). https://doi.org/10.1007/s11227-021-03701-y

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-021-03701-y

Keywords