Skip to main content

Advertisement

Springer Nature Link
Log in

An adaptive RL based approach for dynamic resource provisioning in Cloud virtualized data centers

  • Published:
Computing Aims and scope Submit manuscript

Abstract

Because of numerous parameters existing in the Cloud’s environment, it is helpful to introduce a general solution for dynamic resource provisioning in Cloud that is able to handle uncertainty. In this paper, a novel adaptive control approach is proposed which is based on continuous reinforcement learning and provides dynamic resource provisioning while dealing with uncertainty in the Cloud’s environment. The proposed dynamic resource provisioner is a goal directed controller which provides ability of handling uncertainty specifically in Cloud’s spot markets where competition between Cloud providers requires optimal policies for attracting and maintaining clients. This controller is aimed at hardly preventing from job rejection (as the primary goal) and minimizing the energy consumption (as the secondary goal). Although these two goals almost conflict (because job rejection is a common event in the process of energy consumption optimization), the results demonstrate the perfect ability of the proposed method with reducing job rejection down to near 0 % and minimizing energy consumption down to 9.55 %.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. McKnight W (2014) Cloud computing: on-demand elasticity. Morgan Kaufmann, Burlington

  2. Gupta P, Seetharaman A, Raj JR (2013) The usage and adoption of cloud computing by small and medium businesses. Int J Inf Manag 33:861–874

    Article  Google Scholar 

  3. Maurer M, Brandic I, Sakellariou R (2013) Adaptive resource configuration for Cloud infrastructure management. Future Gener Comput Syst 29:472–487

    Article  Google Scholar 

  4. Wang X, Dub Z, Chen Y (2012) An adaptive model-free resource and power management approach for multi-tier cloud environments. J Syst Softw 85:1135–1146

    Article  Google Scholar 

  5. Zhang W, Liu J, Song Y, Zhu M, Xiao L, Sun Y et al (2011) Dynamic resource allocation based on user experience in virtualized servers. Adv Control Eng Inf Sci 15:3780–3784

    Google Scholar 

  6. Buyya R, Yeo CS, Venugopal S, Broberg J, Brandic I (2009) Cloud computing and emerging IT platforms: vision, hype, and reality for delivering computing as the 5th utility. Future Gener Comput Syst 25:599–616

    Article  Google Scholar 

  7. Huang Y, Lan Y, Thomson SJ, Fang A, Hoffmann WC, Lacey RE (2010) Development of soft computing and applications in agricultural and biological engineering. Comput Electron Agric 71:107–127

    Article  Google Scholar 

  8. Bagherjeiran A, Eick CF, Chen CS, Vilalta R (2005) Adaptive clustering: obtaining better clusters using feedback and past experience. In Proceedings of the fifth IEEE international conference on data mining (ICDM-05)

  9. Salehie M, Tahvildari L (2009) Self-adaptive software: Landscape and research challenges. In: ACM transactions on autonomous and adaptive systems (TAAS), vol 4

  10. Taylor ME, Stone P (2009) Transfer learning for reinforcement learning domains: a survey. J Mach Learn Res 10:1633–1685

    MathSciNet  MATH  Google Scholar 

  11. Zhang Q, Zhu Q, Boutaba R (2011) Dynamic resource allocation for spot markets in cloud computing environments. In: Fourth IEEE international conference on utility and cloud computing (UCC), pp 178–185

  12. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, Cambridge

    Google Scholar 

  13. Wang X-S, Cheng Y-H, Yi J-Q (2007) A fuzzy actor-critic reinforcement learning network. Inf Sci 177:3764–3781

    Article  Google Scholar 

  14. Bahrpeyma F, Zakerolhosseini A, Haghighi H (2015) Using IDS fitted Q to develop a real-time adaptive controller for dynamic resource provisioning in Cloud’s virtualized environment. Appl Soft Comput 26:285–298

    Article  Google Scholar 

  15. Islam S, Keung J, Lee K, Liu A (2012) Empirical prediction models for adaptive resource provisioning in the cloud. Future Gener Comput Syst 28:155–162

    Article  Google Scholar 

  16. Bahrpeyma F, Golchin B, Cranganu C (2013) Fast fuzzy modeling method to estimate missing logsin hydrocarbon reservoirs. J Petrol Sci Eng 112:310–321

    Article  Google Scholar 

  17. Farahnakian F, Liljeberg P, Plosila J (2014) Energy-efficient virtual machines consolidation in cloud data centers using reinforcement learning. In: 22nd euromicro international conference on parallel, distributed and network-based processing (PDP), Torino

  18. Yang B, Xu X, Tan F, Park DH (2011) An utility-based job scheduling algorithm for Cloud computing considering reliability factor. In: 2011 international conference on cloud and service computing (CSC), Hong Kong

  19. Guo Y, Lama P, Jiang C, Zhou X (2014) Automated and agile server parametertuning by coordinated learning and control. IEEE Trans Parallel Distrib Syst 25:876–886

    Article  Google Scholar 

  20. Bu X, Rao J, Xu C-Z (2013) Coordinated self-configuration of virtual machines and appliances using a model-free learning approach. In: IEEE transactions on parallel and distributed systems, vol 24, pp 681–690

  21. Hussin M, Lee YC, Zomaya AY (2011) Efficient energy management using adaptive reinforcement learning-based scheduling in large-scale distributed systems. In: International conference on parallel processing (ICPP), pp 385–393

  22. Barto AG, Sutton R (1996) Reinforcement learning: an introduction, adaptive computation and machine learning

  23. Khan SG, Herrmann G, Lewis FL, Pipe T, Melhuish C (2012) Reinforcement learning and optimal adaptive control: an overview and implementation examples. Annu Rev Control 36:42–59

    Article  Google Scholar 

  24. Lin YP, Li XY (2003) Reinforcement learning based on local state feature learning and policy adjustment. Inf Sci

  25. Watkins C (1989) Learning from delayed rewards. PhD, University of Cambridge

  26. Lin L, Xie H, Zhang D, Shen L (2010) Supervised neural \({\rm Q}\_\)learning based motion control for bionic underwater robots. J Bionic Eng 7:177–184

  27. Tesauro GJ (1992) Practical issues in temporal-difference learning. Mach Learn 8:257–277

    MATH  Google Scholar 

  28. Ilg W, Berns K, Miihlfriedel T, Dillmann R (1997) Hybrid learning concepts based on self-organizing neural networks for adaptive control of walking machines. Robot Autonom Syst 22:317–327

    Article  Google Scholar 

  29. Gabel T, Lutz C, Riedmiller M (2011) Improved neural fitted Q iteration applied to a novel computer gaming and learning benchmark. In: 2011 IEEE symposium on adaptive dynamic programming and reinforcement learning (ADPRL), Paris

  30. Watkins C, Dayan P (1992) Q-learning. Mach Learn 8:279–292

    MATH  Google Scholar 

  31. Rodrigues J, Nogueira A, Salvador P (2010) Improving the traffic prediction capability of neural networks using sliding window and multi-task learning mechanisms presented at the second international conference on evolving internet (INTERNET)

  32. Levenberg K (1944) A method for the solution of certain non-linear problems in least squares. Q Appl Math 2:164–168

    MathSciNet  MATH  Google Scholar 

  33. Zhang Q, Faten ZM, Zhang S, Zhu Q, Boutaba R, Hellerstein JL (2012) Dynamic energy-aware capacity provisioning for cloud computing environments. In: ICAC’12, California, USA, San Jose

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Shahid Beheshti University, G.C., Tehran, Iran

    Fouad Bahrpeyma, Hassan Haghighi & Ali Zakerolhosseini

Authors
  1. Fouad Bahrpeyma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hassan Haghighi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ali Zakerolhosseini
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fouad Bahrpeyma.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bahrpeyma, F., Haghighi, H. & Zakerolhosseini, A. An adaptive RL based approach for dynamic resource provisioning in Cloud virtualized data centers. Computing 97, 1209–1234 (2015). https://doi.org/10.1007/s00607-015-0455-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00607-015-0455-8

Keywords

Mathematics Subject Classification