Abstract
Public cloud system offers Infrastructure-as-a-Service (IaaS) to deliver the computational resources on demand. Resource requirements of a cloud environment are always fluctuating because of the dynamic nature of the arriving workload, and traditional reactive scaling techniques are employed to deal with this problem. Automated resource provisioning is an effective methodology for handling workload fluctuations by provisioning the resources on demand. Simple reactive approaches affect the performance of elastic system by over-provisioning the resources that substantially increase the costs whereas under-provisioning leads to starvation. An intelligent resource provisioning mechanism overcomes the stated issues by allocating necessary resources by learning the environment dynamically. In this article, RLPAS (Reinforcement Learning based Proactive Auto-Scaler) algorithm is proposed, and it is based on the existing Reinforcement Learning (RL)-SARSA algorithm that learns the environment in parallel and allocates the resources. The performance of RLPAS algorithm is validated using real workloads, and it outperforms existing auto-scaling approaches in terms of CPU utilization, response time and throughput. Further, it also converges at an optimal time step and proves to be feasible for the extensive range of real cloud applications.









Similar content being viewed by others
Explore related subjects
Discover the latest articles and news from researchers in related subjects, suggested using machine learning.References
Barham P, Dragovic B, Fraser K, Hand S, Harris T, Ho A, Neugebauer R, Pratt I, Warfield A (2003) Xen and the art of virtualization. In Proceedings of the nineteenth acm symposium on operating systems principles, SOSP ‘03. ACM: New York, pp. 164–177
VMware virtualisation software. (Available from: http://www.vmware.com/in.html [Accessed date:17 Oct 2016]
Abadi DJ, Ahmad Y, Balazinska M, Cetintemel U, Cherniack M, Hwang JH, Lindner W, Maskey AS, Rasin A, Ryvkina E, Tatbul N, Xing Y, Zdonik S (2005) The design of the borealis stream processing engine, in CIDR’05, pp. 277–289
Wu E, Diao Y, Rizvi S (2006) High-performance complex event processing over streams, in Proceedings of the ACM SIGMOD international conference on Management of data, SIGMOD ‘06, New York, pp. 407–418
Armbrust M, Fox A, Griffith R, Joseph AD, Katz R, Konwinski A, Lee G, Patterson D, Rabkin A, Stoica I, Zaharia M (2010) A view of cloud computing. Commun ACM 53(4):50–58
Almeida V, Arlitt M, Rolia J (2002) Analyzing a Web-Based System’s Performance measures at Multiple Time Scales. SIGMETRICS Performance. Eval Rev 30(2):3–9
Jacyno M, Bullock S, Payne T, Luck M (2007) Understanding decentralised control of resource allocation in a minimal multi-agent system, Autonomous Agents and Multi-Agent Systems (AAMAS 07), Article No. 208
Right Scale Cloud Management. http://www.rightscale.com/ [Accessed date :17 Oct 2016]
Botran TL, Alonso JM, Lozano JA (2014) A review of Auto-scaling Techniques for Elastic Applications in Cloud Environments. Journal of Grid Computing 12(4):559–592
Khatua S, Ghosh A, Mukherjee N (2010) Optimizing the utilization of virtual resources in Cloud environment, IEEE International Conference on Virtual Environments, Human-Computer Interfaces and Measurement Systems, pp. 82–87
Eldin AA, Tordsson J, Elmroth E. (2012) An adaptive hybrid elasticity controller for cloud infrastructures, In Network Operations and Management Symposium (NOMS), IEEE, pp. 204–212
Patikirikorala T, Colman A. (2010) Feedback controllers in the cloud. APSEC 2010, Cloud workshop, Sydney
Heinze M, Pappalardo V, Jerzak Z, Fetzer C (2014) Auto-scaling Techniques for Elastic Data Stream Processing, Proceedings of the 8th ACM International Conference on Distributed Event-Based Systems, DEBS '14, Mumbai, India, pp. 318–321
Chevaleyre Y, Dunne PE, Endriss U, Lang J, Lemaitre M, Maudet N, Padget J, Phelps S, Aguilar RJA, Sousa P (2006) Issues in Multiagent Resource Allocation. Informatica 30:3–31
Sutton RS, Barto AG (2012) Reinforcement Learning: An Introduction, First Edition
Maillard OA, Coulom R, Preux P (2005) Parallelization of the TD(λ) Learning Algorithm, In Proceedings of the 7th European Workshop on Reinforcement Learning, EWRL7
Rao J, Bu X, Xu CZ, Wang L, Yin G (2009) VCONF: a reinforcement learning approach to virtual machines auto-configuration. In Proceedings of the 6th international conference on Autonomic computing, ICAC '09, New York, pp. 137–146
Barett E, Howley E, Duggan J (2012) Applying reinforcement learning towards automating resource allocation and application scalability in the cloud. Concurrency and Computation: Practice and Experience 25(12):1656–1674
Tesauro G, Jong NK, Das R, Bennani MN (2007) On the use of hybrid reinforcement learning for autonomic resource allocation. Clust Comput 10:287–299
Dutreilh X, Kirgizov S, Melekhova O, Malenfant J, Rivierre N, Truck I, (2011) Using Reinforcement Learning for Autonomic Resource Allocation in Clouds: towards a fully automated workfloow, In Seventh International Conference on Autonomic and Autonomous Systems, ICAS 2011, IEEE, pp. 67–74, May 2011
Dutreilh X, Moreau A, Malenfant J, Rivierre N, Truck I (2010) From data center resource allocation to control theory and back, In Cloud Computing (CLOUD), IEEE, pp. 410–417
Jamshidi P, Sharifloo AM, Metzger A, Estrada G (2015 Self-Learning Cloud Controllers: Fuzzy Q-Learning for Knowledge Evolution, Proceedings of the International Conference on Cloud and Autonomic Computing, Washington, pp. 208–211
Veni T, Bhanu SMS (2016) Auto-scale: automatic scaling of virtualised resources using neuro-fuzzy reinforcement learning approach. International Journal of Big Data Intelligence 3(3)
Bu X, Rao J, Xu CZ (2013) Coordinated Self-Configuration of Virtual Machines and Appliances Using a Model-Free Learning Approach. IEEE Trans Parallel Distrib Syst 24(4):681–690
RUBiS: Rice University Bidding System Benchmark, http://rubis.ow2.org/. Accessed 17 Oct 2016
RUBBoS: Bulletin Board Benchmark, http://jmob.ow2.org/rubbos.html. Accessed 17 Oct 2016
Olio/PHP, https://svn.apache.org/repos/asf/incubator/olio/docs/php_setup_kit.html. Accessed 17 Oct 2016
Bibal Benifa JV, Dejey D (2017) Performance Improvement of MapReduce for Heterogeneous Clusters Based on Efficient Locality and Replica Aware Scheduling (ELRAS) Strategy. Wireless Pers Commun. https://doi.org/10.1007/s11277-017-3953-5
Arabnejad H, Jamshidi P, Estrada G, Ioini NE, Pahl C (2016) An Auto-Scaling Cloud Controller using Fuzzy Q-Learning - Implementation in OpenStack, Service-Oriented and Cloud Computing, pp. 152–167, Vienna, September 5–7
Kretchmar RM (2002) Parallel Reinforcement Learning, in The 6th World Conference on Systemics, Cybernetics, and Informatics
Amoui M, Salehie M, Mirarab S, Tahvildari L (2005) Adaptive Action Selection in Autonomic Software Using Reinforcement Learning, Proceedings of Second International Conference on Autonomic Computing, Washington, USA
Tesauro G, Das R, Walsh WE, Kephart JO (2005) Utility-Function-Driven Resource Allocation in Autonomic Systems”, Proceedings of the Second International Conference on Autonomic Computing (ICAC’05), Washington, USA
Acknowledgements
Anna University Regional Campus, Tirunelveli’s support for the work in terms of computing facilities is greatly acknowledged.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Bibal Benifa, J.V., Dejey, D. RLPAS: Reinforcement Learning-based Proactive Auto-Scaler for Resource Provisioning in Cloud Environment. Mobile Netw Appl 24, 1348–1363 (2019). https://doi.org/10.1007/s11036-018-0996-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11036-018-0996-0