Skip to main content
SpringerLink
Log in

Combined use of coral reefs optimization and reinforcement learning for improving resource utilization and load balancing in cloud environments

  • Regular Paper
  • Published:
Computing Aims and scope Submit manuscript

Abstract

Resource management is the process of task scheduling and resource provisioning to provide requirements of cloud users. Since cloud resources are often heterogeneous, task scheduling and resource provisioning are major challenges in this area. Various methods have been introduced to improve resource utilization and thus increase the efficiency of cloud computing. Existing methods can be divided into several categories, including mathematical and statistical methods, heuristic- and meta-heuristic-based methods, and machine-learning-based methods. Since the resource management problem is NP-complete, several optimization methods have been also exploited in this area. Coral reefs algorithm is an evolutionary method that has showed appropriate convergence and response time for some problems, and thus is used in this paper to combine with reinforcement learning to improve efficiency of resource management in cloud environments. The proposed method of this paper consists of two phases. The initial allocation of resources to ready-to-perform tasks is done using the coral reefs algorithm in the first phase. The tasks are considered as corals and the resources are considered reefs in this method. The second phase utilizes reinforcement learning to avoid falling into the local optima and to make optimal use of resources using a long-term approach. The proposed model of this paper, called MO-CRAML, introduces a new hybrid algorithm for improving utilization and load balancing of cloud resources using the combination of coral reefs optimization algorithm and reinforcement learning. The results of the experiments show that the proposed algorithm has better performance in cloud resource utilization and load balancing in comparison with some other important methods of the literature.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Zhou Q (2016) Research on heterogeneous data integration model of group enterprise based on cluster computing. Cluster Comput 19(3):1275–1282

    Google Scholar 

  2. Schwiegelshohn U, Badia RM, Bubak M, Danelutto M, Dustdar S, Gagliardi F, Geiger A, Hluchy L, Kranzlmüller D, Laure E, Priol T (2010) Perspectives on grid computing. Future Gener Comput Syst 26(8):1104–1115

    Google Scholar 

  3. Zhang Q, Cheng L, Boutaba R (2010) Cloud computing: state-of-the-art and research challenges. J Internet Serv Appl 1(1):7–18

    Google Scholar 

  4. Höfer CN, Karagiannis G (2011) Cloud computing services: taxonomy and comparison. J Internet Serv Appl 2(2):81–94

    Google Scholar 

  5. Manvi SS, Shyam GK (2014) Resource management for infrastructure as a service (IaaS) in cloud computing: a survey. J Netw Comput Appl 41:424–440

    Google Scholar 

  6. Ghomi EJ, Rahmani AM, Qader NN (2017) Load-balancing algorithms in cloud computing: a survey. J Netw Comput Appl 88:50–71

    Google Scholar 

  7. Salcedo-Sanz S (2017) A review on the coral reefs optimization algorithm: new development lines and current applications. Prog Artif Intell 6(1):1–15

    Google Scholar 

  8. Wiering M, Von Otterlo M (2012) Reinforcement learning: state-of-the-art. Adaptation, learning, and optimization. Springer, Berlin

    Google Scholar 

  9. Parejo JA, Ruiz-Cortés A, Lozano S, Fernandez P (2012) Metaheuristic optimization frameworks: a survey and benchmarking. Soft Comput 16(3):527–561

    Google Scholar 

  10. Von Lücken C, Barán B, Brizuela C (2014) A survey on multi-objective evolutionary algorithms for many-objective problems. Comput optim appl 58(3):707–756

    MathSciNet  MATH  Google Scholar 

  11. Sohrabi MK, Azgomi H (2020) A survey on the combined use of optimization methods and game theory. Arch Comput Methods Eng 27:59–80

    MathSciNet  Google Scholar 

  12. Kalra M, Singh S (2015) A review of metaheuristic scheduling techniques in cloud computing. Egypt Inf J 16(3):275–295

    Google Scholar 

  13. Gong Y-J, Chen W-N, Zhan Z-H, Zhang J, Li Y, Zhang Q, Li J-J (2015) Distributed evolutionary algorithms and their models: a survey of the state-of-the-art. Appl Soft Comput 34:286–300

    Google Scholar 

  14. Gen M, Lin L (2008) Genetic algorithms. Wiley encyclopedia of computer science and engineering. Wiley, Hoboken. https://doi.org/10.1002/9780470050118.ecse169

    Book  Google Scholar 

  15. Akbari M, Rashidi H, Alizadeh SH (2017) An enhanced genetic algorithm with new operators for task scheduling in heterogeneous computing systems. Eng Appl Artif Intell 61:35–46

    Google Scholar 

  16. Guerrero C, Lera I, Juiz C (2018) Genetic algorithm for multi-objective optimization of container allocation in cloud architecture. J Grid Comput 16(1):113–135

    Google Scholar 

  17. Kamalinia A, Ghaffari A (2017) Hybrid task scheduling method for cloud computing by genetic and DE algorithms. Wirel Pers Commun 97(4):6301–6323

    Google Scholar 

  18. Karimi MB, Isazadeh A, Rahmani AM (2017) QoS-aware service composition in cloud computing using data mining techniques and genetic algorithm. J Supercomput 73(4):1387–1415

    Google Scholar 

  19. Kaur K, Kaur N, Kaur K (2018) A novel context and load-aware family genetic algorithm based task scheduling in cloud computing. In: Satapathy S, Bhateja V, Raju K, Janakiramaiah B (eds) Data engineering and intelligent computing. Advances in intelligent systems and computing, vol 542. Springer, Singapore. https://doi.org/10.1007/978-981-10-3223-3_51

    Chapter  MATH  Google Scholar 

  20. Keshanchi B, Souri A, Navimipour NJ (2017) An improved genetic algorithm for task scheduling in the cloud environments using the priority queues: formal verification, simulation, and statistical testing. J Syst Softw 124:1–21

    Google Scholar 

  21. Krause J, Cordeiro J, Parpinelli RS, Lopes HS (2013) A survey of swarm algorithms applied to discrete optimization problems. In: Swarm intelligence and bio-inspired computation, theory and applications. pp 169–191

  22. Masdari M, Salehi F, Jalali M, Bidaki M (2017) A survey of PSO-based scheduling algorithms in cloud computing. J Netw Syst Manag 25(1):122–158

    Google Scholar 

  23. Verma A, Kaushal S (2017) A hybrid multi-objective particle swarm optimization for scientific workflow scheduling. Parallel Comput 62:1–19

    MathSciNet  Google Scholar 

  24. Liu Z, Wang X (2012) A PSO-based algorithm for load balancing in virtual machines of cloud computing environment. In: Tan Y, Shi Y, Ji Z (eds) Advances in swarm intelligence. ICSI 2012. Lecture notes in computer science, vol 7331. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-30976-2_17

    Chapter  Google Scholar 

  25. Mohan BC, Baskaran R (2012) A survey: ant colony optimization based recent research and implementation on several engineering domain. Exp Syst Appl 39(4):4618–4627

    Google Scholar 

  26. Gao Y, Guan H, Qi Z, Hou Y, Liu L (2013) A multi-objective ant colony system algorithm for virtual machine placement in cloud computing. J Comput Syst Sci 79(8):1230–1242

    MathSciNet  MATH  Google Scholar 

  27. Wu Z, Liu X, Ni Z, Yuan D, Yang Y (2013) A market-oriented hierarchical scheduling strategy in cloud workflow systems. J Supercomput 63(1):256–293

    Google Scholar 

  28. Gandomi AH, Yang X-S, Alavi AH (2013) Cuckoo search algorithm: a metaheuristic approach to solve structural optimization problems. Eng Comput 29(1):17–35

    Google Scholar 

  29. Agarwal M, Srivastava GMS (2018) A cuckoo search algorithm-based task scheduling in cloud computing. In: Bhatia S, Mishra K, Tiwari S, Singh V (eds) Advances in computer and computational sciences. Advances in intelligent systems and computing, vol 554. Springer, Singapore. https://doi.org/10.1007/978-981-10-3773-3_29

    Chapter  Google Scholar 

  30. Tavana M, Shahdi-Pashaki S, Teymourian E, Santos-Arteaga FJ, Komaki M (2018) A discrete cuckoo optimization algorithm for consolidation in cloud computing. Comput Ind Eng 115:495–511

    Google Scholar 

  31. Salcedo-Sanz S, Del Ser J, Landa-Torres I, Gil-López S, Portilla-Figueras JA (2014) The coral reefs optimization algorithm: a novel metaheuristic for efficiently solving optimization problems. Sci World J 2014:1–15

    Google Scholar 

  32. Salcedo-Sanz S, Cuadra L, Vermeij MJA (2016) A review of computational intelligence techniques in coral reef-related applications. Ecol Inf 32:107–123

    Google Scholar 

  33. Azgomi H, Sohrabi MK (2019) A novel coral reefs optimization algorithm for materialized view selection in data warehouse environments. Appl Intell 49(11):3965–3989

    Google Scholar 

  34. Ficco M, Esposito C, Palmieri F, Castiglione A (2018) A coral-reefs and game theory-based approach for optimizing elastic cloud resource allocation. Future Gener Comput Syst 78:343–352

    Google Scholar 

  35. Chen W, Xu H (2019) Cloud computing task scheduling method based on a coral reefs optimization algorithm. In: 2019 IEEE 25th international conference on parallel and distributed systems (ICPADS). https://doi.org/https://doi.org/10.1109/ICPADS47876.2019.00013

  36. Xiao Y, Chen D, Wei D, Li Q, Wang H, Xu M (2019) Rumor propagation dynamic model based on evolutionary game and anti-rumor. Nonlinear Dyn 95(1):523–539

    Google Scholar 

  37. Xiao Y, Song C, Liu Y (2019) Social hotspot propagation dynamics model based on multidimensional attributes and evolutionary games. Commun Nonlinear Sci 67:13–25

    MathSciNet  MATH  Google Scholar 

  38. Xiao Y, Yang Q, Sang C, Liu Y (2020) rumor diffusion model based on representation learning and anti-rumor. IEEE T Netw Serv Man 13(3):1910–1923

    Google Scholar 

  39. Asghari A, Sohrabi MK, Yaghmaee F (2020) Online scheduling of dependent tasks of cloud’s workflows to enhance resource utilization and reduce the makespan using multiple reinforcement learning-based agents. Soft Comput 24:16177–16199

    Google Scholar 

  40. Asghari A, Sohrabi MK, Yaghmaee F (2020) A cloud resource management framework for multiple online scientific workflows using cooperative reinforcement learning agents. Comput Netw 179:107340

    Google Scholar 

  41. Asghari A, Sohrabi MK, Yaghmaee F (2021) Task scheduling, resource provisioning, and load balancing on scientific workflows using parallel SARSA reinforcement learning agents and genetic algorithm. J Supercomput 77:2800–2828

    Google Scholar 

  42. Tong Z, Deng X, Chen H, Mei J, Liu H (2020) QL-HEFT: a novel machine learning scheduling scheme base on cloud computing environment. Neural Comput Appl 32:5553–5570

    Google Scholar 

  43. Topcuoglu H, Hariri S, Wu M-Y (2002) Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans Parall Distrib 13(3):260–274

    Google Scholar 

  44. Kumar J, Singh AK, Mohan A (2020) Resource-efficient load-balancing framework for cloud data center networks. ETRI J. https://doi.org/10.4218/etrij.2019-0294

    Article  Google Scholar 

  45. Premarathne US, Rajasingham S (2020) Trust based multi-agent cooperative load balancing system (TCLBS). Future Gener Comput Syst 112:185–192

    Google Scholar 

  46. Devaraj AFS, Elhoseny M, Dhanasekaran S, Lydia EL, Shankar K (2020) Hybridization of firefly and improved multi-objective particle swarm optimization algorithm for energy efficient load balancing in cloud computing environments. J Parallel Distrib Comput 142:36–45

    Google Scholar 

  47. Adhikari M, Nandy S, Amgoth T (2019) Meta heuristic-based task deployment mechanism for load balancing in IaaS cloud. J Netw Comput Appl 128:64–77

    Google Scholar 

  48. Priya V, Kumar CS, Kannan R (2019) Resource scheduling algorithm with load balancing for cloud service provisioning. Appl Soft Comput 76:416–424

    Google Scholar 

  49. Singh P, Dutta M, Aggarwal N (2017) A review of task scheduling based on meta-heuristics approach in cloud computing. Knowl Inf Syst 52(1):1–51

    Google Scholar 

  50. Kumar A, Bawa S (2019) Generalized ant colony optimizer: swarm-based meta-heuristic algorithm for cloud services execution. Computing 101:1609–1632

    MathSciNet  Google Scholar 

  51. Nirmala SJ, Bhanu SMS (2016) Catfish-PSO based scheduling of scientific workflows in IaaS cloud. Computing 98:1091–1109

    MathSciNet  Google Scholar 

  52. Durán-Rosal AM, Gutiérrez PA, Salcedo-Sanz S, Hervás-Martínez C (2018) A statistically-driven coral reef optimization algorithm for optimal size reduction of time series. Appl Soft Comput 63:139–153

    Google Scholar 

  53. Genin A, Karp L, Miroz A (1994) Effects of flow on competitive superiority in scleractinian corals. Limnol Oceanogr 39(4):913–924

    Google Scholar 

  54. Chadwick NE (1987) Interspecific aggressive behavior of the corallimorpharian Corynactis californica (Cnidaria: Anthozoa): effects on sympatric corals and sea anemones. Biol Bull 173(1):110–125

    Google Scholar 

  55. Salcedo-Sanz S, Del Ser J, Landa-Torres I, Gil-López S, Portilla-Figueras JA (2013) The coral reefs optimization algorithm: an efficient meta-heuristic for solving hard optimization problems. In: Proceedings of the 15th international conference on applied stochastic models and data analysis (ASMDA2013), Mataró. pp. 751–758

  56. Stone P, Veloso M (2000) Multiagent systems: a survey from a machine learning perspective. Auton Robot 8(3):345–383

    Google Scholar 

  57. Kaelbling LP, Littman ML, Moore AW (1996) Reinforcement learning: a survey. J Artif Intell Res 4:237–285

    Google Scholar 

  58. Watkins CJCH, Dayan P (1992) Q-learning. Mach Learn 8:279–292

    MATH  Google Scholar 

  59. Yang K, Gu J, Zhao T, Sun G (2011) An optimized control strategy for load balancing based on live migration of virtual machine. In: 2011 6th annual chinagrid conference. IEEE, pp 141–146

  60. Calheiros RN, Ranjan R, Beloglazov A, De Rose CAF, Buyya R (2011) CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Softw Pract Exp 41(1):23–50

    Google Scholar 

  61. Ramezani F, Lu J, Hussain FK (2014) Task-based system load balancing in cloud computing using particle swarm optimization. Int J Parallel Prog 42(5):739–754

    Google Scholar 

  62. Wang B, Li J (2016) Load balancing task scheduling based on multi-population genetic algorithm in cloud computing. In: 2016 35th Chinese control conference (CCC). IEEE, pp 5261–5266

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Semnan Branch, Islamic Azad University, Semnan, Iran

    Ali Asghari & Mohammad Karim Sohrabi

Authors
  1. Ali Asghari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammad Karim Sohrabi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammad Karim Sohrabi.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Asghari, A., Sohrabi, M.K. Combined use of coral reefs optimization and reinforcement learning for improving resource utilization and load balancing in cloud environments. Computing 103, 1545–1567 (2021). https://doi.org/10.1007/s00607-021-00920-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00607-021-00920-2

Keywords

Mathematics Subject Classification

Search

Navigation