Skip to main content
SpringerLink
Log in

An adaptive multi-objective evolutionary algorithm for constrained workflow scheduling in Clouds

  • Published:
Distributed and Parallel Databases Aims and scope Submit manuscript

Abstract

The Cloud workflow scheduling is to find proper Cloud resources for the execution of workflow tasks to efficiently utilize resources and meet different user’s quality of service requirements. Cloud workflow scheduling is a constrained and NP-complete problem and multi-objective evolutionary algorithms have shown their excellent ability to solve such problem. But most existing works simply use static penalty function to handle constraints which usually result in premature when the constraints become strict. On the other hand, with the search space being more tremendous and chaotic, how to balance the ability of exploring the entire search space and exploiting the important regions during the evolutionary process is increasingly important. In this paper, an adaptive individual-assessment scheme based on evolutionary states is proposed to handle the constraints in multi-objective optimization problems. In addition, the evolutionary parameters are also adjusted accordingly to balance the exploration and exploitation ability. These are distinguishable from most previous studies that directly incorporate multi-objective evolutionary algorithm to search excellent solutions for Cloud workflow scheduling. Experimental results demonstrate the proposed algorithm outperforms other state-of-the-art methods in convergence and diversity, and it also achieves better optimization ability when it is applied to solve Cloud workflow scheduling problem.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Vöckler, J.S., Juve, G., Deelman, E., et al.: Experiences using cloud computing for a scientific workflow application, pp. 15–24. International Workshop on Scientific Cloud, Computing (2011)

  2. Zhu, Z., Zhang, G., Li, M., et al.: Evolutionary multi-objective workflow scheduling in cloud. IEEE Trans. Parallel Distrib. Syst. 27(5), 1344–1357 (2016)

    Article  Google Scholar 

  3. Juve, G., Chervenak, A., Deelman, E., et al.: Characterizing and profiling scientific workflows. Future Gener. Comput. Syst. 29(3), 682–692 (2013)

    Article  Google Scholar 

  4. Buyya, R., Yeo, C.S., Venugopal, S., et al.: Cloud computing and emerging IT platforms: vision, hype, and reality for delivering computing as the 5th utility. Future Gener. Comput. Syst. 25(6), 599–616 (2009)

    Article  Google Scholar 

  5. Foster, I., Zhao, Y., Raicu, I., et al.: Cloud computing and grid computing 360-degree compared. Grid Computing Environments Workshop, 2008. GCE ’08, pp. 1–10. IEEE, New York (2009)

  6. Wang, X., Yeo, C.S., Buyya, R., et al.: Optimizing the makespan and reliability for workflow applications with reputation and a look-ahead genetic algorithm. Future Gener. Comput. Syst. 27(8), 1124–1134 (2011)

    Article  Google Scholar 

  7. Deb, K., Pratap, A., Agarwal, S., et al.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 6(2), 182–197 (2002)

    Article  Google Scholar 

  8. Liu, L., Zhang, M.: Multi-objective optimization model with AHP decision-making for Cloud service composition. Ksii Trans. Internet Inform. Syst. 9(9), 3293–3311 (2015)

    Google Scholar 

  9. Zhang, Q., Li, H.: MOEA/D: a multiobjective evolutionary algorithm based on decomposition. IEEE Trans. Evol. Comput. 11(6), 712–731 (2008)

    Article  Google Scholar 

  10. Zitzler, E., Laumanns, M., Thiele, L.: SPEA2: Improving the Strength Pareto Evolutionary Algorithm. Springer, Berlin (2001)

    Google Scholar 

  11. Coello, C.A.C., Pulido, G.T., Lechuga, M.S.: Handling multiple objectives with particle swarm optimization. IEEE Trans. Evol. Comput. 8(3), 256–279 (2004)

    Article  Google Scholar 

  12. Rodriguez, M.A., Buyya, R.: Deadline based resource provisioning and scheduling algorithm for scientific workflows on Clouds. IEEE Trans. Cloud Comput. 2(2), 222–235 (2014)

    Article  Google Scholar 

  13. Malawski, M., Juve, G., Deelman, E., et al.: Cost- and deadline-constrained provisioning for scientific workflow ensembles in IaaS clouds. In: SC ’12 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, pp. 1–18 (2012)

  14. Liu, L., Zhang, M., Lin, Y., et al.: A survey on workflow management and scheduling in cloud computing. IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, pp. 837–846. ACM, New York (2014)

  15. Deng, K., Ren, K., Zhu, M., et al.: A data and task co-scheduling algorithm for scientific Cloud workflows. IEEE Trans. Cloud Comput. 7161, 1 (2015)

    Article  Google Scholar 

  16. Goldberg, D.E.: Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley Longman Publishing Co. Inc., Boston (1989)

    MATH  Google Scholar 

  17. Zhang, J., Chung, S.H., Lo, W.L.: Clustering-based adaptive crossover and mutation probabilities for genetic algorithms. IEEE Trans. Evol. Comput. 11(3), 326–335 (2007)

    Article  Google Scholar 

  18. Hu, W., Yen, G.G.: Adaptive multiobjective particle swarm optimization based on parallel cell coordinate system. IEEE Trans. Evol. Comput. 19(1), 1–18 (2015)

    Article  Google Scholar 

  19. Inselberg, A.: The plane with parallel coordinates. Visual Comput. 1(2), 69–91 (1985)

    Article  MathSciNet  MATH  Google Scholar 

  20. Hu, W., Yen, G.G., Zhang, X.: Multiobjective particle swarm optimization based on Pareto entropy. J. Softw. 25(5), 1025–1050 (2014). (in Chinese)

    MATH  Google Scholar 

  21. Toombs, R., Reed, J., Barricelli, N.A.: Simulation of biological evolution and machine learning. J. Theor. Biol. 17(3), 319 (1967)

    Article  Google Scholar 

  22. Zhi-Hui, Z., Jun, Z., Yun, L., et al.: Adaptive particle swarm optimization. IEEE Trans. Syst. Man Cybern. Part B 39(6), 1362–81 (2009)

    Article  Google Scholar 

  23. Calheiros, R.N., Ranjan, R., Beloglazov, A., et al.: CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Softw. Pract. Exp. 41(1), 23–50 (2011)

    Article  Google Scholar 

  24. Chen, W., Deelman, E.: WorkflowSim: a toolkit for simulating scientific workflows in distributed environments. In: IEEE International Conference on E-Science. IEEE, pp. 1–8 (2012)

  25. Nanakorn, P., Meesomklin, K.: An adaptive penalty function in genetic algorithms for structural design optimization. Comput. Struct. 79(29–30), 2527–2539 (2001)

    Article  Google Scholar 

  26. Tessema, B., Yen, G.G.: An adaptive penalty formulation for constrained evolutionary optimization. IEEE Trans. Syst. Man Cybern. Part A 39(3), 565–578 (2009)

    Article  Google Scholar 

  27. Deb, K., Jain, H.: An evolutionary many-objective optimization algorithm using reference-point-based nondominated sorting approach, part I: solving problems with box constraints. IEEE Trans. Evol. Comput. 18(4), 577–601 (2014)

    Article  Google Scholar 

  28. Zhang, Q., Zhou, A., Zhao, S., et al.: Multiobjective optimization test instances for the CEC 2009 special session and competition. Technical Report, University of Essex (2008)

  29. Deb, K., Sinha, A., Kukkonen, S.: Multi-objective test problems, linkages, and evolutionary methodologies. In: Genetic and Evolutionary Computation Conference, GECCO 2006, Proceedings, Seattle, Washington, USA, pp. 1141–1148 (2006)

  30. Deb, K., Kalyanmoy, D.: Multi-Objective Optimization Using Evolutionary Algorithms. Wiley, New York (2001)

    MATH  Google Scholar 

  31. He, Q., Wang, L.: An effective co-evolutionary particle swarm optimization for constrained engineering design problems. Eng. Appl. Artif. Intell. 20(1), 89–99 (2007)

    Article  MathSciNet  Google Scholar 

  32. Topcuoglu, H., Hariri, S., Wu, M.Y.: Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans. Parallel Distrib. Syst. 13(3), 260–274 (2002)

    Article  Google Scholar 

  33. Zheng, W., Sakellariou, R.: Budget-deadline constrained workflow planning for admission control. J. Grid Comput. 11(4), 105–119 (2013)

    Article  Google Scholar 

  34. Arabnejad, H., Barbosa, J.G.: A budget constrained scheduling algorithm for workflow applications. J. Grid Comput. 12(4), 15 (2014)

    Article  Google Scholar 

  35. Durillo, J.J., Prodan, R.: Multi-objective workflow scheduling in amazon ec2. Cluster Comput. 17(2), 169–189 (2014)

    Article  Google Scholar 

  36. Sawant, S.: A genetic algorithm scheduling approach for virtual machine resources in a Cloud computing environment. Master thesis, San Jose State University (2011)

  37. Garg, R., Singh, A.K.: multi-objective workflow grid scheduling based on discrete particle swarm optimization. In: International Conference, Semcco, Visakhapatnam, Andhra Pradesh, India, December, pp. 183–190 (2011)

  38. Garg, R., Singh, A.K.: Multi-objective workflow grid scheduling using \(\epsilon \)-fuzzy dominance sort based discrete particle swarm optimization. J. Supercomput. 68(2), 709–732 (2014)

    Article  Google Scholar 

  39. Yu, J., Kirley, M., Buyya, R.: Multi-objective planning for workflow execution on Grids. In: IEEE/ACM International Conference on Grid Computing. IEEE Computer Society, pp. 10–17 (2007)

  40. Wang, X., Xia, JW., Li, JZ., et al.: Grid workflow scheduling with various QoS constraints using SPEA2+. In: Fourth International Conference on Genetic and Evolutionary Computing. IEEE Xplore, pp. 829–832 (2011)

  41. Bader, J., Zitzler, E.: HypE: an algorithm for fast hypervolume-based many-objective optimization. Evol. Comput. 19(1), 45–76 (2011)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Information and Electronics, Beijing Institute of Technology, Beijing, China

    Miao Zhang & Huiqi Li

  2. School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing, China

    Li Liu

  3. The University of Melbourne, Parkville, Australia

    Rajkumar Buyya

Authors
  1. Miao Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huiqi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Li Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rajkumar Buyya
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huiqi Li.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, M., Li, H., Liu, L. et al. An adaptive multi-objective evolutionary algorithm for constrained workflow scheduling in Clouds. Distrib Parallel Databases 36, 339–368 (2018). https://doi.org/10.1007/s10619-017-7215-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10619-017-7215-z

Keywords

Search

Navigation