Abstract
With the advent of cloud computing and the availability of data collected from increasingly powerful scientific instruments, workflows have become a prevailing mean to achieve significant scientific advances at an increased pace. Scheduling algorithms are crucial in enabling the efficient automation of these large-scale workflows, and considerable effort has been made to develop novel heuristics tailored for the cloud resource model. The majority of these algorithms focus on coarse-grained billing periods that are much larger than the average execution time of individual tasks. Instead, our work focuses on emerging finer-grained pricing schemes (e.g., per-minute billing) that provide users with more flexibility and the ability to reduce the inherent wastage that results from coarser-grained ones. We propose a scheduling algorithm whose objective is to optimize a workflow’s execution time under a budget constraint; quality of service requirement that has been overlooked in favor of optimizing cost under a deadline constraint. Our proposal addresses fundamental challenges of clouds such as resource elasticity, abundance, and heterogeneity, as well as resource performance variation and virtual machine provisioning delays. The simulation results demonstrate our algorithm’s responsiveness to environmental uncertainties and its ability to generate high-quality schedules that comply with the budget constraint while achieving faster execution times when compared to state-of-the-art algorithms.
- Eun-Kyu Byun, Yang-Suk Kee, Jin-Soo Kim, and Seungryoul Maeng. 2011. Cost optimized provisioning of elastic resources for application workflows. Future Gener. Comput. Syst. 27, 8 (2011), 1011--1026. Google ScholarDigital Library
- Rodrigo N. Calheiros, Rajiv Ranjan, Anton Beloglazov, César A. F. De Rose, and Rajkumar Buyya. 2011. CloudSim: A toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Softw. Pract. Exper. 41, 1 (2011), 23--50. Google ScholarDigital Library
- Thiago A. L. Genez, Luiz F. Bittencourt, and Edmundo R. M. Madeira. 2012. Workflow scheduling for SaaS/PaaS cloud providers considering two SLA levels. In Proceedings of the Network Operations Management Symposium (NOMS’12).Google Scholar
- Yolanda Gil, Ewa Deelman, Mark Ellisman, Thomas Fahringer, Geoffrey Fox, Dennis Gannon, Carole Goble, Miron Livny, Luc Moreau, and Jim Myers. 2007. Examining the challenges of scientific workflows. IEEE Comput. 40, 12 (2007), 26--34. Google ScholarDigital Library
- Google. 2015a. Amazon Simple Storage Service. (Nov 2015). Retrieved October 2016 from http://aws.amazon.com/s3/.Google Scholar
- Google. 2015b. Google Cloud Storage. (Nov 2015). Retrieved October 2016 from https://cloud.google.com/storage/.Google Scholar
- Google. 2015c. Google Compute Engine. (Nov 2015). Retrieved October 2016 from https://cloud.google.com/compute/.Google Scholar
- A. Gupta and D. Milojicic. 2011. Evaluation of HPC applications on cloud. In Proceedings of the 2011 6th Open Cirrus Summit (OCS’11). 22--26. Google ScholarDigital Library
- A. Iosup, S. Ostermann, M. N. Yigitbasi, R. Prodan, T. Fahringer, and D. Epema. 2011. Performance analysis of cloud computing services for many-tasks scientific computing. IEEE Trans. Parallel Distrib. Syst. 22, 6 (June 2011), 931--945. Google ScholarDigital Library
- Keith R. Jackson, Lavanya Ramakrishnan, Krishna Muriki, Shane Canon, Shreyas Cholia, John Shalf, Harvey J. Wasserman, and Nicholas J. Wright. 2010. Performance analysis of high performance computing applications on the amazon web services cloud. In Proceedings of the International Conference on Cloud Computing Technology and Science (CloudCom). Google ScholarDigital Library
- Gideon Juve, Ann Chervenak, Ewa Deelman, Shishir Bharathi, Gaurang Mehta, and Karan Vahi. 2013. Characterizing and profiling scientific workflows. Future Gener. Comput. Syst. 29, 3 (2013), 682--692. Google ScholarDigital Library
- Maciej Malawski, Kamil Figiela, Marian Bubak, Ewa Deelman, and Jarek Nabrzyski. 2015. Scheduling multilevel deadline-constrained scientific workflows on clouds based on cost optimization. Sci. Program. 2015, 5 (2015). Google ScholarDigital Library
- Maciej Malawski, Gideon Juve, Ewa Deelman, and Jarek Nabrzyski. 2012. Cost-and deadline-constrained provisioning for scientific workflow ensembles in IaaS clouds. In Proceedings of the International Conference on High Performance Computing, Networking, and Storage Analysis (SC’12). Google ScholarDigital Library
- Ming Mao and Marty Humphrey. 2011. Auto-scaling to minimize cost and meet application deadlines in cloud workflows. In Proceedings of the International Conference on High Performance Computing, Networking, and Storage Analysis (SC’11). Google ScholarDigital Library
- Ming Mao and Marty Humphrey. 2013. Scaling and scheduling to maximize application performance within budget constraints in cloud workflows. In Proceedings of the International Parallel 8 Distributed Processing Symposium (IPDPS'13). IEEE, 67--78. Google ScholarDigital Library
- Microsoft. 2015. Microsoft Azure. (Nov 2015). Retrieved October 2016 from https://azure.microsoft.com.Google Scholar
- Simon Ostermann, Alexandria Losup, Nezih Yigitbasi, Radu Prodan, Thomas Fahringer, and Dick Epema. 2010. A performance analysis of EC2 cloud computing services for scientific computing. In Cloud Computing. Springer, 115--131.Google Scholar
- Ilia Pietri, Maciej Malawski, Gideon Juve, Ewa Deelman, Jarek Nabrzyski, and Rizos Sakellariou. 2013. Energy-constrained provisioning for scientific workflow ensembles. In Proceedings of the International Conference on Cloud Green Computing (CGC’13). Google ScholarDigital Library
- Deepak Poola, Kotagiri Ramamohanarao, and Rajkumar Buyya. 2014. Fault-tolerant workflow scheduling using spot instances on clouds. Proc. Comput. Sci. 29 (2014), 523--533.Google ScholarCross Ref
- Rackspace. 2015. Rackspace Block Storage. (Nov 2015). Retrieved October 2016 from http://www.rackspace.com.au/cloud/block-storage.Google Scholar
- Jörg Schad, Jens Dittrich, and Jorge-Arnulfo Quiané-Ruiz. 2010. Runtime measurements in the cloud: Observing, analyzing, and reducing variance. Proc. VLDB Endow. 3, 1--2 (2010), 460--471. Google ScholarDigital Library
- Jianwu Wang, Prakashan Korambath, Ilkay Altintas, Jim Davis, and Daniel Crawl. 2014. Workflow as a service in the cloud: Architecture and scheduling algorithms. Proc. Comput. Sci. 29 (2014), 546--556.Google ScholarCross Ref
- Chunlin Wu, Xingqin Lin, Daren Yu, Wei Xu, and Luoqing Li. 2015. End-to-end delay minimization for scientific workflows in clouds under budget constraint. IEEE Trans. Cloud Comput. 3, 2 (2015), 169--181.Google ScholarCross Ref
- Zhangjun Wu, Zhiwei Ni, Lichuan Gu, and Xiao Liu. 2010. A revised discrete particle swarm optimization for cloud workflow scheduling. In Proceedings of the International Conference on Computational Intelligence Security (CIS’10). Google ScholarDigital Library
- Jia Yu, Kotagiri Ramamohanarao, and Rajkumar Buyya. 2009. Deadline/budget-based scheduling of workflows on utility grids. Market-Oriented Grid and Utility Computing (2009), John Wiley 8 Sons, Inc., 427--450.Google ScholarCross Ref
- Lingfang Zeng, Bharadwaj Veeravalli, and Xiaorong Li. 2012. Scalestar: Budget conscious scheduling precedence-constrained many-task workflow applications in cloud. In Proceedings of the International Conference on Advanced Information Network Applications (AINA’12). Google ScholarDigital Library
- Amelie Chi Zhou, Bingsheng He, and Cheng Liu. 2016. Monetary cost optimizations for hosting workflow-as-a-service in IaaS clouds. IEEE Trans. Cloud Comput. 4, 1 (2016), 34--48. Google ScholarDigital Library
Index Terms
- Budget-Driven Scheduling of Scientific Workflows in IaaS Clouds with Fine-Grained Billing Periods
Recommendations
Deadline distribution strategies for scientific workflow scheduling in commercial clouds
UCC '16: Proceedings of the 9th International Conference on Utility and Cloud ComputingCommercial clouds have become a viable platform for performing a significant range of large scale scientific analyses - due to the offerings of elasticity, specialist hardware, software infrastructure and pay-as-you-go cost model. Such clouds represent ...
A Provenance-based Adaptive Scheduling Heuristic for Parallel Scientific Workflows in Clouds
In the last years, scientific workflows have emerged as a fundamental abstraction for structuring and executing scientific experiments in computational environments. Scientific workflows are becoming increasingly complex and more demanding in terms of ...
Adaptive Resource Provisioning and Scheduling Algorithm for Scientific Workflows on IaaS Cloud
AbstractScientific workflow applications are deployed to run extensive volumes of data and to manage comprehensive observations and simulations. They are resource-intensive and time-utilizing applications that profit from processing in distributed ...
Comments