On the Feasibility of Simulation-Driven Portfolio Scheduling for Cyberinfrastructure Runtime Systems

Casanova, Henri; Wong, Yick Ching; Pottier, Loic; Ferreira Da Silva, Rafael

doi:10.1007/978-3-031-22698-4_1

Title: On the Feasibility of Simulation-Driven Portfolio Scheduling for Cyberinfrastructure Runtime Systems

Conference · Sun Jan 01 00:00:00 EST 2023

DOI:https://doi.org/10.1007/978-3-031-22698-4_1· OSTI ID:1922323

Casanova, Henri ^[1]; Wong, Yick Ching ^[1]; Pottier, Loic ^[2];

^[3]

University of Hawaii at Manoa, Honolulu
University of Southern California, Information Sciences Institute
ORNL

Runtime systems that automate the execution of applications on distributed cyberinfrastructures need to make scheduling decisions. Researchers have proposed many scheduling algorithms, but most of them are designed based on analytical models and assumptions that may not hold in practice. The literature is thus rife with algorithms that have been evaluated only within the scope of their underlying assumptions but whose practical effectiveness is unclear. It is thus difficult for developers to decide which algorithm to implement in their runtime systems.To obviate the above difficulty, we propose an approach by which the runtime system executes, throughout application execution, simulations of this very execution. Each simulation is for a different algorithm in a scheduling algorithm portfolio, and the best algorithm is selected based on simulation results. The main objective of this work is to evaluate the feasibility and potential merit of this portfolio scheduling approach, even in the presence of simulation inaccuracy, when compared to the traditional one-algorithm approach. We perform this evaluation via a case study in the context of scientific workflows. Our main finding is that portfolio scheduling can outperform the best one-algorithm approach even in the presence of relatively large simulation inaccuracies.

View Conference

Cite

Export

Save

Research Organization:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)

DOE Contract Number:: AC05-00OR22725

OSTI ID:: 1922323

Resource Relation:: Journal Volume: 13592; Conference: 25th Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP 2022) - Lyon, , France - 6/3/2022 4:00:00 AM-6/3/2022 4:00:00 AM

Country of Publication:: United States

Language:: English

References (24)

A Survey on Scheduling Strategies for Workflows in Cloud Environment and Emerging Trends Adhikari, Mainak; Amgoth, Tarachand; Srirama, Satish Narayana ACM Computing Surveys, Vol. 52, Issue 4 https://doi.org/10.1145/3325097	journal	August 2019
Validity of the single processor approach to achieving large scale computing capabilities Amdahl, Gene M. Proceedings of the April 18-20, 1967, spring joint computer conference on - AFIPS '67 (Spring) https://doi.org/10.1145/1465482.1465560	conference	January 1967
Workflow scheduling algorithms in cloud environment - A survey Arya, Lokesh Kumar; Verma, Amandeep 2014 Recent Advances in Engineering and Computational Sciences (RAECS) https://doi.org/10.1109/RAECS.2014.6799514	conference	March 2014
GridSim: a toolkit for the modeling and simulation of distributed resource management and scheduling for Grid computing Buyya, Rajkumar; Murshed, Manzur Concurrency and Computation: Practice and Experience, Vol. 14, Issue 13-15 https://doi.org/10.1002/cpe.710	journal	November 2002
CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms Calheiros, Rodrigo N.; Ranjan, Rajiv; Beloglazov, Anton Software: Practice and Experience, Vol. 41, Issue 1 https://doi.org/10.1002/spe.995	journal	August 2010
Obtaining dynamic scheduling policies with simulation and machine learning Carastan-Santos, Danilo; de Camargo, Raphael Y. Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '17 https://doi.org/10.1145/3126908.3126955	conference	January 2017
Versatile, scalable, and accurate simulation of distributed applications and platforms Casanova, Henri; Giersch, Arnaud; Legrand, Arnaud Journal of Parallel and Distributed Computing, Vol. 74, Issue 10 https://doi.org/10.1016/j.jpdc.2014.06.008	journal	October 2014
Developing accurate and scalable simulators of production workflow management systems with WRENCH Casanova, Henri; Ferreira da Silva, Rafael; Tanaka, Ryan Future Generation Computer Systems, Vol. 112 https://doi.org/10.1016/j.future.2020.05.030	journal	November 2020
WfCommons: A framework for enabling scientific workflow research and development Coleman, Tainã; Casanova, Henri; Pottier, Loïc Future Generation Computer Systems, Vol. 128 https://doi.org/10.1016/j.future.2021.09.043	journal	March 2022
Exploring portfolio scheduling for long-term execution of scientific workloads in IaaS clouds Deng, Kefeng; Song, Junqiang; Ren, Kaijun Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1145/2503210.2503244	conference	November 2013
Self-tuning systems Feitelson, D. G.; Naaman, M. IEEE Software, Vol. 16, Issue 2 https://doi.org/10.1109/52.754053	journal	January 1999
Online Tuning of EASY-Backfilling using Queue Reordering Policies Gaussier, Eric; Lelong, Jerome; Reis, Valentin IEEE Transactions on Parallel and Distributed Systems, Vol. 29, Issue 10 https://doi.org/10.1109/TPDS.2018.2820699	journal	October 2018
Workflow scheduling in heterogeneous computing systems : A survey Gupta, Ashish; Garg, Ritu 2017 International Conference on Computing and Communication Technologies for Smart Nation (IC3TSN) https://doi.org/10.1109/IC3TSN.2017.8284499	conference	October 2017
LogGOPSim: simulating large-scale applications in the LogGOPS model Hoefler, Torsten; Schneider, Timo; Lumsdaine, Andrew Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing - HPDC '10 https://doi.org/10.1145/1851476.1851564	conference	January 2010
DISSECT-CF: A simulator to foster energy-aware scheduling in infrastructure clouds Kecskemeti, Gabor Simulation Modelling Practice and Theory, Vol. 58 https://doi.org/10.1016/j.simpat.2015.05.009	journal	November 2015
Fostering Energy-Awareness in Simulations behind Scientific Workflow Management Systems Kecskemeti, Gabor; Ostermann, Simon; Prodan, Radu 2014 IEEE/ACM 7th International Conference on Utility and Cloud Computing https://doi.org/10.1109/UCC.2014.11	conference	December 2014
A Survey of Data-Intensive Scientific Workflow Management Liu, Ji; Pacitti, Esther; Valduriez, Patrick Journal of Grid Computing, Vol. 13, Issue 4 https://doi.org/10.1007/s10723-015-9329-8	journal	March 2015
CloudNetSim++: A toolkit for data center simulations in OMNET++ Malik, Asad W.; Bilal, Kashif; Aziz, Khurram 2014 11th Annual High Capacity Optical Networks and Emerging/Enabling Technologies (Photonics for Energy) https://doi.org/10.1109/HONET.2014.7029371	conference	December 2014
FogNetSim++: A Toolkit for Modeling and Simulation of Distributed Fog Environment Qayyum, Tariq; Malik, Asad Waqar; Khan Khattak, Muazzam A. IEEE Access, Vol. 6 https://doi.org/10.1109/ACCESS.2018.2877696	journal	January 2018
A taxonomy and survey on scheduling algorithms for scientific workflows in IaaS cloud computing environments: Workflow Scheduling Algorithms for Clouds Rodriguez, Maria Alejandra; Buyya, Rajkumar Concurrency and Computation: Practice and Experience, Vol. 29, Issue 8 https://doi.org/10.1002/cpe.4041	journal	December 2016
The self-tuning dynP job-scheduler Streit, A. Proceedings 16th International Parallel and Distributed Processing Symposium https://doi.org/10.1109/IPDPS.2002.1015662	conference	January 2002
Portfolio-Based Selection of Robust Dynamic Loop Scheduling Algorithms Using Machine Learning Sukhija, Nitin; Malone, Brandon; Srivastava, Srishti 2014 IEEE International Parallel & Distributed Processing Symposium Workshops https://doi.org/10.1109/IPDPSW.2014.183	conference	May 2014
PSINS: An Open Source Event Tracer and Execution Simulator for MPI Applications Tikir, Mustafa M.; Laurenzano, Michael A.; Carrington, Laura Lecture Notes in Computer Science https://doi.org/10.1007/978-3-642-03869-3_16	book	January 2009
On the validity of flow-level tcp network models for grid and cloud simulations Velho, Pedro; Schnorr, Lucas Mello; Casanova, Henri ACM Transactions on Modeling and Computer Simulation, Vol. 23, Issue 4 https://doi.org/10.1145/2517448	journal	October 2013

Similar Records

Runtime Scheduling Policies for Distributed Graph Algorithms

Conference · Mon Aug 06 00:00:00 EDT 2018 · OSTI ID:1922323

Firoz, Jesun S.; Zalewski, Marcin J.; Lumsdaine, Andrew; +1 more

Runtime Scheduling Policies for Distributed Graph Algorithms

Conference · Wed Jun 06 00:00:00 EDT 2018 · OSTI ID:1922323

Firoz, Jesun S.; Zalewski, Marcin J.; Barnas, Martina; +1 more

Data Locality Enhancement of Dynamic Simulations for Exascale Computing (Final Report)

Technical Report · Fri Nov 29 00:00:00 EST 2019 · OSTI ID:1922323

Shen, Xipeng

Title: On the Feasibility of Simulation-Driven Portfolio Scheduling for Cyberinfrastructure Runtime Systems

Citation Formats

References (24)

Similar Records

Related Subjects