A DAG scheduling scheme on heterogeneous computing systems using double molecular structure-based chemical reaction optimization
Introduction
A job consisting of a group of tasks with precedence constraints is often modeled as a Directed Acyclic Graph (DAG). When scheduling a DAG job, the main objective is to optimize its makespan, which is defined as the duration between the time when the first task in the DAG starts execution and the time when the last task finishes execution. This problem has been well-studied for many decades. Achieving this objective has been proved to be a NP-complete problem [9], which means that the time needed to find the optimal solution increases exponentially as the problem size increases. Therefore, two schools of scheduling methods, heuristic scheduling and meta-heuristic scheduling, have been proposed to find the sub-optimal solution with lower time overhead.
Heuristic scheduling algorithms exploit the heuristics to identify a good solution. An important class of heuristic scheduling is list scheduling. List scheduling maintains an ordered list of tasks in a DAG job according to some greedy heuristics. The tasks are selected in the specified order for mapping to the computing nodes which allow the earliest start times. Heuristic scheduling algorithms can find solutions with low time complexity, since the attempted solutions are narrowed down by greedy heuristics to a very small portion of the entire solution space. However, the quality of the solutions obtained by these algorithms is heavily dependent on the effectiveness of the heuristics, and it is not likely for the greedy heuristics to produce consistent results on a wide range of problems, especially when the complexity of the DAG scheduling problem becomes high.
Meta-heuristic scheduling (or Guided-random-search-based) techniques work by guiding the searching for solutions in a solution space. Although meta-heuristic scheduling typically takes longer time, they can achieve good performance consistently for a wide range of scheduling scenarios. Well-known examples of meta-heuristic scheduling techniques include Genetic Algorithms (GA), Particle Swarm Optimization (PSO), Ant Colony Optimization (ACO), Simulated Annealing (SA) and Tabu Search (TS), etc. Very recently, a new meta-heuristic method, called Chemical Reaction Optimization (CRO), has been proposed. The method encodes solutions as molecules and mimics the interactions of molecules in chemical reactions to search the optimal solutions. The CRO method has demonstrated its capability in solving NP-hard optimization problems.
In this paper, we integrate the CRO framework to schedule DAG jobs on heterogeneous computing systems. Scheduling DAG jobs on heterogeneous systems involves making decisions about the execution order of tasks and task-to-computing-node mapping. This paper adapts the conventional CRO framework and proposes a Double Molecular Structure-based CRO (DMSCRO) scheme to formulate the scheduling of DAG jobs. In DMSCRO, one molecular structure is used to encode the execution order of the tasks in a DAG job, while the other molecular structure to encode the task-to-computing-node mapping. DMSCRO also designs the necessary elementary chemical reaction operations and the fitness function suitable for the scenario of DAG scheduling.
According to No-Free-Lunch Theorem in the area of meta-heuristics [34], all meta-heuristic methods that search for optimal solutions are the same in performance when averaged over all possible objective functions. In theory, as long as an effective meta-heuristic method runs for long enough, it will gradually approach the optimal solution. We have conducted the experiments over a large set of randomly generated graphs, and also the graphs abstracted from two well-known real applications: Gaussian elimination and molecular dynamics application. The experimental results show that the proposed DMSCRO can achieve better performance than the heuristic algorithms, but achieves similar performance as GA in the literature in terms of makespan. We will show in Section 5 that DMSCRO combines the advantages of GA and Simulated Annealing (SA), and therefore may have better performance in terms of searching efficiency. The experimental results presented in Section 6.4 indeed demonstrate that DMSCRO is able to find good solutions faster than GA.
The three major contributions of this work are summarized below:
- •
Applying the Chemical Reaction Optimization framework to solve DAG scheduling problems in heterogeneous computing systems.
- •
Developing DMSCRO by adapting the conventional CRO framework and designing a new solution encoding method, new operations for performing elementary chemical reactions and a new fitness function suitable for the scheduling scenarios considered in this work.
- •
Conducting simulation experiments to verify the effectiveness and efficiency of the proposed DMSCRO. Our experimental results show that (1) DMSCRO is able to achieve the similar makespan as GA, but it finds good solutions faster than GA by 26.5% on average (by 58.9% in the best case), and (2) DMSCRO consistently achieves smaller makespans than two heuristic scheduling algorithms (HEFT_B and HEFT_T) which it has been shown in the literature outperform other heuristic scheduling algorithms. Compared with HEFT_B and HEFT_T, the makespan reduction achieved by DMSCRO is 10% and 12.8% on average, respectively.
The remainder of this paper is organized as follows: In Section 2, the related work about scheduling algorithms on heterogeneous systems is presented. Section 3 presents the background knowledge of CRO. Section 4 describes the system and workload model. In Section 5, the DMSCRO scheme is presented for DAG scheduling, aiming to minimize the makespan on heterogeneous computing systems. Section 6 compares the performance of the proposed scheme with the existing heuristic algorithms. Finally, Section 7 concludes the paper.
Section snippets
Related work
In this section, we discuss the related work on heuristic scheduling, meta-heuristic (or guided-random-search-based) scheduling and job-shop scheduling.
Background of CRO
CRO mimics the process of a chemical reaction where molecules undergo a sequence of reactions between each other or with the environment in a closed container. A molecule has a unique structure of atoms, which represents a solution of the optimization problem. Potential energy (PE) and Kinetic energy (KE) are two key properties attached to a molecule structure. The former corresponds to the fitness value of the solution and the fitness of a solution is judged by the PE energy of the molecule,
MODELS
This section discusses the system, application and task scheduling model assumed in this work. The definition of the notations can be found in Table 1.
Design of DMSCRO
The concepts in DAG scheduling can be mapped to those in CRO. DAG scheduling involves making decisions about (1) scheduling order of tasks (i.e., the order of the tasks in the waiting queue of the scheduler) and (2) resource allocation (i.e., which computing node is used to run the task). The quality of a scheduling solution is determined by makespan. The shorter makespan, the better scheduling solution. In CRO, there are the concepts of molecule, atoms, molecular structure and energy of a
Simulation and results
To illustrate the power of DMSCRO-based DAG scheduling algorithm, we compare this algorithm with the previously proposed heuristics (HEFT_B and HEFT_T) [29] and also with a well known meta-heuristic algorithm, Genetic Algorithm (GA), presented in [12]. The makespan performance obtained by GA is used as the baseline performance. The GA used in the simulation experiments is therefore labeled as BGA in the figures.
The reason why we select HEFT_B and HEFT_T as the representatives of heuristic
Conclusions
In this paper, we developed a DMSCRO for DAG scheduling on heterogeneous computing systems. The algorithm incorporates two molecular structures: one evolves to generate priority queueing of subtasks in a DAG and the other to generate task-to-computing-node mappings. Four elementary chemical reaction operations are designed in DMSCRO, and they take into account the precedence relations of the subtasks and guarantee that the newly generated priority queueing complies with those precedence
Acknowledgments
This research was partially funded by the Key Program of National Natural Science Foundation of China (Grant No. 61133005), the National Natural Science Foundation of China (Grant Nos. 61070057, 61173045), Key Projects in the National Science & Technology Pillar Program, the Cultivation Fund of the Key Scientific and Technical Innovation Project (2012BAH09B02), and the Ph.D. Programs Foundation of Ministry of Education of China (20100161110019). The project was supported by the National Science
Yuming Xu received the master’s degree from Hunan University, China, in 2009. He is currently working toward the Ph.D. degree at Hunan University of China. His research interests include modeling and scheduling for distributed computing systems, Parallel algorithms, Grid and Cloud computing.
References (39)
- A. Amini, T.Y. Wah, M. Saybani, S. Yazdi, A study of density-grid based clustering algorithms on data streams, in: 2011...
- et al.
Improving scheduling of tasks in a heterogeneous environment
IEEE Transactions on Parallel and Distributed Systems
(2004) - T. Chen, B. Zhang, X. Hao, Y. Dai, Task scheduling in grid based on particle swarm optimization, in: The Fifth...
- H. Cheng, A high efficient task scheduling algorithm based on heterogeneous multi-core processor, in: 2010 2nd...
- et al.
Hybrid scheduling of dynamic task graphs with selective duplication for multiprocessors under memory and time constraints
IEEE Transactions on Parallel and Distributed Systems
(2008) - et al.
Developing resource consolidation frameworks for moldable virtual machines in clouds
Future Generation Computer Systems
(2012) - et al.
Scheduling parallel program tasks onto arbitrary target machines
Parallel and Distributed Computing
(1990) - et al.
Ant colony heuristic for mapping and scheduling tasks and communications on heterogeneous embedded systems
IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
(2010) - et al.
Computers and Intractability: A Guide to the Theory of NP-Completeness
(1979) - et al.
An effective PSO and AIS-based hybrid intelligent algorithm for job-shop scheduling
IEEE Transactions on Systems, Man, and Cybernetics Part A: Systems and Humans
(2008)
Solving multiple-objective flexible job shop problems by evolution and local search
IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews
A genetic algorithm for multiprocessor scheduling
IEEE Transactions on Parallel and Distributed Systems
Scheduling precedence graphs in systems with interprocessor communication times
SIAM Journal on Computing
Static scheduling algorithms for allocating directed task graphs to multiprocessors
ACM Computing Surveys (CSUR)
Chemical-reaction-inspired metaheuristic for optimization
IEEE Transactions on Evolutionary Computation
Cited by (102)
A scheduling algorithm for heterogeneous computing systems by edge cover queue
2023, Knowledge-Based SystemsA novel algorithm for priority-based task scheduling on a multiprocessor heterogeneous system
2022, Microprocessors and MicrosystemsDevelopment of a new task scheduling in cloud computing
2023, International Journal of System Assurance Engineering and ManagementMPEFT: a makespan minimizing heuristic scheduling algorithm for workflows in heterogeneous computing systems
2023, CCF Transactions on High Performance Computing
Yuming Xu received the master’s degree from Hunan University, China, in 2009. He is currently working toward the Ph.D. degree at Hunan University of China. His research interests include modeling and scheduling for distributed computing systems, Parallel algorithms, Grid and Cloud computing.
Kenli Li received the Ph.D. in computer science from Huazhong University of Science and Technology, China, in 2003, and the M.Sc. in mathematics from Central South University, China, in 2000. He was a visiting scholar at University of Illinois at Champaign and Urbana from 2004 to 2005. Now He is a professor of Computer science and Technology at Hunan University, associate director of National Supercomputing Center in Changsha, a senior member of CCF. His major research includes parallel computing, Grid and Cloud computing, and DNA computer. He has published more than 70 papers in international conferences and journals, such as IEEE TC, JPDC, PC, ICPP, and CCGrid.
Ligang He received the Bachelor’s and Master’s degrees from the Huazhong University of Science and Technology, Wuhan, China, and received the Ph.D. degree in Computer Science from the University of Warwick, UK. He was also a Post-doctoral researcher at the University of Cambridge, UK. In 2006, he joined the Department of Computer Science at the University of Warwick as an Assistant Professor, and then became an Associate Professor. His areas of interest are parallel and distributed computing, Grid computing and Cloud computing. He has published more than 50 papers in international conferences and journals, such as IEEE TPDS, IPDPS, Cluster, CCGrid, and MASCOTS. He also served as a member of the program committee for many international conferences and was the reviewer for a number of international journals, including IEEE TPDS, IEEE TC, IEEE TASE, etc. He is a member of the IEEE.
Tung Khac Truong received B.S. in Mathematic from Hue College, Hue University, Vietnam, in 2001. He received M.S. in computer science from Hue University, Vietnam, in 2007. He is currently working toward the Ph.D. degree in computer science at school of Computer and Communication, Hunan University. His research interests are soft computing and parallel computing.