A DAG scheduling scheme on heterogeneous computing systems using double molecular structure-based chemical reaction optimization

https://doi.org/10.1016/j.jpdc.2013.05.005Get rights and content

Highlights

  • Applying CRO to solve DAG scheduling problems in heterogeneous computing systems.

  • Developing DMSCRO by adapting the conventional CRO framework.

  • Designing a new solution encoding method for DAG scheduling.

  • Designing new operations for performing elementary chemical reactions in this work.

  • Conducting experiments to verify the effectiveness and efficiency of the DMSCRO.

Abstract

A new meta-heuristic method, called Chemical Reaction Optimization (CRO), has been proposed very recently. The method encodes solutions as molecules and mimics the interactions of molecules in chemical reactions to search the optimal solutions. The CRO method has demonstrated its capability in solving NP-hard optimization problems. In this paper, the CRO scheme is used to formulate the scheduling of Directed Acyclic Graph (DAG) jobs in heterogeneous computing systems, and a Double Molecular Structure-based Chemical Reaction Optimization (DMSCRO) method is developed. There are two molecular structures in DMSCRO: one is used to encode the execution order of the tasks in a DAG job, and the other to encode the task-to-computing-node mapping. The DMSCRO method also designs four elementary chemical reaction operations and the fitness function suitable for the scenario of DAG scheduling. In this paper, we have also conducted the simulation experiments to verify the effectiveness and efficiency of DMSCRO over a large set of randomly generated graphs and the graphs for real-world problems.

Introduction

A job consisting of a group of tasks with precedence constraints is often modeled as a Directed Acyclic Graph (DAG). When scheduling a DAG job, the main objective is to optimize its makespan, which is defined as the duration between the time when the first task in the DAG starts execution and the time when the last task finishes execution. This problem has been well-studied for many decades. Achieving this objective has been proved to be a NP-complete problem  [9], which means that the time needed to find the optimal solution increases exponentially as the problem size increases. Therefore, two schools of scheduling methods, heuristic scheduling and meta-heuristic scheduling, have been proposed to find the sub-optimal solution with lower time overhead.

Heuristic scheduling algorithms exploit the heuristics to identify a good solution. An important class of heuristic scheduling is list scheduling. List scheduling maintains an ordered list of tasks in a DAG job according to some greedy heuristics. The tasks are selected in the specified order for mapping to the computing nodes which allow the earliest start times. Heuristic scheduling algorithms can find solutions with low time complexity, since the attempted solutions are narrowed down by greedy heuristics to a very small portion of the entire solution space. However, the quality of the solutions obtained by these algorithms is heavily dependent on the effectiveness of the heuristics, and it is not likely for the greedy heuristics to produce consistent results on a wide range of problems, especially when the complexity of the DAG scheduling problem becomes high.

Meta-heuristic scheduling (or Guided-random-search-based) techniques work by guiding the searching for solutions in a solution space. Although meta-heuristic scheduling typically takes longer time, they can achieve good performance consistently for a wide range of scheduling scenarios. Well-known examples of meta-heuristic scheduling techniques include Genetic Algorithms (GA), Particle Swarm Optimization (PSO), Ant Colony Optimization (ACO), Simulated Annealing (SA) and Tabu Search (TS), etc. Very recently, a new meta-heuristic method, called Chemical Reaction Optimization (CRO), has been proposed. The method encodes solutions as molecules and mimics the interactions of molecules in chemical reactions to search the optimal solutions. The CRO method has demonstrated its capability in solving NP-hard optimization problems.

In this paper, we integrate the CRO framework to schedule DAG jobs on heterogeneous computing systems. Scheduling DAG jobs on heterogeneous systems involves making decisions about the execution order of tasks and task-to-computing-node mapping. This paper adapts the conventional CRO framework and proposes a Double Molecular Structure-based CRO (DMSCRO) scheme to formulate the scheduling of DAG jobs. In DMSCRO, one molecular structure is used to encode the execution order of the tasks in a DAG job, while the other molecular structure to encode the task-to-computing-node mapping. DMSCRO also designs the necessary elementary chemical reaction operations and the fitness function suitable for the scenario of DAG scheduling.

According to No-Free-Lunch Theorem in the area of meta-heuristics  [34], all meta-heuristic methods that search for optimal solutions are the same in performance when averaged over all possible objective functions. In theory, as long as an effective meta-heuristic method runs for long enough, it will gradually approach the optimal solution. We have conducted the experiments over a large set of randomly generated graphs, and also the graphs abstracted from two well-known real applications: Gaussian elimination and molecular dynamics application. The experimental results show that the proposed DMSCRO can achieve better performance than the heuristic algorithms, but achieves similar performance as GA in the literature in terms of makespan. We will show in Section  5 that DMSCRO combines the advantages of GA and Simulated Annealing (SA), and therefore may have better performance in terms of searching efficiency. The experimental results presented in Section  6.4 indeed demonstrate that DMSCRO is able to find good solutions faster than GA.

The three major contributions of this work are summarized below:

  • Applying the Chemical Reaction Optimization framework to solve DAG scheduling problems in heterogeneous computing systems.

  • Developing DMSCRO by adapting the conventional CRO framework and designing a new solution encoding method, new operations for performing elementary chemical reactions and a new fitness function suitable for the scheduling scenarios considered in this work.

  • Conducting simulation experiments to verify the effectiveness and efficiency of the proposed DMSCRO. Our experimental results show that (1) DMSCRO is able to achieve the similar makespan as GA, but it finds good solutions faster than GA by 26.5% on average (by 58.9% in the best case), and (2) DMSCRO consistently achieves smaller makespans than two heuristic scheduling algorithms (HEFT_B and HEFT_T) which it has been shown in the literature outperform other heuristic scheduling algorithms. Compared with HEFT_B and HEFT_T, the makespan reduction achieved by DMSCRO is 10% and 12.8% on average, respectively.

The remainder of this paper is organized as follows: In Section  2, the related work about scheduling algorithms on heterogeneous systems is presented. Section  3 presents the background knowledge of CRO. Section  4 describes the system and workload model. In Section  5, the DMSCRO scheme is presented for DAG scheduling, aiming to minimize the makespan on heterogeneous computing systems. Section  6 compares the performance of the proposed scheme with the existing heuristic algorithms. Finally, Section  7 concludes the paper.

Section snippets

Related work

In this section, we discuss the related work on heuristic scheduling, meta-heuristic (or guided-random-search-based) scheduling and job-shop scheduling.

Background of CRO

CRO mimics the process of a chemical reaction where molecules undergo a sequence of reactions between each other or with the environment in a closed container. A molecule has a unique structure of atoms, which represents a solution of the optimization problem. Potential energy (PE) and Kinetic energy (KE) are two key properties attached to a molecule structure. The former corresponds to the fitness value of the solution and the fitness of a solution is judged by the PE energy of the molecule,

MODELS

This section discusses the system, application and task scheduling model assumed in this work. The definition of the notations can be found in Table 1.

Design of DMSCRO

The concepts in DAG scheduling can be mapped to those in CRO. DAG scheduling involves making decisions about (1) scheduling order of tasks (i.e., the order of the tasks in the waiting queue of the scheduler) and (2) resource allocation (i.e., which computing node is used to run the task). The quality of a scheduling solution is determined by makespan. The shorter makespan, the better scheduling solution. In CRO, there are the concepts of molecule, atoms, molecular structure and energy of a

Simulation and results

To illustrate the power of DMSCRO-based DAG scheduling algorithm, we compare this algorithm with the previously proposed heuristics (HEFT_B and HEFT_T)  [29] and also with a well known meta-heuristic algorithm, Genetic Algorithm (GA), presented in  [12]. The makespan performance obtained by GA is used as the baseline performance. The GA used in the simulation experiments is therefore labeled as BGA in the figures.

The reason why we select HEFT_B and HEFT_T as the representatives of heuristic

Conclusions

In this paper, we developed a DMSCRO for DAG scheduling on heterogeneous computing systems. The algorithm incorporates two molecular structures: one evolves to generate priority queueing of subtasks in a DAG and the other to generate task-to-computing-node mappings. Four elementary chemical reaction operations are designed in DMSCRO, and they take into account the precedence relations of the subtasks and guarantee that the newly generated priority queueing complies with those precedence

Acknowledgments

This research was partially funded by the Key Program of National Natural Science Foundation of China (Grant No. 61133005), the National Natural Science Foundation of China (Grant Nos. 61070057, 61173045), Key Projects in the National Science & Technology Pillar Program, the Cultivation Fund of the Key Scientific and Technical Innovation Project (2012BAH09B02), and the Ph.D. Programs Foundation of Ministry of Education of China (20100161110019). The project was supported by the National Science

Yuming Xu received the master’s degree from Hunan University, China, in 2009. He is currently working toward the Ph.D. degree at Hunan University of China. His research interests include modeling and scheduling for distributed computing systems, Parallel algorithms, Grid and Cloud computing.

References (39)

  • A. Amini, T.Y. Wah, M. Saybani, S. Yazdi, A study of density-grid based clustering algorithms on data streams, in: 2011...
  • R. Bajaj et al.

    Improving scheduling of tasks in a heterogeneous environment

    IEEE Transactions on Parallel and Distributed Systems

    (2004)
  • T. Chen, B. Zhang, X. Hao, Y. Dai, Task scheduling in grid based on particle swarm optimization, in: The Fifth...
  • H. Cheng, A high efficient task scheduling algorithm based on heterogeneous multi-core processor, in: 2010 2nd...
  • P. Choudhury et al.

    Hybrid scheduling of dynamic task graphs with selective duplication for multiprocessors under memory and time constraints

    IEEE Transactions on Parallel and Distributed Systems

    (2008)
  • L. He et al.

    Developing resource consolidation frameworks for moldable virtual machines in clouds

    Future Generation Computer Systems

    (2012)
  • H. El-Rewin et al.

    Scheduling parallel program tasks onto arbitrary target machines

    Parallel and Distributed Computing

    (1990)
  • F. Ferrandi et al.

    Ant colony heuristic for mapping and scheduling tasks and communications on heterogeneous embedded systems

    IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems

    (2010)
  • M.R. Garey et al.

    Computers and Intractability: A Guide to the Theory of NP-Completeness

    (1979)
  • H.-W. Ge et al.

    An effective PSO and AIS-based hybrid intelligent algorithm for job-shop scheduling

    IEEE Transactions on Systems, Man, and Cybernetics Part A: Systems and Humans

    (2008)
  • N.B. Ho et al.

    Solving multiple-objective flexible job shop problems by evolution and local search

    IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews

    (2008)
  • E. Hou et al.

    A genetic algorithm for multiprocessor scheduling

    IEEE Transactions on Parallel and Distributed Systems

    (1994)
  • J. Hwang et al.

    Scheduling precedence graphs in systems with interprocessor communication times

    SIAM Journal on Computing

    (1989)
  • M. Iverson, F. Ozguner, G. Follen, Parallelizing existing applications in a distributed heterogeneous environment, in:...
  • M. Kashani, M. Jahanshahi, Using simulated annealing for task scheduling in distributed systems, in: International...
  • S. Kim, J. Browne, A general approach to mapping of parallel computation upon multiprocessor architectures, in:...
  • Y.-K. Kwok et al.

    Static scheduling algorithms for allocating directed task graphs to multiprocessors

    ACM Computing Surveys (CSUR)

    (1999)
  • A. Lam et al.

    Chemical-reaction-inspired metaheuristic for optimization

    IEEE Transactions on Evolutionary Computation

    (2010)
  • H. Li, L. Wang, J. Liu, Task scheduling of computational grid based on particle swarm algorithm, in: 2010 Third...
  • Cited by (102)

    • Development of a new task scheduling in cloud computing

      2023, International Journal of System Assurance Engineering and Management
    View all citing articles on Scopus

    Yuming Xu received the master’s degree from Hunan University, China, in 2009. He is currently working toward the Ph.D. degree at Hunan University of China. His research interests include modeling and scheduling for distributed computing systems, Parallel algorithms, Grid and Cloud computing.

    Kenli Li received the Ph.D. in computer science from Huazhong University of Science and Technology, China, in 2003, and the M.Sc. in mathematics from Central South University, China, in 2000. He was a visiting scholar at University of Illinois at Champaign and Urbana from 2004 to 2005. Now He is a professor of Computer science and Technology at Hunan University, associate director of National Supercomputing Center in Changsha, a senior member of CCF. His major research includes parallel computing, Grid and Cloud computing, and DNA computer. He has published more than 70 papers in international conferences and journals, such as IEEE TC, JPDC, PC, ICPP, and CCGrid.

    Ligang He received the Bachelor’s and Master’s degrees from the Huazhong University of Science and Technology, Wuhan, China, and received the Ph.D. degree in Computer Science from the University of Warwick, UK. He was also a Post-doctoral researcher at the University of Cambridge, UK. In 2006, he joined the Department of Computer Science at the University of Warwick as an Assistant Professor, and then became an Associate Professor. His areas of interest are parallel and distributed computing, Grid computing and Cloud computing. He has published more than 50 papers in international conferences and journals, such as IEEE TPDS, IPDPS, Cluster, CCGrid, and MASCOTS. He also served as a member of the program committee for many international conferences and was the reviewer for a number of international journals, including IEEE TPDS, IEEE TC, IEEE TASE, etc. He is a member of the IEEE.

    Tung Khac Truong received B.S. in Mathematic from Hue College, Hue University, Vietnam, in 2001. He received M.S. in computer science from Hue University, Vietnam, in 2007. He is currently working toward the Ph.D. degree in computer science at school of Computer and Communication, Hunan University. His research interests are soft computing and parallel computing.

    View full text