Real-time scheduling of divisible loads in cluster computing environments

https://doi.org/10.1016/j.jpdc.2009.11.009Get rights and content

Abstract

Cluster computing has become an important paradigm for solving large-scale problems. To enhance the quality of service (QoS) and provide performance guarantees in a cluster computing environment, various real-time scheduling algorithms and workload models have been investigated. Computational loads that can be arbitrarily divided into independent tasks represent many real-world applications. However, the problem of providing performance guarantees to divisible load applications has only recently been studied systematically. In this work, three important and necessary design decisions, (1) workload partitioning, (2) node assignment, and (3) task execution order, are identified for real-time divisible load scheduling. A scheduling framework that can configure different policies for each of the three design decisions is proposed and used to generate various algorithms. This paper systematically studies these algorithms and identifies scenarios where the choices of design parameters have significant effects.

Introduction

Cluster computing has become an important paradigm for solving large-scale problems. However, as the size of a cluster increases, so does the complexity of resource management and maintenance. Therefore, automated performance control and resource management are expected to play critical roles in sustaining the evolution of cluster computing. The current cluster scheduling practice is similar in sophistication to early supercomputer batch scheduling algorithms, and no consideration is given to desired quality-of-service (QoS) attributes. To fully take advantage of the power of computational clusters, new scheduling theory that provides high performance, QoS assurance, and streamlined management of the cluster resources needs to be developed.

Real-time scheduling theory has been very successful in providing deterministic QoS in desktop systems [10], [11], [25]. A significant challenge in developing real-time scheduling theory for cluster computing, however, is to support various types of cluster applications. Broadly speaking, computational loads submitted to a cluster can be structured in three primary ways: indivisible, modularly divisible, and arbitrarily divisible. An indivisible load is essentially a sequential job which cannot be further divided, and thus must be assigned to a single processor. Modularly divisible loads can be divided a priori into a certain number of subtasks and are often described by a task (or processing) graph. Arbitrarily divisible loads, also called embarrassingly parallel workloads, can be partitioned into an arbitrarily large number of independent load fractions. Examples of arbitrarily divisible loads can be easily found in high-energy and particle physics as well as biometrics. For example, the CMS (Compact Muon Solenoid) [15] and ATLAS (A Toroidal LHC Apparatus) [6] projects, which are associated with the LHC (Large Hadron Collider) at CERN (European Laboratory for Particle Physics), execute cluster-based applications with arbitrarily divisible loads. Usually all elements in such computational loads demand an identical type of processing, and relative to the huge total computation, the processing on each individual element is infinitesimally small. The problem of providing QoS or real-time guarantees for sequential and modularly divisible jobs in distributed systems has been studied extensively. However, despite the increasing importance of arbitrarily divisible applications [33], to the best of our knowledge, the real-time scheduling of arbitrarily divisible loads has not been systematically investigated.

Scheduling of arbitrarily divisible loads represents a problem of great significance for cluster-based research computing facilities such as the US CMS Tier-2 sites [37]. For example, one of the management goals at the University of Nebraska–Lincoln (UNL) Research Computing Facility (RCF) (a CMS Tier-2 site) is to provide a multi-tiered QoS scheduling framework in which applications “pay” according to the response time requested for each job [37]. Existing real-time cluster scheduling algorithms assume the existence of a task graph for all applications, which are not appropriate for arbitrarily divisible loads. To better manage these high-end clusters and control their performance, we need new real-time scheduling algorithms for arbitrarily divisible applications.

Three contributions are made in this paper. First, three important and necessary design decisions for real-time divisible load scheduling are identified. Second, a scheduling framework that integrates the three components is proposed. By configuring different policies for each of these three decisions, various real-time scheduling algorithms can be generated. Third, we systematically investigate the effects of these design decisions and system parameters on a set of real-time scheduling algorithms. Analysis and experimental results demonstrate a need for adaptive scheduling approaches.

The remainder of this paper is organized as follows. Section 2 describes both task and system models. In Section 3, the real-time scheduling algorithms investigated in this paper are discussed. Detailed analysis of design decisions are given in Sections 4 Analysis of task partitioning methods, 5 Analysis of node assignment policies. We evaluate the performance of algorithms in Section 6 and present the related work in Section 7. Section 8 concludes the paper.

Section snippets

Task and system models

In this section we describe our task and system models briefly, and state assumptions related to these models.

Task model. We assume a real-time aperiodic task model in which each aperiodic task Ti consists of a single invocation specified by the tuple (Ai,σi,Di), where Ai is the task arrival time, σi is the total data size of the task, and Di is the task relative deadline. The task absolute deadline is given by Ai+Di. Section 4 presents, in detail, how the task execution time is dynamically

Algorithms

This section presents real-time scheduling algorithms for divisible loads. To develop the algorithms, we need to make three important decisions. The first is to adopt a scheduling policy to determine the order of execution for tasks (Section 3.1). The second decision is to choose a strategy to partition the task (Section 3.2); that is, to partition the task data among a given number of computing resources. The last decision is to determine the number n of processing nodes to assign to each task

Analysis of task partitioning methods

Section 3.2 has introduced the two partitioning methods that we will investigate in this paper. In this section, we analyze these methods in detail. Since different partitioning methods lead to different task executions, we derive the task execution time and nmin for each of these methods. These analysis provide essential ingredients for the real-time scheduling algorithms (Fig. 1).

The following notations, partially adopted from [39], are used in the analysis.

  • T=(A,σ,D): A divisible task, where

Analysis of node assignment policies

While scheduling, the number of nodes assigned to a real-time divisible task could be between the minimum number nmin of nodes the task needs to meet its deadline and all available N nodes. The two plots in Fig. 5 show the relationship between the task execution time E (Eq. (16)) and n, the number of nodes assigned, when the setup costs are different. As demonstrated in Fig. 5(a), when the setup costs are small, assigning a greater number of nodes to a task will always reduce its execution

Performance evaluation

In previous sections, we have proposed and analyzed various real-time cluster-based scheduling algorithms for divisible loads. In this section, their performance relative to each other and to changes of configuration parameters are experimentally evaluated.

We have developed a discrete simulator, called DLSim, to simulate real-time divisible load scheduling in clusters. This simulator, implemented in Java, is a component-based tool, where the main components include a workload generator, a

Related work

The development of high-performance computing environments has gained considerable momentum. By linking a large number of computers together, a cluster provides a cost-effective facility for solving complex problems. A resource management system (RMS), which provides real-time guarantees or QoS, is central to cluster performance.

Utility-driven cluster computing [40], [35], [22], [28] has been well studied to improve the value of utility delivered to users. Proposed cluster RMSs [12], [3] have

Conclusion

In this paper, we address the problem of providing deterministic QoS to arbitrarily divisible applications executing in cluster environments. For real-time scheduling of divisible loads, three important design decisions need to be made: (1) workload partitioning, (2) node assignment, and (3) task execution order. By systematically studying a set of real-time divisible load scheduling algorithms, we have made many interesting discoveries. In particular, we found that, for real-time cluster-based

Xuan Lin is a Ph.D. student in the Department of Computer Science and Engineering, University of Nebraska–Lincoln. His research interests include grid computing, real-time scheduling and divisible load theory. Xuan Lin received his M.S. and B.S. degrees in Computer Science from Southeast University, Nanjing, China, in 2001 and 1998, respectively.

References (41)

  • D. Babbar, P. Krueger, On-line hard real-time scheduling of parallel tasks on partitionable multiprocessors, in:...
  • L. Barsanti, A.C. Sodan, Adaptive job scheduling strategies via predictive job resource allocation, in: Workshop on Job...
  • V. Bharadwaj et al.

    Scheduling Divisible Loads in Parallel and Distributed Systems

    (1996)
  • S.A. Brandt, S. Banachowski, C. Lin, T. Bisson, Dynamic integrated scheduling of hard real-time, soft real-time and...
  • H. Cheng, S. Goddard, Vre-net: A qos-supported network subsystem for multimedia applications, in: Proceedings of the...
  • B.N. Chun, D.E. Culler, Market-based proportional resource sharing for clusters, EECS Department, University of...
  • G. Chun, H. Dail, H. Casanova, A. Snavely, Benchmark probes for grid assessment, in: Proceedings of the 18th...
  • Compact Muon Solenoid (CMS) Experiment for the Large Hadron Collider at CERN (European Lab for Particle Physics), Cms...
  • M.L. Dertouzos et al.

    Multiprocessor online scheduling of hard-real-time tasks

    IEEE Transactions on Software Engineering

    (1989)
  • Maciej Drozdowski, Pawel Wolniewicz, Experiments with scheduling divisible tasks in clusters of workstations, in:...
  • Cited by (18)

    • Optimum scheduling in fog computing using the Divisible Load Theory (DLT) with linear and nonlinear loads

      2023, Computer Networks
      Citation Excerpt :

      Considering and calculating the delays and latencies of the communication links among the processors is the key point of this type of schedule [6]. Although considerable DLT-based scheduling has been done in the cloud [2,3], cluster [7], and edge environments [8], and significant results have been achieved, regarding the motivation of this research, we should answer the question of why we are doing this research in the first place. Based on the literature of the research, the scheduling algorithm that was previously proposed for the fog computing environment does not address important properties such as data-intensive and divisibility.

    • Flexible fault tolerance in cloud through replicated cooperative resource group

      2019, Computer Communications
      Citation Excerpt :

      These physical hosts further spawn a number virtual machines (VMs) which are the basic computing instances of deployment/un-deployment/migration as per the demand, and hence justify the elasticity of cloud [5]. In spite of numerous attractive features, cloud is prone to many types of faults due to its dynamicity and growing complexity [6–8]. For example, hardware/software faults, network faults, power faults, and reboot faults are the commonly occurring faults in cloud [4].

    • Fault tolerance in cloud computing environment: A systematic survey

      2018, Computers in Industry
      Citation Excerpt :

      Though, cloud has gathered much attention over the time, but it is still considered adolescent in terms of fault handling capability [6]. The cloud computing architecture is dynamic and growing in complexity [7–9]. Its deployment uses millions of commodity components rather than conventional ones [10].

    • Requirement-aware strategies for scheduling real-time divisible loads on clusters

      2013, Journal of Parallel and Distributed Computing
      Citation Excerpt :

      Several recent algorithms have also been proposed for scheduling real-time divisible loads in cluster environments. In [14] Lin et al. proposed a real-time divisible load scheduling algorithm for cluster environments. Their algorithm employs the DLT rule to compute the minimum number of processors required to meet an application’s deadline.

    • Fault Tolerance of Cloud Infrastructure with Machine Learning

      2023, Cybernetics and Information Technologies
    View all citing articles on Scopus

    Xuan Lin is a Ph.D. student in the Department of Computer Science and Engineering, University of Nebraska–Lincoln. His research interests include grid computing, real-time scheduling and divisible load theory. Xuan Lin received his M.S. and B.S. degrees in Computer Science from Southeast University, Nanjing, China, in 2001 and 1998, respectively.

    Anwar Mamat is a Ph.D. student in the Department of Computer Science and Engineering, University of Nebraska–Lincoln. He received his B.E. and M.S. degrees from Chang’an University, Xi’an, China, in 1999 and 2002, respectively. His research interests are grid and cluster computing, high-performance computing, and real-time scheduling.

    Ying Lu received her B.S. degree in Computer Science in 1996 from the Southwest Jiaotong University, Chengdu, China, and her M.CS. and Ph.D. degrees in Computer Science in 2001 and 2005, respectively, from the University of Virginia. Since August 2005, she has been at the University of Nebraska–Lincoln, where she is an Assistant Professor in the Department of Computer Science and Engineering. Her research interests include real-time systems, autonomic computing, cluster and grid computing, and web systems. She has done significant work on the feedback control of computing systems.

    Jitender Deogun received his B.S. (1966) with Honors from Punjab University, Chandigarh, India, and his M.S. (1974), and Ph.D. (1979) degrees from University of Illinois, Urbana, IL. He joined the University of Nebraska in 1981 as an Assistant Professor of Computer Science. He was promoted to Associate Professor in 1984 and Full Professor in 1988. His research interests include high-performance optical switch architectures, routing and wavelength assignment, real-time scheduling, network coding, and structural and algorithmic graph theory.

    Steve Goddard is a Professor and Chair in the Department of Computer Science and Engineering at the University of Nebraska–Lincoln. He received his B.A. degree in Computer Science and Mathematics from the University of Minnesota (1985). He received his M.S. and Ph.D. degrees in Computer Science from the University of North Carolina at Chapel Hill (1995 and 1998, respectively). His research interests are embedded, real-time and distributed systems with an emphasis in real-time, rate-based scheduling theory.

    This work is funded by NSF grant No. 0720810.

    View full text