An experimental comparison of different real-time schedulers on multicore systems

Abstract

In this work, an experimental comparison among the Rate Monotonic (RM) and Earliest Deadline First (EDF) multiprocessor real-time schedulers is performed, with a focus on soft real-time systems. We generated random workloads of synthetic periodic task sets and executed them on a big multi-core machine, using Linux as Operating System, gathering an extensive amount of data related to their exhibited performance under various real-time scheduling strategies. The comparison involves the fixed-priority scheduler for multiprocessors as available in the Linux kernel (with priorities set so as to achieve RM), and on our own implementation of EDF, both configured in global, partitioned and clustered mode. The impact of the various scheduling strategies on the performance of the applications, as well as the generated scheduling overheads, are compared presenting an extensive set of experimental results. These provide a comprehensive view of the performance achievable by the different schedulers under various workload conditions.

Introduction

Multi-processor and multi-core computing platforms are nowadays largely used in the vast majority of application domains, ranging from embedded systems, to personal computing, to server-side computing including GRIDs and Cloud Computing, and finally high-performance computing. In embedded systems, small multi-core platforms are considered a viable and cost-effective solution, especially for their lower power requirements as compared to a traditional single processor system with equivalent computing capabilities. The increased level of parallelism in these systems may be conveniently exploited to run multiple real-time applications, like found in industrial control, aerospace or military systems; or to support soft real-time Quality of Service (QoS) oriented applications, like found in multimedia, gaming or virtual reality systems.

Servers and data centres are shifting towards (massively) parallel architectures with enhanced maintainability, often accompanied by a decrease in the clock frequency driven by the increasing need for “green computing” (von Weizsaecker et al., 2009).² Cloud Computing applications promise to move most of the increasing personal computing needs of users into the “cloud”. This is leading to an unprecedented need for supporting a large number of interactive and real-time applications, often involving on-the-fly media streaming, processing and transformations with demanding performance and latency requirements. These applications usually exhibit nearly periodic workload patterns which often cannot saturate the available computing power of a single (powerful) CPU. Therefore, there is a strong industrial interest in executing an ever-increasing number of applications of this type on the same system, node, physical CPU and even core, whenever possible, in order to minimise the number of needed nodes (and reduce both power consumption and costs).

In this context, a key role is played by real-time CPU scheduling algorithms for multi-processor systems, due to their potential impact on the performance experienced by the scheduled applications. These can be roughly categorised into global schedulers and partitioned schedulers. In global scheduling, the ready tasks with the highest priorities execute on the available processors at any time. This implies the need for dynamically migrating tasks among processors. On the other hand, in partitioned scheduling each task is statically allocated on one processor, according to a specific allocation algorithm, and tasks cannot migrate. Clustered schedulers reside somewhat in the middle, where the available processors are partitioned into clusters to which tasks are statically assigned, but in each cluster tasks are globally scheduled. In a multi-core system, the use of partitioned or clustered scheduling policies brings the additional problem of how to partition the tasks among cores or clusters of cores.

An orthogonal way to categorise schedulers is according to the way task priorities are assigned. If tasks’ priorities never change during the task lifetime, we have a fixed priority scheduler, otherwise we have a dynamic priority scheduler. In this paper, we focus on the two most popular schedulers: Rate Monotonic (RM) priority assignment for fixed priority schedulers; and Earliest Deadline First (EDF) for dynamic priority schedulers.

The designer of a real-time system often needs to compare different available real-time scheduling strategies, in terms of their impact on the performance of the hosted applications.

Many recent papers compare different scheduling strategies (global/partitioned, and fixed/dynamic priority) from a theoretical point of view (see Section 2 for an analysis of related work). In these works different schedulers are compared with respect to their achievable overall utilisation, under the constraint of maintaining the task-set schedulable, assuming worst-case conditions for the tasks execution, i.e., the analysis is based on the Worst-Case Execution Times (WCET). Although appropriate for hard real-time systems, in soft real-time ones such approaches are at risk of neglecting (or merely reminding to WCET analysis for) many practical issues, such as the overhead of the scheduler, the increased execution times as due to migrations and increased cache misses, and the presence of variability of memory access times in Non-Uniform Memory-Access (NUMA) machines. These issues may have a great influence on the actual performance, especially as the number of cores increases.

For instance, partitioned schedulers typically present less overhead. However, in an open system where tasks may dynamically enter and leave the system, a static allocation strategy may lead to underutilised systems. On the other hand, global scheduling is more flexible as it automatically balances the load across all processors. In addition, global dynamic priority schedulers (like EDF) are known to guarantee bounded tardiness as long as the total load does not exceed the system capacity (Valente and Lipari, 2005, Devi and Anderson, 2009, Devi and Anderson, 2008). However, global schedulers typically present a higher overhead; they cause migrations, which in turn may lead to a non-negligible increase in the tasks execution times.

Therefore, it is important to quantify such overheads in order to complement the theoretical properties of a scheduler with its practical performance figures. In this way, the designer can take a more informed decision on which scheduler to select for various application workload types.

In this paper the performance of partitioned, clustered and global variants of Rate Monotonic (RM) and Earliest Deadline First (EDF) scheduling algorithms in the Linux OS are compared. The experimental comparison is conducted on the Linux OS due to its wide applicability (with various kernel-level patches) in the domain of real-time systems.

We compare our own implementation of Global EDF (G-EDF) in the Linux kernel with respect to the fixed priority Linux scheduler (configured so as to realise RM). The goal is not to demonstrate the effectiveness of our scheduler, but rather to make a thorough performance comparison, and establish which scheduler performs better in different contexts. In order to precisely control the experiments, our methodology consists in generating sets of synthetic real-time tasks with various characteristics in terms of execution time and memory requirements and usage. The task set is then executed on a multi-core platform and the tasks’ performance is measured. The focus is on the metrics typically of interest for developers and other people who investigate on performance issues, and not purely on schedulability analysis. Indeed, we consider the laxity (tasks should not complete too close to their deadlines), the number of migrations and context switches (they may potentially affect negatively the performance), and the number and type of cache misses (they have a direct impact on the application execution times and performance). Since we focus on the comparison among different CPU scheduling policies, in this paper we only consider independent tasks. The hardware platform is a AMD^® Opteron™ 6168 with 48 cores (4 sockets with 12 cores for each processor).

Our implementation of G-EDF has been made available as open-source code. This, together with the details about the configuration of the experiments, allows to reproduce and verify all the results that have been included in the paper (see the Section 6). Also, this allows other researchers to perform independent investigations on partitioned, clustered and global EDF scheduling on Linux, as well as to develop new schedulers and concretely compare their performance with these policies. Last, but not least, this gives to any developer the possibility to try these policies for their real-time applications.

The remainder of this paper is organised as follows: in Section 2, the related work is briefly recalled. In Section 3, the background concepts needed to understand the remainder of the paper are introduced, and in Section 4 the modifications performed on the Linux kernel, to make it support global EDF scheduling, are sketched out. Sections 5 and 6 describe the methodology and report the results of the experimental evaluation phase, respectively. Finally, in Section 7 conclusions are drawn and possible directions for future work are envisioned in Section 8.