EM-KDE: A locality-aware job scheduling policy with distributed semantic caches

Highlights

• We propose a locality-aware scheduling policy for distributed query processing.

• Load balance and data reuse are equally important for query processing throughput.

• Distributed semantic caching needs a scheduler that balances load and data reuse.

Abstract

In modern query processing systems, the caching facilities are distributed and scale with the number of servers. To maximize the overall system throughput, the distributed system should balance the query loads among servers and also leverage cached results. In particular, leveraging distributed cached data is becoming more important as many systems are being built by connecting many small heterogeneous machines rather than relying on a few high-performance workstations. Although many query scheduling policies exist such as round-robin and load-monitoring, they are not sophisticated enough to both balance the load and leverage cached results. In this paper, we propose distributed query scheduling policies that take into account the dynamic contents of distributed caching infrastructure and employ statistical prediction methods into query scheduling policy.

We employ the kernel density estimation derived from recent queries and the well-known exponential moving average (EMA) in order to predict the query distribution in a multi-dimensional problem space that dynamically changes. Based on the estimated query distribution, the front-end scheduler assigns incoming queries so that query workloads are balanced and cached results are reused. Our experiments show that the proposed query scheduling policy outperforms existing policies in terms of both load balancing and cache hit ratio.

Introduction

Load-balancing has been extensively investigated in various fields in the past including multiprocessor systems, computer networks, and distributed systems. For load balance, various scheduling algorithms have been introduced; one of the simplest algorithms is the round robin scheduling, and more intelligent load balancing algorithms were proposed, taking additional performance factors into account, such as the current system load, heterogeneous computational power, and network connection of the servers.

In many computational domains, including scientific and business applications, the application dataset has been growing in its size. Moreover, recent computing trend is to analyze massive volumes of data and identify certain patterns. Hence many modern applications spend a large amount of execution time on I/O and manipulation of the data. The fundamental challenge for improving performance of such data-intensive applications is managing massive amounts of data, and reducing data movement and I/O.

In order to reduce the I/O on the large datasets, distributed data analysis frameworks place huge demands on cluster-wide memory capabilities, but the size of the memory in a cluster is not often big enough to hold all the datasets, making in-memory computing impossible. However, the caching facilities scale with the number of distributed servers, and leveraging the large distributed caches plays an important role in improving the overall system throughput as many large scale systems are being built by connecting small machines.

In distributed environments, orchestrating a large number of distributed caches for high cache-hit ratio is difficult. The traditional query scheduling policies such as load-monitoring  [1] are not sophisticated enough to consider cached results in distributed servers; they only consider load balance. Without high cache-hit ratio, the distributed caches will be underutilized, leading to slow query responses. On the contrary, scheduling policies that are solely based on data reuse will fail to balance the system load. If a certain server has very hot cached items, that single server will be flooded with a majority of queries while the other servers are all idle. In order to maximize the system throughput by achieving load balancing as well as exploiting cached query results, it is required to employ query scheduling policies that are more intelligent than the traditional round-robin and load-monitoring scheduling policies.

In this paper, we propose novel distributed query scheduling policies for multi-dimensional scientific data-analysis applications. The proposed distributed query scheduling policies make query scheduling decisions by interpreting the queries as multi-dimensional points, and cluster them so that similar queries cluster together for high cache-hit ratio. The proposed scheduling policies also balance the load among the servers by leveling the cluster sizes in the caches. Our clustering algorithms differ from the well-known clustering algorithms such as $k$-means or BIRCH  [29] in that the complexity of the proposed query scheduling algorithms is very light weight and independent of the number of cached data items since the query scheduling decisions should be made dynamically at run time for the incoming stream queries. Moreover, the goal of the well known clustering methods is to minimize the distance of each object to the belonged cluster, while the distributed query scheduling policies try to balance the number of assigned objects in each cluster while increasing the data locality.

To evaluate the performance of the proposed scheduling policies, we implemented the scheduling policies on top of a component-based distributed query processing framework. Scientific data analysis application developers can implement the interface of user-defined operators to process scientific queries on top of the framework. We conducted extensive experimental studies using the framework and show the proposed query scheduling policies significantly outperform the conventional scheduling policies in terms of both load balancing and cache hit ratio.

The rest of the paper is organized as follows: In Section  2, we discuss other research efforts related to cache-aware query scheduling and query optimization. In Section  3, we describe the architecture of our distributed query processing framework. In Section  4, we discuss BEMA (Balanced Exponential Moving Average) scheduling policy  [15] and analyze its load balancing behavior. In Section  5, we propose a novel scheduling policy—EM-KDE (Exponential Moving-Kernel Density Estimation) that improves load balancing. In Section  6 we present an extensive experimental evaluation, where we examine the performance impact of different scheduling policies, measuring both query execution and waiting time, as well as load balancing. Finally we conclude in Section  7.