A scheduling framework for distributed key-value stores and its application to tail latency minimization

Abstract

Distributed key-value stores employ replication for high availability. Yet, they do not always efficiently take advantage of the availability of multiple replicas for each value and read operations often exhibit high tail latencies. Various replica selection strategies have been proposed to address this problem, together with local request scheduling policies. It is difficult, however, to determine what is the absolute performance gain each of these strategies can achieve. We present a formal framework allowing the systematic study of request scheduling strategies in key-value stores. We contribute a definition of the optimization problem related to reducing tail latency in a replicated key-value store as a minimization problem with respect to the maximum weighted flow criterion. By using scheduling theory, we show the difficulty of this problem and therefore the need to develop performance guarantees. We also study the behavior of heuristic methods using simulations that highlight which properties enable limiting tail latency: for instance, the EarliestFinishTime strategy—which uses the earliest next available time of servers—exhibits a tail latency that is less than half that of state-of-the-art strategies, often matching the lower bound. Our study also emphasizes the importance of metrics such as the stretch to properly evaluate replica selection and local execution policies.

Appendices

Appendix A Proof of Theorem 5

Proof

First, we build an instance designed to reach an arbitrarily large ratio. Then, we determine a lower bound on the objective achieved with Max-Flow, and finally, an upper bound on the optimal one.

Instance characteristics For an arbitrary competitive ratio $k\ge 1$, we build the following instance with n requests. The first k requests have a weight $w_i=k$ and release time $r_i=0$. Then, a new request arrives at each new time step with a weight that is the highest integer lower than or equal to $1+1/k$ times the weight of the previous request (i.e., $w_i=\lfloor (1+1/k)w_{i-1}\rfloor$ and $r_i=i-k$ for $k<i\le n$). In total, $n=k^2+11$ requests are submitted.

Lower bound At time $t=0$, Max-Flow starts one of the first k requests because they are the only ones that are ready. We now prove that at any time t such that $1\le t<k$, Max-Flow starts one of the remaining first k requests, which delays all arriving requests (any request $T_i$ such that $k<i<2k$).

On the one hand, $w_i(t+1-r_i)=k(t+1)$ for any of the first k requests ($1\le i\le k$). On the other hand, for $k<i\le n$, $w_i\le (1+1/k) w_{i-1}$, and thus, $w_i\le (1+1/k)^{i-k} k$. Therefore, $w_i(t+1-r_i)\le (1+1/k)^{i-k} k(t+1-i+k)$.

Let us show that at any time t such that $1\le t<k$, any of the first k requests has the highest value, that is $k(t+1)\ge (1+1/k)^{i-k} k(t+1-i+k)$ for all $k<i\le t+k$. By changing variables ($j=i-k$ and $t'=t+1$), this corresponds to proving $(1+1/k)^j (t'-j)\le t'$ for all $1\le j<t'\le k$. We show by induction that $(1+1/k)^j (t'-j)\le t'$ for all $0\le j$ and for a given $t'$ ($2\le t'\le k$). The induction basis with $j=0$ is direct. The induction step assumes $(1+1/k)^j (t'-j)\le t'$ to be true for a given $j\ge 0$. We have

$$\begin{aligned} (1+1/k)\frac{t'-j-1}{t'-j}= & {} (1+1/k)\left( 1-\frac{1}{t'-j}\right) \\= & {} 1+1/k-\frac{1}{t'-j}-\frac{1}{k(t'-j)}\le 1. \end{aligned}$$

The last line is obtained by remarking that $t'\le k$ and $j\ge 0$ (thus, $1/k\le \frac{1}{t'-j}$). Therefore,

$$\begin{aligned}{} & {} (1+1/k)^{j+1} (t'-(j+1))\\{} & {} \quad = (1+1/k)^j(1+1/k)(t'-j)\frac{t'-j-1}{t'-j} \\{} & {} \quad \le (1+1/k)^j(t'-j)\le t', \end{aligned}$$

which concludes the induction proof.

At time $t=k$, all of the first k requests have been completed. We now prove that at any time t such that $k\le t<n$, Max-Flow starts request $T_{t+1}$. This would mean that at time t, only requests $T_i$ such that $t<i\le t+k$ are ready and not completed. We prove by induction that at time $k\le t<n$, all requests $T_i$ with $i\le t$ are completed. The induction basis with $t=k$ is already proven above. Assume the hypothesis is true for a given $k\le t<n$. It remains to prove that at time $t'=t+1$, $T_{t+2}$ is started among requests $T_i$ such that $t'<i\le t'+k$.

On the one hand, $w_i(t'+1-r_i)=w_{t+2}k$ for request $T_{t+2}$. On the other hand, for $t'+1<i\le t'+k$, $w_i(t'+1-r_i)\le (1+1/k)^{i-t'-1} w_{t+2}(t'+1-i+k)$. Let us show that $(1+1/k)^{i-t'-1} w_{t+2}(t'+1-i+k)<w_{t+2}k$ for $t'+1<i\le t'+k$ and for a given $k\le t<n$. By changing variables ($j=i-t'-1$), this corresponds to proving that $(1+1/k)^j (k-j)<k$ for all $0<j<k$. We show this again by induction on j for a given $k\ge 1$. For the induction basis, $(1+1/k)(k-1)=k+1-1-1/k<k$. For the induction step, we can show that $(1+1/k)\frac{k-j+1}{k-j}\le 1$ by remarking that $k>k-j$, which concludes the induction proof.

To conclude on the performance of Max-Flow, request $T_i$ is started at time $i-1$ and therefore, the objective value is at least $w_n F_n=w_n(n-(n-k))=k w_n$.

Upper bound A better objective value can be obtained by starting all requests as soon as they arrive except for the first k ones: request $T_1$ is started at time $t=0$; then, request $T_i$ is started at time $t=i-k$ for $k<i\le n$; finally, the remaining requests among the first k ones are started ($T_i$ is started at time $t=n-k+i-1$ for $1<i\le k$). We analyze the objective value for request $T_k$ because it is the last one to be executed among the first k requests, and $T_n$ because it is the one with the highest weight among the last $n-k$ requests. For $T_k$, $w_k F_k=k(C_k-r_k)=k n$. For $T_n$, $w_n F_n=w_n$.

We prove that $w_n\ge k n$ by deriving a lower bound on $w_n$. The weights increase in multiple stages. At first, each increment is unitary: $w_{i+1}=w_i+1$ for $k\le i<2k$. Then, the increment increases at the second stage and $w_{i+1}=w_i+2$ for $2k\le i<2k+\lceil k/2\rceil$. At the k-th stage, $w_{i+1}=w_i+k$ for a single request. At a given stage j, the increment of the weight is j for at most $\lceil k/j\rceil$ requests. Let $n_1=\sum _{j=1}^k\lceil k/j\rceil$ be the number of such requests (assuming $n-k\ge n_1$). Finally, the remaining $n_2=n-k-n_1$ requests are incremented by a value that increases by at least 1 for each new request: $w_{i+1}\ge w_i+(k+i-n+n_2)$ for $n-n_2<i\le n$.

The last weight $w_n$ is at least the sum of the increments of all these stages:

$$\begin{aligned} w_n\ge k+\sum \limits _{j=1}^k j\lceil k/j\rceil +\sum \limits _{j=1}^{n_2}(k+j). \end{aligned}$$

Thus, $w_n\ge k(k+1)+k n_2+n_2^2 /2$. Our hypothesis is that $w_n\ge k n$, which would be verified if

$$\begin{aligned} k(k+1)+k n_2+n_2^2/2\ge kn. \end{aligned}$$

By replacing $n_2$ and simplifying, the condition becomes

$$\begin{aligned} n\ge k+n_1+\sqrt{2k(n_1-1)}. \end{aligned}$$

(2)

We bound $n_1$ using the asymptotic expansion of the harmonic number $H_k$:

$$\begin{aligned} n_1=\sum \limits _{j=1}^k\lceil k/j\rceil< & {} k\sum \limits _{j=1}^k\frac{1}{j}+k \\< & {} k(H_k+1) \\< & {} k\left( \log (k)+\gamma +\frac{1}{2k}+1\right) , \end{aligned}$$

where $\gamma \approx 0.577$ is the Euler–Mascheroni constant.

Thus, the optimal objective is at most $w_n$ and the one achieved with Max-Flow is at least $k w_n$, which concludes the proof. $\square$

Appendix B Notations

Table 5 summarizes the notations used in this paper.

Appendix C Approximation

We provide a comprehensive summary of results related to approximation and competitive analysis of scheduling problems that address the minimization of flow time. Table 6 presents competitive analysis results that are related to average weighted flow.

Table 5 List of the most used notations

Table 6 Existing results on average (weighted) flow minimization

Table 7 Complexity of sum-flow minimization problems

Appendix D Complexity

To complete our survey on scheduling problems related to our study, we present a summary on complexity results related to sum-flow in Table 7.