Formalizing Data Locality in Task Parallel Applications

Abstract

Task-based programming provides programmers with an intuitive abstraction to express parallelism, and runtimes with the flexibility to adapt the schedule and load-balancing to the hardware. Although many profiling tools have been developed to understand these characteristics, the interplay between task scheduling and data reuse in the cache hierarchy has not been explored. These interactions are particularly intriguing due to the flexibility task-based runtimes have in scheduling tasks, which may allow them to improve cache behavior.

This work presents StatTask, a novel statistical cache model that can predict cache behavior for arbitrary task schedules and cache sizes from a single execution, without programmer annotations. StatTask enables fast and accurate modeling of data locality in task-based applications for the first time. We demonstrate the potential of this new analysis to scheduling by examining applications from the BOTS benchmarks suite, and identifying several important opportunities for reuse-aware scheduling.

This work was supported by the Swedish Foundation for Strategic Research project FFL12-0051, the Swedish Research Council Linnaeus UPMARC centre of excellence.

A Appendix: Proofs

Lemma

Let E and $E'$ be execution traces such that they share the exact same set of accesses, but in different order. Then

$$ (x_j, x_k) \in \tilde{\mathcal {R}}^{E} \Rightarrow (x_j, x_k) \in \tilde{\mathcal {R}}^{E'} \vee (x_k, x_j) \in \tilde{\mathcal {R}}^{E'}. $$

Proof

Let $(x_j, x_k)$ be an element of $\tilde{\mathcal {R}}^{E}$. By definition, $a^j = a^k$. Since E and $E'$ share the same set of accesses, there exist $x_0$ and $x_1$ in $E'$ such that $x_j=x_0$ and $x_k=x_1$. Lets assume $x_0 < x_1$, since $a^j = a^k$ then $(x_0, x_1) \in \tilde{\mathcal {R}}^{E'}$. If $x_1 < x_0$ then $(x_0, x_1) \in \tilde{\mathcal {R}}^{E'}$

Lemma

$\mathcal {M}^{-1}$ and $\mathcal {M}$ are inverses.

Proof

Let T be a task, such that $E_T = x_1\dots x_r$. We can see that

$$ \mathcal {M}(\mathcal {M}^{-1}(x_1\dots x_r)) = \mathcal {M}(T(x_1)) = \mathcal {M}(T) = E_T = x_1\dots x_r $$

Conversely,

$$ \mathcal {M}^{-1}(\mathcal {M}(T)) = \mathcal {M}^{-1}(E_T) = \mathcal {M}^{-1}(x_1\dots x_r) = T(x_1) = T $$

Theorem

$Q = \mathcal {R}^{E_{S'}}$

Proof

It is straightforward to prove that $Q\subseteq \mathcal {R}^{E_{S'}}$. It is enough to observe that $\mathcal {C}_{S}(T, T')$ gives al the pairs in $\tilde{\mathcal {R}}^{E_S}$ that start in T and end in $T'$. Since this set is execution independent, those pairs are also in $\tilde{\mathcal {R}}^{E_S'}$. The condition $\forall T_q \in S'$ such that $T(x_k)< T_q < T(x_j) \Rightarrow a^j \notin a(T_q)$ filters out the non-consecutive reuses. Therefore, all the elements of Q are consecutive reuses.

We will now show an outline for the proof that $R^{E_{S'}} \subseteq Q$. If $(x_j, x_{j+d}) \in (T_0 \rightarrow T_0)_{S'}$, then $T(x_j) = T(x_{j+d}) = T_0$. As S and $S'$ use the same task universe, $\exists T_r \in S$ such that $T_0^S = T_r^{S'}$. By Lemma 1, $E_{T_0} = E_{T_r}$. Therefore, $\exists x_p \in E_{T_r}$ such that $x_p = x_j$ and $x_{p+d} = x_{j+d}$, as private reuses are relatively offset. Then $(x_j, x_{j+d}) \in (T_r \rightarrow T_r)_S \in Q_{T_r, T_r}\subseteq Q$.

Let’s now consider the case $(x_j, x_{j+d}) \in (T_n \rightarrow T_m)_{S'}$. The tasks $T_n$ and $T_m$ also occur in S. We will assume that $T_n < T_m$. Let $T_1,\dots ,T_k$ such that $T_n< T_1< \cdots< T_k < T_m$. The proof is by induction on k.

When $k=0$, then the sequence $T_nT_m$ occur in S. Therefore, it exists $x_r$ in $E_S$ such $[x_j, x_j+d] = [x_r, x_r+d]$. Since $x_r = x_j$ and $(x_j, x_{j+d}) \in \mathcal {R}^{E_{S'}}$, we know that $\forall x_r< x_s < x_{r+d}$, $a^r \ne a^s$. Therefore $(x_j, x_{j+d}) = (x_r, x_{r+d}) \in (T_n \rightarrow T_m)^0 \subseteq Q_{Tn,Tm} \subseteq Q$.

When $k=1$, then $T_nT_{n+1}T_m \in S$. Lets assume that $a(T_{n+1}) \ne a(T_n)$, thus, $\forall x_s$ such that $x_j< x_s < x_j+d \Rightarrow a^s \ne a^j$. Therefore $(x_j, x_{j+d}) \in (T_n \rightarrow T_m) \subseteq Q_{T_n,T_m}$.

If that does not happen, it is enough to assume that there are unique $x_1, \dots , x_q$ such that $x_j< x_1< \dots< x_q < x_{j+d}$ and that $a^j = a^1 = \cdots = a^q = a^{j+d}$, with all the accesses in between with different addresses. This means that the pairs $(x_j, x_1), (x_1, x_2), \dots , (x_q, x_{j+d})$ are elements of $\mathcal {R}^{E_S}$. Therefore, by definition of $\mathcal {C}$, this means that $(x_j, x_{j+d}) \in \mathcal {C}(T_n, T_m) \subseteq Q_{T_n,T_m} \subseteq Q$.

The final case is $k \Rightarrow k+1$. Let $T_nT_1\dots T_{k+1}T_m \in S$. Two cases are necessary to prove. The first one, where the set of addresses are of tasks $T_1,\dots , T_k$ are disjoint from $T_n, T_m$. If that happens, then the only thing left to check is the set of addresses of $T_k$, which is analogous to the case $k=1$ for $T_{k+1}$. Otherwise, a unique number of accesses with the same address appear in $T_1, \dots , T_k$, which by inductive hypothesis can be used transitively to obtain a pair in Q.

The proof for when $T_m < T_n$ is analogous.

Theorem

$\gamma = \delta _{\mathcal {R}^{E_{S'}}}$

Proof

Let $(x_j, x_k) \in \mathcal {R}^{E_{S'}}$, and let $T_{x_j} = T(x_j)$ and $T_{x_k} = T(x_k)$. Let $T_{n_1},\dots ,T_{n_r}$ be such that $T_{x_j}T_{n_1}\dots T_{n_r}T_{x_k} \in S'$. These are the tasks scheduled between the starting and ending tasks causing the reuse in $S'$. Lets also assume $T_{m_1},\dots ,T_{m_s}$ such that $T_{x_j}T_{m_1}\dots T_{m_s}T_{x_k} \in S$, thus representing the tasks between the starting and ending tasks of the reuse in S. It is easy to see the following memory access sequence when $S'$ is executed:

$$\begin{aligned} x_j \dots x_l^{T_{x_j}} x_1^{T_{n_1}} \dots x_l^{T_{n_1}} \dots x_1^{T_{n_r}} \dots x_l^{T_{n_r}} x_1^{T_{x_k}} \dots x_k = x_j \dots x_l^{T_{x_j}} E_{T_{n_1}} \dots E_{T_{n_r}} x_1^{T_{x_k}} \dots x_k. \end{aligned}$$

Therefore, the since the access distance is linear, we can see that

$$\begin{aligned} \delta _{\mathcal {R}^{E_{S'}}}(x_j, x_k)= & {} \delta _{\tilde{\mathcal {R}}^{E_{S'}}}(x_j, x_l^{T_{x_j}}) + |E_{T_{n_1}}| + \cdots + |E_{T_{n_r}}| + \delta _{\tilde{\mathcal {R}}^{E_{S'}}}(x_1^{T_{x_k}}, x_k)\\= & {} \delta _{\tilde{\mathcal {R}}^{E_{S'}}}(x_j, x_l^{T_{x_j}}) + |\mathcal {M}(T_{n_1})| + \cdots + |\mathcal {M}(T_{n_r})| + \delta _{\tilde{\mathcal {R}}^{E_{S'}}}(x_1^{T_{x_k}}, x_k)\\= & {} \delta _{\tilde{\mathcal {R}}^{E_{S'}}}(x_j, x_l^{T_{x_j}}) + \nu _{S'}(T_{x_j}, T{x_k}) + \delta _{\tilde{\mathcal {R}}^{E_{S'}}}(x_1^{T_{x_k}}, x_k) \end{aligned}$$

On the other hand, we can also see the following sequence when S is executed:

$$ x_j \dots x_l^{T_{x_j}} x_1^{T_{m_1}} \dots x_l^{T_{m_1}} \dots x_1^{T_{m_s}} \dots x_l^{T_{m_s}} x_1^{T_{x_k}} \dots x_k, $$

analogously, see that $\delta _{\mathcal {R}^{E_{S}}}(x_j, x_k) = \delta _{\tilde{\mathcal {R}}^{E_{S}}}(x_j, x_l^{T_{x_j}}) + \nu _{S}(T_{x_j},T_{x_k}) + \delta _{\tilde{\mathcal {R}}^{E_{S}}}(x_1^{T_{x_k}}, x_k)$, and therefore, $\delta _{\mathcal {R}^{E_{S}}}(x_j, x_l^{T_{x_j}}) + \delta _{\tilde{\mathcal {R}}^{E_{S}}}(x_1^{T_{x_k}}, x_k) =\delta _{\tilde{\mathcal {R}}^{E_{S}}}(x_j, x_k) - \nu _{S}(T_{x_j},T_{x_k})$. Since the sequence $x_j \dots x_l^{T_{x_j}}$ is identical both in $E_S$ and $E_{S'}$ then

$\delta _{\tilde{\mathcal {R}}^{E_{S}}}(x_j, x_l^{T_{x_j}}) = \delta _{\tilde{\mathcal {R}}^{E_{S'}}}(x_j, x_l^{T_{x_j}})$. The same holds for $x_1^{T_{x_k}} \dots x_k$. Then,

$$\begin{aligned} \delta _{\mathcal {R}^{E_{S'}}}(x_j, x_k)= & {} \delta _{\tilde{\mathcal {R}}^{E_{S'}}}(x_j, x_l^{T_{x_j}}) + \nu _{S'}(T_{x_j}, T{x_k}) + \delta _{\tilde{\mathcal {R}}^{E_{S'}}}(x_1^{T_{x_k}}, x_k) \\= & {} \delta _{\tilde{\mathcal {R}}^{E_{S'}}}(x_j, x_l^{T_{x_j}}) + \delta _{\tilde{\mathcal {R}}^{E_{S'}}}(x_1^{T_{x_k}}, x_k) + \nu _{S'}(T_{x_j}, T{x_k})\\= & {} \delta _{\tilde{\mathcal {R}}^{E_{S}}}(x_j, x_k) - \nu _{S}(T_{x_j},T_{x_k}) + \nu _{S'}(T_{x_j}, T_{x_k})\\= & {} \gamma (x_j, x_k) \end{aligned}$$