Uneven memory regulation for scheduling IMA applications on multi-core platforms

Abstract

The adoption of multi-cores for mixed-criticality systems has fueled research on techniques for providing scheduling isolation guarantees to applications of different criticalities. These are especially hard to provide in the presence of contention in shared resources of the system, such as buses and DRAMs. The state-of-the-art Single-Core Equivalence (SCE) framework improves timing isolation by enforcing periodic memory access budgets per core, which allows computing safe stall delays for the cores as input to the schedulability analysis. In this work, we extend the theoretical toolkit for this state-of-the-art framework by considering EDF and server-based scheduling, instead of partitioned fixed-priority scheduling which SCE has assumed so far. A second extension to the theory of SCE consists in additionally allowing memory access budgets to be uneven and defined on a per-server basis, rather than just on a per-core basis, which is what was supported until now. This added flexibility allows better memory bandwidth efficiency, especially when servers with dissimilar memory access requirements co-exist on a given core, and this in turn improves schedulability. Finally, we also formulate an Integer-Linear Programming Model (ILP) guaranteed to find a feasible mapping of a given set of servers to processors, including their execution time and memory access budgets, if such a mapping exists. Our experiments with synthetic task sets confirm that considerable improvement in schedulability can result from the use of per-server memory access budgets under the SCE framework.

Appendices

Proofs of Lemmas in Sect. 5.2

In this section we present the proofs of Lemmas 5 and 6 (For notation, see Table 7.)

Table 7 List of symbols used in the stall term analysis in Sect. 5.2

Lemma 9

Let $\mu _i \le K_i \cdot r_{max}$. If $P-K_iL_{min} \ge K_i(m-1)L_{max}$, then the worst-case stall time of a job of task i that executes for up to $r_{max}$ regulation periods, ignoring any regulation stall upon its first memory access, is upper bounded by:

$$\begin{aligned} stall_i^r(K_i,r_{max}) =&\; \left\lfloor \frac{\mu _i}{K_i} \right\rfloor (P-K_i L_{min}) + r_0 (K-K_i)L_{max}\nonumber \\&+ min(a_{{\bar{r}}}, ({r_{{\bar{r}}}}_{max}-r_0)(a_0-1))(m-1)L_{max} \end{aligned}$$

(20)

where:

$$\begin{aligned}&a_0 = \left\lceil \frac{K- K_i}{m-1} \right\rceil \nonumber \\&a_{{\bar{r}}} = \mu _i - \left\lfloor \frac{\mu _i}{K_i} \right\rfloor K_i \nonumber \\&r_0 = {\left\{ \begin{array}{ll} 0 &{} \quad \text {if } K_i \le a_0 \\ max(min( a_{{\bar{r}}}-(a_0-1){r_{{\bar{r}}}}_{max}, {r_{{\bar{r}}}}_{max}),0) &{} \text { otherwise} \end{array}\right. }\nonumber \\&{r_{{\bar{r}}}}_{max} = r_{max} - \left\lfloor \frac{\mu _i}{K_i} \right\rfloor \end{aligned}$$

(21)

Fig. 18

Stall time in a regulation period as a function of the number of memory accesses, when $P-K_iL_{min} \ge K_i(m-1)L_{max}$

Proof

If $P-K_iL_{min} \ge K_i(m-1)L_{max}$, then the worst case occurs for the maximum number of regulation stalls as illustrated by Fig. 18, which plots the stall per regulation period as a function of the number of memory accesses in that period. Note that, by definition of K and given that $L_{min} \le L_{max}$, it is always the case that $P-K_iL_{min} \ge (K-K_i)L_{max}$.

The maximum number of periods with regulation stalls in any job of task i is $\left\lfloor \frac{\mu _i}{K_i} \right\rfloor$. Let $r_{max}$ be the maximum number of regulation periods over which the job can execute. Therefore, when the number of regulation stalls is maximum, (1) the maximum number of periods without regulation stalls is: ${r_{{\bar{r}}}}_{max} = r_{max} - \left\lfloor \frac{\mu _i}{K_i} \right\rfloor$, if $\mu _i \le K_i \cdot r_{max}$; and 2) the number of memory accesses in periods without regulation stalls is: $a_{{\bar{r}}} = \mu _i - \left\lfloor \frac{\mu _i}{K_i} \right\rfloor K_i$.

These $a_{{\bar{r}}}$ accesses will be distributed over several regulation periods. Let $r_0$ be the number of these regulation periods with at least $a_0$ memory accesses. Then, by Lemma 1 and Observation 1, the total contention stall time in all periods without regulation stalls is:

$$\begin{aligned} r_0 (K-K_i)L_{max} + \left( a_{{\bar{r}}}-\sum _{j=1}^{r_0}a_j\right) (m-1)L_{max} \end{aligned}$$

(22)

where $a_j$ is the number of memory accesses in regulation period j with at least $a_0$ memory accesses. To determine the maximum of (22), we consider two cases, depending on whether $a_0$ is smaller or larger than $K_i$. Note that, it cannot be $a_0 = K_i$, since in this case, we would have a regulation stall upon the $a_0$th access, and therefore all these accesses would be in periods with regulation stalls.

$case\,1$ If $a_0 > K_i$, then the value of (22) is maximum when $r_0$ is 0.

Indeed, if $a_0 > K_i$ then the core stalls before it performs $a_0$ accesses. Thus, (22) becomes:

$$\begin{aligned}&a_{{\bar{r}}}(m-1)L_{max}&\text {by} \; r_0 = 0 \\&= min(a_{{\bar{r}}}, ({r_{{\bar{r}}}}_{max}-r_0)(a_0-1))(m-1)L_{max}&\hbox {by}\; a_0 > K_i \hbox {and}\; r_0 = 0 \end{aligned}$$

Indeed:

$$\begin{aligned}&({r_{{\bar{r}}}}_{max}-r_0)(a_0-1)&\\&= r_{{\bar{r}}}(a_0-1)&\text {by} \; r_0 = 0 \\&\ge r_{{\bar{r}}}K_i&\hbox {by} \; a_0> K_i \\&> a_{{\bar{r}}} \end{aligned}$$

since, otherwise, there would be at least one regulation stall in the $r_{{\bar{r}}}$ regulation periods without regulation stalls. Thus, (20) provides an upper bound on the worst-case contention stall time.

$Case\,2$ If $a_0 < K_i$. We consider two further cases depending on the number of memory accesses in periods without regulation stalls.

$Case\,2.1$ If $a_{{\bar{r}}} \le {r_{{\bar{r}}}}_{max}(a_0-1)$, then it is possible to distribute all the $a_{{\bar{r}}}$ memory over the ${r_{{\bar{r}}}}$ regulation periods in such a way that there is less than $a_0$ memory accesses in each regulation period, which by Lemma 1 leads to the worst-case contention stall time. Thus, $r_0 = 0$, and therefore, by the same arguments as in Case 1, (20) provides an upper bound on the worst-case contention stall time.

Note also that in this case, $a_{{\bar{r}}} \le {r_{{\bar{r}}}}_{max}(a_0-1)$, the first argument of the min() function in the second case of (21) is negative, and therefore, (21) takes value 0, as it should.

$Case\,2.2$ If $a_{{\bar{r}}} > {r_{{\bar{r}}}}_{max}(a_0-1)$, then there must be regulation periods without regulation stalls and with at least $a_0$ memory accesses.

In this case, the value of (22) is maximum when all regulation periods have at least $a_0-1$ memory accesses and we maximize the value of $r_0$. Indeed, as illustrated in Fig. 18, in regulation periods without regulation stalls and at least $a_0$ memory accesses, additional memory accesses do not increase the stall time upper bound. On the other hand, the total stall time on a regulation period increases upon the $a_0$th access.

Thus, in this case, the worst case occurs when each memory access above $(a_0-1){r_{{\bar{r}}}}_{max}$ occurs in a different regulation period of the ${r_{{\bar{r}}}}_{max}$ regulation periods without regulation stalls. However, there are only ${r_{{\bar{r}}}}_{max}$ of these regulation periods, hence the second argument of the min() function in the second case of (21). Furthermore, in this case, $a_{{\bar{r}}} > {r_{{\bar{r}}}}_{max}(a_0-1)$, the min() function takes a positive value, and therefore $r_0$ takes the value of the min() function, as it should.

Finally, the first factor of the second term of (22), that is the number of accesses in periods with less than $a_0$ memory accesses is given by $({r_{{\bar{r}}}}_{max}-r_0)(a_0-1)$, which is smaller than $a_{{\bar{r}}}$ in this case, $a_{{\bar{r}}} > {r_{{\bar{r}}}}_{max}(a_0-1)$, since $r_0 \ge 0$. Therefore, (20) provides an upper bound for the worst-case stall-time also in this case. $\square$

Lemma 10

Let $\mu _i \le K_i \cdot r_{max}$. If $P-K_iL_{min} < K_i(m-1)L_{max}$, the worst-case stall time of a job of task i that executes for up to $r_{max}$ regulation periods, ignoring any regulation stall upon its first memory access, is upper bounded by:

$$\begin{aligned} stall^c_i(K_i, r_{max}) = {\left\{ \begin{array}{ll} \mu _i(m-1)L_{max} &{} \text { if } \mu _i \le r_{max}a_{\overline{r0}} \\ (r_{max}-r_0 - r_r) a_{\overline{r0}}(m-1)L_{max} \\ + r_0(K-K_i)L_{max} \\ + r_r(P-K_iL_{min}) &{} \text { otherwise} \end{array}\right. } \end{aligned}$$

(23)

where:

$$\begin{aligned}&a_0 = \left\lceil \frac{K- K_i}{m-1} \right\rceil \\&a_{\overline{r0}} = min(K_i, a_0) - 1 \\&\varDelta _0=(K-K_i)L_{max} - (a_0-1)(m-1)L_{max} \\&\varDelta _r = \frac{(P-K_iL_{min})-(a_0-1)(m-1)L_{max}}{K_i-(a_0-1)}\\&r_r = {\left\{ \begin{array}{ll} max(0, \mu _i-r_{max}(K_i-1)) &{} \text {if } K_i< a_0 \text { or } (K_i> a_0 \text { and } \varDelta _0> \varDelta _r)\\ max\left( 0, \left\lfloor \frac{\mu _i-r_{max}(a_0-1)}{K_i-(a_0-1)} \right\rfloor \right) &{} \text {if } K_i> a_0 \text { and } \varDelta _0 \le \varDelta _r \\ 0 &{} \text {if } K_i = a_0 \end{array}\right. } \\&r_0 = {\left\{ \begin{array}{ll} 0 &{} \text {if } K_i < a_0 \\ max(0,min(\mu _i - r_{max}(a_0-1), r_{max}-r_r)) \\ &{} \text {if } K_i> a_0\text { and } \varDelta _0> \varDelta _r\\ max(0, min(\mu _i - (r_{max}-r_r)(a_0-1)-r_{r}K_i, r_{max}-r_r)) \\ &{} \text {if } (K_i > a_0 \text { and } \varDelta _0 \le \varDelta _r) \text { or } K_i = a_0 \\ \end{array}\right. } \end{aligned}$$

Proof

If $P-K_iL_{min} < K_i(m-1)L_{max}$, then the worst-case scenario occurs when the number of contention stalls outside periods with 1) regulation stalls or, by Lemma 1, 2) at least $a_0$ memory accesses, is maximum.

The maximum number of memory accesses per regulation period outside periods with (1) or (2) is given by $a_{\overline{r0}} = min(K_i, a_0) - 1$.

$Case\,1$$\mu _i \le r_{max}a_{\overline{r0}}:$ In this case, all memory accesses may occur outside periods with (1) or (2) and, by Observation 1, an upper bound of the worst-case stall time is:

$$\begin{aligned} \mu _i(m-1)L_{max} \end{aligned}$$

(24)

$case\,2$$\mu _i > r_{max}a_{\overline{r0}}:$ In this case, at least one period will have either a regulation stall or at least $a_0$ memory accesses. Let $r_{r}$ and $r_0$ be the number of periods of each, respectively. Then, by Observation 1 and by Lemmas  3 and 1, an upper bound of the stall time is:

$$\begin{aligned}&(r_{max}-r_0 - r_r) a_{\overline{r0}}(m-1)L_{max}\nonumber \\&\; + r_0(K-K_i)L_{max} + r_r(P-K_iL_{min}) \end{aligned}$$

(25)

To complete the proof, we need to derive the values of $r_0$ and $r_r$. We do that by case analysis, in which we use the following two parameters:

$$\begin{aligned} \varDelta _0=(K-K_i)L_{max} - (a_0-1)(m-1)L_{max} \\ \varDelta _r = \frac{(P-K_iL_{min})-(a_0-1)(m-1)L_{max}}{K_i-(a_0-1)} \end{aligned}$$

$\varDelta _0$ denotes the additional stall time upon the $a_0$th memory access, whereas $\varDelta _r$ is the average stall time per memory access after the $a_0-1$th, when core i has a regulation stall. Figure 19 illustrates the meaning of these parameters.

We consider three cases, depending on the relative values of $K_i$ and $a_0$, and we divide one of these cases in two subcases, depending on the relative values of $\varDelta _0$ and $\varDelta _r$.

$Case\,2.1$$K_i < a_0:$ If $K_i < a_0$ then the core will stall before it performs $a_0$ memory accesses. Therefore $r_0$ is always 0, and, by $P-K_iL_{min} < K_i(m-1)L_{max}$, (25) will be maximum when $r_r$ is minimum. This occurs when all memory accesses are spread by the $r_{max}$ periods in such a way that a regulation stall will occur only when all $r_{max}$ periods have $(K_i-1)$ memory accesses. Thus, $r_r = max(0, \mu _i-r_{max}(K_i-1))$, if $\mu _i \le K_i(\lceil D_i/P \rceil + 1)$.

Fig. 19

Stall time in a regulation period as a function of the number of memory accesses, when $\varDelta _0 > \varDelta _r$

$Case\,2.2$$K_i > a_0:$ By $P-K_iL_{min} < K_i(m-1)L_{max}$ and by Lemma 1, (25) will be maximum when each regulation period has at least $(a_0-1)$ accesses and either (1) $r_0$ is maximum, or (2) $r_r$ is maximum. This is shown next, using a case analysis depending on the relative values of the parameters $\varDelta _0$ and $\varDelta _r$ defined above.

$Case\,2.2.2$$\varDelta _0 > \varDelta _r:$ This case is illustrated by Fig. 19, which plots the per regulation period stall as a function of memory accesses per period. By the definition of $\varDelta _0$ and $\varDelta _r$, the worst case occurs when the accesses above $r_{max}(a_0-1)$ are spread over the $r_{max}$ regulation periods so as to maximize $r_0$. In this case, $r_0$ is given by:

$$\begin{aligned} r_0= max(0,min(\mu _i - r_{max}(a_0-1),r_{max}-r_r) \end{aligned}$$

(26)

Indeed, $r_0$ cannot be lower than 0, and cannot be larger than $r_{max}-r_r$.

To ensure that $r_0$ is maximum, $r_r$ must be minimum and, if $\mu _i \le K_i(\lceil D_i/P \rceil + 1)$, is given by:

$$\begin{aligned} r_r = max\left( 0,\mu _i - r_{max}(K_i-1)\right) \end{aligned}$$

(27)

Indeed, $r_r$ cannot always be 0. Once a job performs $K_i-1$ memory accesses in every regulation period, each additional memory access leads to one additional regulation stall, thus increasing $r_r$ by one.

Note that although Fig. 19 illustrates the case of $P-K_iL_{min} > (K-K_i)L_{max}$, by definition of $\varDelta _0$ and $\varDelta _r$, $\varDelta _0 > \varDelta _r$ also when $P-K_iL_{min} = (K-K_i)L_{max}$ and $K_i > a_0$. (Note that, as shown in the proof of Lemma 3, $P-K_iL_{min} \ge (K-K_i)L_{max}$.)

Fig. 20

Stall time in a regulation period as a function of the number of memory accesses, when $\varDelta _0 < \varDelta _r$

$Case\,2.2.1$$\varDelta _0 \le \varDelta _r:$ This case is illustrated in Fig. 20, which plots the per regulation period stall as a function of memory accesses per period. By the definition of $\varDelta _0$ and $\varDelta _r$, the worst case occurs when the accesses above $r_{max}(a_0-1)$ are distributed over the $r_{max}$ regulation periods so as to maximise $r_r$. Thus, if $\mu _i \le K_i(\lceil D_i/P \rceil + 1)$, the maximum number of regulation periods with regulation stalls is:

$$\begin{aligned} r_r = max\left( 0, \left\lfloor \frac{\mu _i-r_{max}(a_0-1)}{K_i-(a_0-1)} \right\rfloor \right) \end{aligned}$$

(28)

When all regulation periods have at least $(a_0-1)$ memory accesses and$r_r$ is maximum (as given by (28)), (25) will be maximum, if $r_0$ is also maximised. This means, if there are additional memory accesses that are not enough to cause one additional regulation stall, by Lemma 1, the stall will be maximum if these additional memory accesses are spread over as many regulation periods as possible, rather than clustered into a single regulation period. Thus, if $\mu _i \le K_i(\lceil D_i/P \rceil + 1)$, the value of $r_0$ that maximises (25) is:

$$\begin{aligned} r_0 = max(0, min(\mu _i - (r_{max}-r_r)(a_0-1)-r_{r}K_i,r_{max}-r_r)) \end{aligned}$$

(29)

$Case\,2.3$$K_i = a_0:$ In this case, every access above $r_{max}(a_0-1)$ both causes a regulation stall and is the $a_0$th access of one regulation period. Thus, for each of these regulation periods, we have two upper bounds on the per regulation period stall: $(K-K_i)L_{max}$, by Lemma 1, and $P-K_iL_{min}$, by Lemma 3. Since we want the tightest upper bound of the total stall, we must use the smallest of these values. Nevertheless, we can still use (25), as for the other cases, as long as we set $r_0$ and $r_r$ to the appropriate values, derived below.

As shown in Case 2 of Lemma 3, $(K-K_i)L_{max} \le K - K_i L_{min}$. Therefore, we consider each additional access as leading to the $a_0$th access, i.e. $r_r = 0$ and $r_0 = max(0, \mu _i-r_{max}(a_0-1))$, if $\mu _i \le K_i(\lceil D_i/P \rceil + 1)$.

Note that for $r_r = 0$, the expression of $r_0$ in Case 2.2, see (29), reduces to the expression of $r_0$ in this case. Therefore, we also use (29) for this case in the formulation of this lemma, thus avoiding to introduce yet another case in the expression of $r_0$. $\square$

Linearisation of constraints

In this section, we present a linearisation of constraints (10), (14), (15) and (17), thus making the ILP model presented in Sect. 7 strictly linear.

$$\begin{aligned}&\sum _{\forall j} \sum _{\forall k} q_{ijk}K_k \le K, \; \forall i\quad \quad \quad \quad \quad \quad \quad \qquad \qquad {(10)} \\&\mathbf{or }, \;\; \sum _{\forall j} \sum _{\forall k} z_{ijk} \le K, \; \forall i \\&K_k - K (1-q_{ijk}) \le z_{ijk}, \; \forall i,j,k \\&z_{ijk}\ge 0, \; \forall i,j,k \\&z_{ijk}\le K_k, \; \forall i,j,k \\&z_{ijk}\le K q_{ijk}, \; \forall i,j,k \\&z_{ijk} \in [0, K]\; \\&X_k \in [0,Q]\; \\&K_k \in [0,K]\; \end{aligned}$$

The $z_{ijk}$ decision variables are the artefacts of the linearisation of (10)—each product $q_{ijk}K_k$ is replaced by a variable $z_{ijk}$—and therefore it is not defined earlier. The linearisation of constraints (14) and (15) are presented as follows.

$$\begin{aligned}&f_{ik} = 1 \Rightarrow \sum _{\ell =0}^{i-1} q_{\ell jk} = 0, \quad \forall i,j,k \quad \quad \quad \quad \quad \quad {(14)} \\&\mathbf{or }, \;\; \; (i \times f_{ik}) + \sum _{\ell =0}^{i-1}q_{\ell jk} \le i, \; \forall i,j,k \\&l_{ik} = 1 \Rightarrow \sum _{\ell =i+1}^{Q-1} q_{\ell jk} = 0, \quad \forall i,j,k \quad \quad \quad \quad \quad \quad {(15)} \\&\mathbf{or }, \;\; \; (Q-i-1)l_{ik} + \sum _{\ell =i+1}^{Q-1}q_{\ell jk} \le Q-i-1, \; \forall i,j,k \end{aligned}$$

Similarly, we present the linearisation of (17). Similar to previous case (10), the $b_{ijk}$ decision variables results from the linearisation of (17) and denote the each product $q_{ijk}c_{jk}$.

$$\begin{aligned}&c_{jk}= 1 \Rightarrow \sum _{\forall i} q_{ijk} = X_k, \quad \forall j,k \quad \quad \quad \quad \quad \quad {(17)} \\&\mathbf{or }, \;\; \sum _{\forall j}\sum _{\forall i} q_{ijk} c_{jk} = X_k, \; \forall k \\&\mathbf{or }, \;\; \sum _{\forall j}\sum _{\forall i} b_{ijk} = X_k, \; \forall k, \\&b_{ijk} \le c_{jk}, \; \forall i,j,k \\&b_{ijk} \le q_{ijk}, \; \forall i,j,k \\&b_{ijk}\ge c_{jk}+q_{ijk}-1, \; \forall i,j,k \end{aligned}$$