State space collapse for multi-class queueing networks under SBP service policies

Abstract

In Braverman et al. [3], the authors justify the steady-state diffusion approximation of a multiclass queueing network under static buffer priority policy in heavy traffic. A major assumption in [3] is the moment state space collapse (moment-SSC) property of the steady-state queue length. In this paper, we prove that moment-SSC holds under a corresponding state space collapse condition on the fluid model. Our approach is inspired by Dai and Meyn [8], which was later adopted by Budhiraja and Lee [4] to justify the diffusion approximation for generalized Jackson networks. We will verify that the fluid state space collapse holds for various networks.

Appendix A. Proofs of results from Sect. 6

In this appendix, we provide details of the proofs of the statements present in Sect. 6.

1.1 A.1 $L^p$-contraction property for the network

First, we prove Lemma 3.5.

Proof of Lemma 3.5

First, by Definition (2.5), since

$$\begin{aligned} \frac{|q_n|_h}{|x_n|_h} \le 1, \frac{|u_n|_h}{|x_n|_h} \le 1, \frac{|v_n|_h}{|x_n|_h} \le 1 \end{aligned}$$

for all n, there exists a subsequence $\{ x_{n_j} \}$ so that (3.14) holds for all the components except for those corresponding to the queue length of low-priority classes from ${\mathcal {L}}$ and (3.16) holds. Meanwhile, since the scaling $|x_n|_h$ does not take into account the low-priority queue length, along a subsequence of $\{ x_n\}$, we have

$$\begin{aligned} \lim _{|x_{n_j}|_h \rightarrow \infty } \frac{Q_l^{k_{n_j}, x_{n_j}}(0)}{|x_{n_j}|_h} = \overline{Q}^{k_0}_l(0), \ l\in {\mathcal {L}} \end{aligned}$$

where $\overline{Q}^{k_0}_l(0)$ could potentially be infinite. This completes the proof of (3.14).

The proof of (3.15) is similar to the arguments from [7]. The key difference lies in the fact that the limiting procedure we consider here is taken over a sequence of networks while [7] focuses on one network. For the rest of this proof, we denote $|\cdot |_h$ by $|\cdot |$.

First, we prove that the fluid limit $\left( \overline{Q}^{k_0}(t), \overline{T}^{k_0}(t) \right)$ satisfies equation (3.8). Based on Lemma 4.2 and Theorem 4.1 in [7], it suffices to prove that, for each $k \in {\mathcal {K}}$ and $l \in {\mathcal {K}}$, as $n \rightarrow \infty$, almost surely,

$$\begin{aligned} \frac{1}{|x_n|} \Phi _l^{k}( \lfloor |x_n|t \rfloor )&\rightarrow P_{kl}^t t, \ \ u.o.c., \end{aligned}$$

(A.1)

$$\begin{aligned} \frac{1}{|x_n|} E_k^{k_n, x_n} ( |x_n|t)&\rightarrow \alpha _k^{k_0} \left( t-\overline{U}_k^{k_0}\right) ^+, \ \ u.o.c., \end{aligned}$$

(A.2)

$$\begin{aligned} \frac{1}{|x_n|} S_k^{k_n, x_n} ( |x_n|t)&\rightarrow \mu _k^{k_0} \left( t-\overline{V}_k^{k_0}\right) ^+, \ \ u.o.c., \end{aligned}$$

(A.3)

where $\lfloor y \rfloor$ denotes the integer part of $y \in \mathbb {R}$.

First, (A.1) follows from the functional strong law of large numbers. To prove (A.2), it is enough to prove that, almost surely,

$$\begin{aligned} \frac{1}{|x_n|} E_l^{k_n, 0} (|x_n|t) \rightarrow \alpha _l^{k_0} t, \ \ u.o.c. \end{aligned}$$

since

$$\begin{aligned} \frac{1}{|x_n|} E_l^{k_n, x_n} (|x_n|t) = \frac{1}{|x_n|} E_l^{k_n, 0} \left( |x_n| \left( t - \frac{u_l^{k_n}}{|x_n|}\right) ^+\right) . \end{aligned}$$

By (3.1) and (2.7), we have

$$\begin{aligned} E^{k_n, 0}_l(|x_n|t)&= \max \left\{ i \ge 1: u^{k_n}_{l}(1)+\cdots +u^{k_n}_{l}(i-1) \le |x_n|t \right\} \\&= \max \left\{ i \ge 1: u_{l}(1)+\cdots +u_{l}(i-1) \le \alpha _l^{k_n}|x_n|t \right\} \\&= E_l^0 \left( \alpha _l^{k_n}|x_n|t \right) \end{aligned}$$

where we define

$$\begin{aligned} E_l^{0} (t) = \max \{i \ge 1: u_{k}(1)+\cdots +u_{k}(i-1) \le t \}. \end{aligned}$$

(A.4)

By the strong law of large numbers for renewal processes, we have, almost surely,

$$\begin{aligned} \lim _{t \rightarrow \infty } \frac{E_l^{0} (t)}{t} = 1. \end{aligned}$$

Therefore, we conclude that, for each $t \ge 0$, almost surely,

$$\begin{aligned} \lim _{n \rightarrow \infty } \frac{1}{|x_n|} E^{k_n, 0}_l(|x_n|t) = \lim _{n \rightarrow \infty } \alpha _l^{k_n}t \frac{1}{ \alpha _l^{k_n}|x_n|t } E_l^0 \left( \alpha _l^{k_n}|x_n|t \right) = \alpha _l^{k_0}t. \end{aligned}$$

It follows from Lemma A.10 from [9] that (A.2) holds. Finally, one can prove (A.3) in a similar way and thus the details are omitted. To complete the proof, we show that

$$\begin{aligned} \sum _{l \in {\mathcal {H}}(k)} \overline{W}^{k_0}_l(t)>0 \text { implies that } \frac{d}{dt} \left( \sum _{l \in {\mathcal {H}}(k)} \overline{T}^{k_0}_l(t)\right) = 1, \ k \in {\mathcal {K}}. \end{aligned}$$

First, if $\sum _{l \in {\mathcal {H}}(k)} \overline{Q}^{k_0}_l(t)>0$, it follows from Sect. 7.2 in [9] that $\frac{d}{dt} \left( \sum _{l \in {\mathcal {H}}(k)} \overline{T}^{k_0}_l(t)\right) = 1$. Now suppose that

$$\begin{aligned} \sum _{l \in {\mathcal {H}}(k)} \mu _l^{k_0} (\overline{V}^{k_0}_l -\overline{T}^{k_0}_l(t) )^+ >0. \end{aligned}$$

There exists $i \in {\mathcal {H}}(k)$ so that $\overline{V}^{k_0}_i>\overline{T}^{k_0}_i(t)$. Moreover, there exists $n_0>0$ and $\delta >0$ such that for $n\ge n_0$,

$$\begin{aligned} T^{k_n, x_n}_i(|x_n| (t+\delta )) < V^{k_n, x_n}_i, \, n\ge n_0. \end{aligned}$$

This means that the residual service time $V^{k_n,x_n}_i$ has not be depleted yet by time $|x_n|(t+\delta )$. By the preemptive-resume SBP policy, the server has not allocated any service time to classes $k \notin {\mathcal {H}}(i)$ in $[0, |x_n|(t+\delta )]$, namely,

$$\begin{aligned} T^{k_n, x_n}_k(|x_n|(t+\delta )) = 0, \ k \notin {\mathcal {H}}(i). \end{aligned}$$

Since the policy is non-idling, we have

$$\begin{aligned} \sum _{l \in {\mathcal {H}}(k)} T^{k_n, x_n}_l(|x_n| s) =|x_n| s, \ s\le t+\delta \end{aligned}$$

for $n\ge n_0$. Since

$$\begin{aligned} \frac{1}{|x_n|} \sum _{l \in {\mathcal {H}}(k)} T^{k_n, x_n}_l(|x_n| s) = s, \ s\le t+\delta , \end{aligned}$$

by taking $n\rightarrow \infty$, one has

$$\begin{aligned} \sum _{l \in {\mathcal {H}}(k)} \overline{T}^{k_0}_l(s)=s, \ s\le [0, t+\delta ], \end{aligned}$$

which implies that $\frac{d}{dt} \left( \sum _{l \in {\mathcal {H}}(k)} \overline{T}^{k_0}_l(t)\right) = 1$. $\square$

Now we use the property of fluid-SSC to prove that the scaled high-priority queue length uniformly converges to zero.

Lemma A.1

Assume the sequence of fluid models of MCN under SBP policy has fluid-SSC and let $t^*$ be given as in Definition 3.6. Then, for every $\omega$ that Lemma 3.5 holds,

$$\begin{aligned} \lim _{|x|_h\rightarrow \infty } \sup _{n} \frac{1}{|x|_h} \sum _{k \in {\mathcal {H}}} Q_k^{n,x}(t^*|x|_h) = 0. \end{aligned}$$

Proof

Fix an $\omega _0$ satisfying Lemma 3.5. Suppose the contrary. Then, there is some fixed $\epsilon >0$ so that for all $n=1,2,\ldots$, there exists $|x_n|>n$ and network index $k_n$ satisfying

$$\begin{aligned} \frac{ \sum _{k \in {\mathcal {H}}} Q_k^{k_n, x_n}(t^*|x_n|_h)}{|x_n|_h} > \epsilon . \end{aligned}$$

We might as well assume that $k_n \rightarrow k_0$ where $k_0$ is allowed to be $\infty$. By Lemma 3.5, for sample path $\omega _0$, there is a subsequence $\{x_{n_j}\}$ so that

$$\begin{aligned} \lim _{n_j \rightarrow \infty } \frac{ \sum _{k \in {\mathcal {H}}} Q_k^{k_{n_j},x_{n_j}}(t^*|x_{n_j}|_h)}{|x_{n_j}|_h} = \sum _{k \in {\mathcal {H}}} \overline{Q}^{k_0}_k(t^*)> \epsilon \end{aligned}$$

where $\overline{Q}^{k_0}_k(\cdot )$ satisfies (3.8)–(3.13), contradiction to the assumption of fluid-SSC. $\square$

We need the following lemma about the counting processes to prove the $L^p$-contraction property. See [8, 12].

Lemma A.2

(Theorem 5.1 [12]) Let $\{\zeta (k): k \in \mathbb {N}\}$ be an i.i.d. sequence of random variables taking value in $(0,\infty )$. Let E(t) denote the counting process $E(t) = \max \{ n\ge 1: \zeta (1)+ \cdots + \zeta (n-1) \le t\}$. Suppose $\mathbb {E}[\zeta (1)] < \infty$. Then, for any integer $r \ge 1$, we have the following:

• (1) $\lim _{t \rightarrow \infty } \mathbb {E}\left[ (E(t)/t)^r \right] = (1/\mathbb {E}[\zeta (1)])^r$.

• (2) For any $\delta >0$, $\sup _{t \ge \delta } \mathbb {E}[(E(t)/t)^r] < \infty$.

• (3) The random variables $\{(E(t)/t)^r: t \ge 1\}$ are uniformly integrable.

Now we prove Proposition 6.1,6.2 and 6.3. Those results are parallel to Proposition 5.1, 5.3 and 5.4 from [8].

Proof of Proposition 6.1

Since this argument is similar to Proposition 5.1 from [8], we will provide an outline and omit the details.

Define

$$\begin{aligned} \alpha ^* = \sup _n \alpha ^n, \quad \mu ^* = \sup _n \mu ^n \end{aligned}$$

where the sup is taken component-wise. By (2.9), both $\alpha ^*$ and $\mu ^*$ are finite. For the rest of this proof, we denote $|\cdot |_h$ by $|\cdot |$. Assume $t^*$ is the constant used in Definition 3.6.

Notice that

$$\begin{aligned} E_k^{n, 0}(|x| t^*)&= \max \{ i \ge 1: u_k^n(1) + \cdots + u_k^n(i-1) \le |x| t^* \} \nonumber \\&\le \max \{ i \ge 1: u_k(1) + \cdots + u_k(i-1) \le \alpha _k^* |x| t^* \} \nonumber \\&= E^0_k( \alpha _k^* |x| t^*) \end{aligned}$$

(A.5)

where $E_k^0(\cdot )$ is defined in (A.4). Similarly, we have

$$\begin{aligned} S_k^{n, 0}(|x| t^*) \le S_k^0(\mu _k^* |x| t^*) \end{aligned}$$

(A.6)

where $S_k^0(t) = \max \{ i \ge 1: v_k(1) + \cdots + v_k(i-1) \le t\}$.

We first make a useful observation: the total high-priority queue length may be upper bounded by the summation of their initial length, the number of external arrivals to high-priority classes and the number of service completions from low-priority classes. In particular, for all $n \in \mathbb {N}$ and $x \in {\mathcal {X}}$,

$$\begin{aligned} \sum _{k \in {\mathcal {H}}} Q^{n,x}_k(t) \le \sum _{k \in {\mathcal {H}}} Q^{n,x}_k(0) + \sum _{k \in {\mathcal {H}}} E_k^{n,0}(t) + \sum _{l \in {\mathcal {L}}} S_l^{n,0}(t). \end{aligned}$$

(A.7)

Thus, by (A.5), (A.6) and (A.7), we have

$$\begin{aligned} \sup _n \frac{1}{|x|} \sum _{k \in {\mathcal {H}}} Q_k^{n,x}(|x|t^*)&\le 1+ \sup _n \frac{1}{|x|} \sum _{k \in {\mathcal {H}}} E_k^{n,0} (|x|t^*) + \sup _n \frac{1}{|x|} \sum _{l \in {\mathcal {L}}} S_l^{n,0}(|x|t^*) \nonumber \\&\le 1+ \frac{1}{|x|} \sum _{k \in {\mathcal {H}}} E_k^{0} (\alpha _k^* |x| t^*) + \frac{1}{|x|} \sum _{l \in {\mathcal {L}}} S_l^{0}(\mu _l^* |x|t^*). \end{aligned}$$

(A.8)

By Lemma A.2, since each term indexed by |x| in (A.8) is uniformly integrable , the collection of random variables

$$\begin{aligned} \left\{ \sup _{n} \frac{1}{ |x|^{p+1}}\left[ \sum _{k \in {\mathcal {H}}} Q_k^{n,x}(|x|t^*)\right] ^{p+1}: |x| \ge 1 \right\} \end{aligned}$$

is uniformly integrable. By Lemma 3.5 and fluid-SSC, it follows that

$$\begin{aligned} \lim _{|x|\rightarrow \infty } \sup _{n} \frac{1}{|x|} \sum _{k \in {\mathcal {H}}} Q_k^{n,x}(t^*|x|) = 0. \end{aligned}$$

Therefore, it follows that

$$\begin{aligned} \lim _{|x| \rightarrow \infty } \frac{1}{|x|^{p+1}} \sup _{n} \mathbb {E}\left[ \frac{|Q^{n, x}(t^*|x|)|^{p+1}}{|x|^{p+1}} \right] = 0. \end{aligned}$$

The proofs for

$$\begin{aligned}&\lim _{|x| \rightarrow \infty } \frac{1}{|x|^{p+1}} \sup _{n} \mathbb {E}\left[ \frac{|U^{n,x}(t^*|x|)|^{p+1}}{|x|^{p+1}} \right] = 0, \\&\quad \lim _{|x| \rightarrow \infty } \frac{1}{|x|^{p+1}} \sup _{n} \mathbb {E}\left[ \frac{|V^{n,x}(t^*|x|)|^{p+1}}{|x|^{p+1}} \right] = 0. \end{aligned}$$

is similar to those in [8] so we omit the details. $\square$

1.2 A.2 uniform bound on the mean return time

Recall the return time of the Markov process $\{X^n(t), t \ge 0\}$

$$\begin{aligned} \tau ^n_C(\delta ) = \inf \{ t \ge \delta : X^n(t) \in C\}. \end{aligned}$$

Now we prove Proposition 6.2.

Proof of Proposition 6.2

In this proof, fixed constant values are denoted by $c_i$’s, which are all independent of n. For the rest of this proof, we denote $|\cdot |_h$ by $|\cdot |$.

By Proposition 6.1, there exists a subset $C = \{ x \in \mathbb {Z}_+^K \times \mathbb {R}_+^{2K}: |x| \le L\}$ so that for each $x \in C^c$ and each network index n,

$$\begin{aligned} \mathbb {E}_x\left[ 1+|X^{n}(t^*|x|)|^{p+1} \right] \le \frac{1}{2}(1+|x|^{p+1}). \end{aligned}$$

(A.9)

Define

$$\begin{aligned} t(x)&= t^* \max \{L, |x|\}, \\ b&= \sup _{n, |x| \le L} \mathbb {E}_x\left[ 1+|X^{n}(t^*L)|^{p+1} \right] . \end{aligned}$$

Notice that b is finite and well-defined since, by (A.7),

$$\begin{aligned} |X^{n, x}(t^*L)|&= \sum _{k \in {\mathcal {H}}} Q_k^{n,x}(t^* L) + \sum _{k \in {\mathcal {K}}} U_k^{n,x}(t^*L) + \sum _{k \in {\mathcal {K}}} V_k^{n,x}(t^*L) \\&\le L +\sum _{k \in {\mathcal {H}}} E_k^{n,0}(t^*L) + \sum _{l \in {\mathcal {L}}} S_l^{n,0}(t^*L) \\& + \sum _{ k \in {\mathcal {K}}} u_k^n(E_k^{n,x}(t^*L) + 1))+ \sum _{ k \in {\mathcal {K}}} v_k^n(S_k^{n,x}(t^*L) + 1)) . \end{aligned}$$

Focus on the right-hand side of the above inequality. The expected value of the first line is finite due to Lemma A.2. The expected value of the second line is finite since, for each $k \in {\mathcal {K}}$,

$$\begin{aligned} \mathbb {E}\left[ \sum _{i=1}^{E_k^{n,x}(t)+1} u^n_k(i) \right] \le \mathbb {E}\left[ E_k^{n,0}(t)+1 \right] \mathbb {E}[ u^n_k(i) ] \end{aligned}$$

by Wald’s identity. Thus, we may write (A.9) as

$$\begin{aligned} \mathbb {E}_x\left[ 1+|X^{n}(t(x))|^{p+1} \right] \le \frac{1}{2}(1+|x|^{p+1}) + b1_{C}(x), \ x \in {\mathcal {X}}. \end{aligned}$$

(A.10)

Consider the sequence of stopping times given by

$$\begin{aligned} \sigma _0 = 0, \sigma _1 = t(X^n(\sigma _0)), \sigma _{k+1} = \sigma _k + t(X^n(\sigma _k)), \ \ k \ge 1. \end{aligned}$$

By the strong Markov property, the stochastic process

$$\begin{aligned} \hat{X}_k^{n} = X^{n}(\sigma _k), \ k \ge 0 \end{aligned}$$

is a Markov chain with transition kernel

$$\begin{aligned} \hat{P}^{n}(x, A) = \mathbb {P}_x(X^{n}(t(x)) \in A) \end{aligned}$$

for each $x \in {\mathcal {X}}$ and $A \in {\mathcal {B}}({\mathcal {X}})$. Thus, we can write the drift condition (A.10) as

$$\begin{aligned}&\int _{ \mathbb {Z}_+^K \times \mathbb {R}_+^{2K}} (1+|y|^{p+1}) \hat{P}^{n}(x, dy) \le 1+|x|^{p+1} - \frac{1}{2}(1+|x|^{p+1}) \\&\quad + b1_C(x), \ x \in \mathbb {Z}_+^K \times \mathbb {R}_+^{2K}. \end{aligned}$$

It follows from the Comparison Theorem cf. Theorem 14.2.2 [14] that

$$\begin{aligned}&\mathbb {E}_x \left[ \sum _{k=0}^{k_*-1}\left( 1+|X^{n}(\sigma _k)|^{p+1}\right) \right] =\mathbb {E}_x \left[ \sum _{k=0}^{k_*-1} \left( 1+|\hat{X}_k^{n}|^{p+1} \right) \right] \nonumber \\&\quad \le 2\left( 1+|x|^{p+1}+b1_C(x) \right) , \ x \in {\mathcal {X}} \end{aligned}$$

(A.11)

where $k_* = \inf \left\{ k \ge 1: \hat{X}^{n}_k \in C \right\}$ is a stopping time with respect to filtration $\{{\mathcal {F}}_{\sigma _k}, k =0, 1, \ldots \}$.

Now we use (A.11) to derive the bound (6.1). First, we set $\delta = t^*L$ and observe that

$$\begin{aligned} \mathbb {E}_x \left[ \int _0^{\tau ^n_C(\delta )} (1+|X^{n}(t)|^p) \, dt \right] \le \mathbb {E}_x \left[ \int _0^{\sigma _{k_*}} (1+|X^{n}(t)|^p) \, dt \right] , \ x \in {\mathcal {X}} \end{aligned}$$

(A.12)

since $\tau ^n_C(t^*L) \le \sigma _{k_*}$. Furthermore, we may rewrite the right-hand side above as

$$\begin{aligned} \mathbb {E}_x \left[ \int _0^{\sigma _{k_*}} (1+|X^{n}(t)|^p) \,dt \right]&= \mathbb {E}_x\left[ \mathbb {E}\left[ \sum _{k=0}^{k_*-1} \int _{\sigma _k}^{\sigma _{k+1}}(1+|X^{n}(t)|^{p}) \,dt \big | {\mathcal {F}}_{\sigma _k}\right] \right] \nonumber \\&= \mathbb {E}_x\left[ \sum _{k=0}^{k_*-1} \mathbb {E}\left[ \int _{\sigma _k}^{\sigma _{k+1}}(1+|X^{n}(t)|^{p}) \,dt \big | {\mathcal {F}}_{\sigma _k}\right] \right] \end{aligned}$$

(A.13)

where the first equality follows from smoothing property of conditional expectations and the second equality follows from Fubini’s theorem.

Next, we show that for each $k \ge 0$,

$$\begin{aligned} \mathbb {E}\left[ \int _{\sigma _k}^{\sigma _{k+1}}(1+|X^{n}(t)|^{p}) \,dt \big | {\mathcal {F}}_{\sigma _k}\right] \le c_0(|X^n(\sigma _k)|^{p+1}+1) \end{aligned}$$

(A.14)

where $c_0$ is some constant. By the strong Markov property, it is enough to show that for all $x \in \mathbb {Z}_+^K \times \mathbb {R}_+^{2K}$,

$$\begin{aligned} \mathbb {E}_x \left[ \int _{0}^{\sigma _{1}}(1+|X^{n}(t)|^{p}) \,dt\right] \le c_0(|x|^{p+1}+1). \end{aligned}$$

(A.15)

Recall that $|X^{n,x}(t)| = \sum _{k \in {\mathcal {H}}} Q_k^{n,x}(t) + \sum _{k \in {\mathcal {K}}} U_k^{n,x}(t)+ \sum _{k \in {\mathcal {K}}} V_k^{n,x}(t)$. For the rest of the proof, we will derive bounds on $\mathbb {E}_x\left[ \left( U_k^{n,x}(t)\right) ^p\right]$, $\mathbb {E}_x\left[ \left( V_k^{n,x}(t)\right) ^p\right]$ and $\mathbb {E}_x\left[ \left| Q^{n,x}(t)\right| ^p\right]$, one term at a time.

Fix an arbitrary initial condition $x \in {\mathcal {X}}$. Notice that

$$\begin{aligned} (U_k^{n,x}(t))^p \le |x|^p + \sum _{i=1}^{E_k^{n,0}(t)+1} (u^n_k(i))^p \end{aligned}$$

and hence by Wald’s identity and Lemma A.2,

$$\begin{aligned} \mathbb {E}_x\left[ \left( U_i^{n,x}(t)\right) ^p\right]&\le |x|^p + \mathbb {E}\left[ \sum _{i=1}^{E_k^{n,0}(t)+1} (u^n_k(i))^p \right] \\&= |x|^p + \mathbb {E}\left[ E_k^{n,0}(t)+1 \right] \mathbb {E}[ (u^n_k(i))^p ] \\&\le |x|^p + \mathbb {E}\left[ E_k^{0}(\alpha ^*t)+1 \right] \mathbb {E}[ (u^n_k(i))^p ] \\&\le |x|^p+c_1(t+1), \end{aligned}$$

where the last inequality follows from applying (2) in Lemma A.2 with $r=1$ and (2.4).

Recall that $\sigma _1 = \max \{L, |x|\}$. Therefore, it follows that

$$\begin{aligned} \mathbb {E}_x\left[ \int _0^{\sigma _1} \left( U_i^{n,x}(t)\right) ^p \, dt\right] \le \sigma _1|x|^p+c_1(\sigma _1^2/2+\sigma _1) \le c_2(|x|^{p+1}+1). \end{aligned}$$

(A.16)

Similarly, we may show that

$$\begin{aligned} \mathbb {E}_x\left[ \int _0^{\sigma _1} \left( V_k^{n,x}(t)\right) ^p \, dt\right] \le c_3(|x|^{p+1}+1). \end{aligned}$$

(A.17)

Now we consider $\mathbb {E}_x\left[ \left| Q^{n,x}(t)\right| ^p\right]$. Recall from (A.7) that

$$\begin{aligned} \left| Q^{n,x}(t)\right| \le \sum _{k \in {\mathcal {H}}} Q_k^{n,x}(0) + \sum _{k \in {\mathcal {H}}} E_k^{n,x} (t) + \sum _{l \in {\mathcal {L}}} S_l^{n,x}(t). \end{aligned}$$

By Lemma A.2, we have

$$\begin{aligned} \mathbb {E}_x\left[ \left( S_l^{n,x}(t)\right) ^p \right]&\le \mathbb {E}\left[ \left( S_l^{n,0}(t)\right) ^p \right] \le c_4(t^p+1), \ l \in {\mathcal {L}} \\ \mathbb {E}_x\left[ \left( E_k^{n,x}(t)\right) ^p \right]&\le \mathbb {E}\left[ \left( E_k^{n,0}(t)\right) ^p \right] \le c_5(t^p+1), \ k \in {\mathcal {K}} \end{aligned}$$

for all $t \ge 0$. Thus, we may conclude from (A.18) that

$$\begin{aligned} \mathbb {E}_x\left[ \int _0^{\sigma _1} \left| Q^{n,x}(t)\right| ^p \, dt\right] \le c_6 \sigma _1 (|Q^{n,x}(0)|^p+\sigma _1^p) \le c_7(|x|^{p+1}+1). \end{aligned}$$

(A.18)

Combining (A.16), (A.17) and (A.18), we see that (A.15) follows. Recall that (A.15) implies (A.14), which we substitute into (A.13) to get

$$\begin{aligned} \mathbb {E}_x\left[ \sum _{k=0}^{k_*-1} \mathbb {E}\left[ \int _{\sigma _k}^{\sigma _{k+1}}(1+|X^{n}(t)|^{p}) \,dt \big | {\mathcal {F}}_{\sigma _k}\right] \right]&\le c_0 \mathbb {E}_x\left[ \sum _{k=0}^{k_*-1} (|X^n(\sigma _k)|^{p+1}+1)\right] \\&\le c_8(|x|^{p+1}+1) \end{aligned}$$

where the last inequality is due to (A.11). Hence, by using the upper bound (A.12), we have established (6.1). $\square$

1.3 A.3 Proof of Proposition 6.3

This proof follows from Proposition 5.4 in [8]. We first show by induction that for $b = \frac{1}{\delta } \sup _{x \in C} \sup _n V^n(x)$, each positive integer $r \ge 1$ and $x \in {\mathcal {X}}$,

$$\begin{aligned} \mathbb {E}_x\left[ \int _0^{\tau ^n_C(r\delta )} f(X^n(t)) \,dt\right] \le V^n(x) + b r \delta . \end{aligned}$$

(A.19)

For $r=1$, equation (A.19) is true by definition. Assume, for $r = k$, (A.19) is true. Then, it follows from the strong Markov property (2.6) that

$$\begin{aligned}&\mathbb {E}_x\left[ \int _0^{\tau ^n_C((k+1)\delta )} f(X^n(t)) \,dt\right] \\&\le \mathbb {E}_x\left[ \int _0^{\tau ^n_C(\delta )} f(X^n(t)) \,dt\right] + \mathbb {E}_x\left[ \int _{\tau ^n_C(\delta )}^{\tau ^n_C(\delta ) +\theta _{\tau ^n_C(\delta )}\tau ^n_C(k\delta ) } f(X^n(t)) \,dt\right] \\&= V^n(x) + \mathbb {E}_x\left[ \mathbb {E}_{X^{n}(\tau ^n_C(\delta ))} \left[ \int _{0}^{\tau ^n_C(k\delta ) } f(X^n(t)) \,dt\right] \right] \\&\le V^n(x) + \sup _{x \in C} \mathbb {E}_x \left[ \int _{0}^{\tau ^n_C(k\delta ) } f(X^n(t)) \,dt\right] \\&\le V^n(x) + \sup _{x \in C} \sup _n V^n(x) + bk\delta \\&= V^n(x)+ b(k+1)\delta . \end{aligned}$$

Hence, we have established (A.19). For any $t \ge \delta$, there exists $M \ge 1$ so that $M\delta \le t \le (M+1) \delta$ and hence

$$\begin{aligned} \mathbb {E}_x\left[ \int _0^{\tau ^n_C(t)} f(X^n(s)) \,ds\right] \le V^n(x) + b(M+1)\delta \le V^n(x)+2bt. \end{aligned}$$

(A.20)

Moreover, (A.20) holds for $0 \le t <\delta$ by definition and thus (A.20) holds for all $t \ge 0$.

Following the proof of Proposition 5.4 from [8], since for all $t \ge 0$, $t+\theta _t \tau ^n_C(\delta ) \le \tau ^n_C(\delta ) + \theta _{\tau _C^n(\delta )}\tau _C^n(t)$, we combine the bound (A.20) and (2.6) to deduce that

$$\begin{aligned} \mathbb {E}_x[V^n(X^n(t))]&= \mathbb {E}_x\left[ \int _t^{t+\theta _t \tau ^n_C(\delta )} f(X^n(s)) ds\right] \\&\le \mathbb {E}_x\left[ \int _t^{\tau ^n_C(\delta ) + \theta _{\tau _C^n(\delta )}\tau _C^n(t)} f(X^n(s)) ds\right] \\&= V^n(x) - \int _0^t \mathbb {E}_x[f(X^n(s)] \,ds+ \mathbb {E}_x\left[ \int _{\tau ^n_C(\delta )}^{\tau ^n_C(\delta ) + \theta _{\tau _C^n(\delta )}\tau _C^n(t)} f(X^n(u)) du\right] \\&\le V^n(x) - \int _0^t \mathbb {E}_x[f(X^n(s)] \,ds+ \sup _{x \in C} \mathbb {E}_{x} \left[ \int _0^{\tau ^n_C(t)}f(X^n(s)) \,ds\right] \\&\le V^n(x) - \int _0^t \mathbb {E}_x[f(X^n(s)] \,ds+ \sup _{x \in C} \sup _n V^n(x) +2bt \\&= V^n(x) - \int _0^t \mathbb {E}_x[f(X^n(s)] \,ds+ b\delta +2bt. \end{aligned}$$

Dividing the above inequality by $t \ge 1$ completes the proof of Proposition 6.3 with $\kappa = b(\delta +2)$.

1.4 A.4 some Lemmas for LBFS reentrant lines

We list some properties of blocks that directly follow from the definitions of blocks and reentrant lines.

Lemma A.3

The blocks have the following properties:

• 1. The blocks form a partition of the set of classes, i.e., $\bigcup _{j \in {\mathcal {J}}} B_j = {\mathcal {K}}$.

• 2. Suppose $k_1 \in B_{j_1}$ and $k_2 \in B_{j_2}$. If $j_1 < j_2$, then $k_1 < k_2$.

• 3. For $k \in {\mathcal {C}}(j) \setminus B_j$, k must be contained in $\cup _{j' >j} B_{j'}$.

Lemma A.4

Fix a class $k \in {\mathcal {K}}$ with $W_k(t)>0$ and $T_k(t) \ge V_k$. Suppose that for all $k' >k$, $W_{k'}(t) =0$. Then, $d_l(t) = \frac{1}{m^+_k}$ for all $l \ge k$.

Proof

Since for all $k' >k$, $Q_{k'}(t) =0$, we have

$$\begin{aligned} d_k(t) = d_{k+1}(t) = \ldots = d_K(t) >0. \end{aligned}$$

It follows that

$$\begin{aligned} \sum _{i \in {\mathcal {H}}(k)} m_id_i(t) = 1 \end{aligned}$$

which implies that $d_l(t) = \frac{1}{m^+_k}$ for all $l \ge k$. $\square$

Lemma A.5

Suppose $k_0 \in B_j$ is the largest high-priority buffer so that $W_{k_0}(t) > 0$. Assume that all classes after $k_0$, including $k_0$, are undelayed, i.e.,

• For all $j'>j$, $W_k(t') = 0, t' \ge t$ for all $k \in B_{j'}^H$.

• For all $k \ge k_0$, $T_k(t) \ge V_k$.

Then, it follows that

• 1. $d_{k_0}(t)\ge d$ where $d>\alpha _1$ is a fixed number only depending on $\alpha$ and m.

• 2. For any $k \in {\mathcal {C}}(j) \setminus B_j$, $d_k(t) \ge \alpha _1$.

Proof

First, since all the classes k we are dealing with satisfy $k \ge k_0$ and every class larger than $k_0$ is undelayed, we have $W_k(t) = Q_k(t)$ and hence we will exclusively work with Q(t). If $j = J$, then (1) and (2) both hold. Suppose $j <J$. We now claim that there exists $l > l_j$ so that $Q_{l}(t) >0$. Suppose the contrary. Then, for all $k>k_0$, we have $Q_k(t) = 0$. It follows from Lemma A.4 that

$$\begin{aligned} d_{k_0}(t) = \frac{1}{m^+_{k_0}}. \end{aligned}$$

On the other hand, for all $k \in {\mathcal {C}}(j+1)$, we have $d_k(t) = \frac{1}{m^+_{k_0}}$. This implies that

$$\begin{aligned} \sum _{k \in {\mathcal {C}}(j+1)} m_kd_k(t) = \alpha _1^{-1} \times \frac{1}{m^+_{k_0}} >1, \end{aligned}$$

a contradiction since $k_0 \in {\mathcal {H}}$.

Let $l_m$ be the largest low-priority class such that $Q_{l_m}(t) >0$. First, notice that

$$\begin{aligned} {\mathcal {C}}(j) \setminus B_j = \bigcup _{j'>j} \left( {\mathcal {C}}(j) \cap B_{j'} \right) , \end{aligned}$$

meaning that all the classes at station j not belong to block $B_j$ come from blocks after $B_j$. This fact will be repetitively used when we compute $d_k(t)$ later. By Lemma A.3, it is sufficient to show, for the second statement of lemma, that $d_k(t) \ge \alpha _1$ for each $k \in B^H_{j'}, j' > j$. First, since for all $k > l_m$, $Q_k(t) = 0$, we have $d_k(t) = d_{l_m}(t)$ for all $k \ge l_m$. It follows that, for all $k \ge l_m$,

$$\begin{aligned} d_k(t) = \frac{1}{m^+_{l_m}} = \alpha _1. \end{aligned}$$

Next, we consider two cases:

• 1. The second largest class $k<l_m$ with $Q_k(t) >0$ is $k_0$. Then, for all $k_0 \le k < l_m$,

  $$\begin{aligned} d_{k_0}(t) = d_k(t). \end{aligned}$$

  Consider station j. We have

  $$\begin{aligned} \sum _{k \in {\mathcal {C}}(j), k_0 \le k < l_m} m_kd_k(t) + \sum _{ k \in {\mathcal {C}}(j), k >l_m} m_kd_k(t) = 1, \end{aligned}$$

  which implies that

  $$\begin{aligned} d_{k_0}(t) \times \left( \sum _{k \in {\mathcal {C}}(j), k_0 \le k < l_m} m_k\right) + \alpha _1 \left( \sum _{ k \in {\mathcal {C}}(j), k >l_m} m_k\right) = 1 \end{aligned}$$

  Define

  $$\begin{aligned} d = \min _{k_0} \frac{1- \alpha _1 \left( \sum _{ k \in {\mathcal {C}}(j), k >l_m} m_k\right) }{ \sum _{k \in {\mathcal {C}}(j), k_0 \le k < l_m} m_k}. \end{aligned}$$

  (A.21)

  Therefore, we have $d_{k_0}(t) = d > \alpha _1$. And hence $d_k(t) \ge \alpha _1$ for all $k \in {\mathcal {C}}(j), k_0 \le k < l_m$. Hence, the lemma is proved under this case.

• 2. The second largest class k so that $Q_k(t) >0$ is some low-priority class l with $k_0<l<l_m$. Then, for all $l< k <l_m$, $d_l(t) = d_k(t)$. Focus on station $\tilde{j} = s(l)$. Note that ${\mathcal {C}}(\tilde{j}) = \left( {\mathcal {C}}(\tilde{j}) \cap \{k < l_m\} \right) \cup \left( {\mathcal {C}}(\tilde{j})\cap \{k>l_m \} \right)$. By (3.13), it follows that

  $$\begin{aligned} \sum _{k \in {\mathcal {C}}(\tilde{j}) \cap \{k < l_m\}} m_kd_k(t) + \sum _{k \in {\mathcal {C}}(\tilde{j}) \cap \{k > l_m\}}m_kd_k(t) = 1, \end{aligned}$$

  which implies

  $$\begin{aligned} d_l(t) \times \left( \sum _{k \in {\mathcal {C}}(\tilde{j}) \cap \{k < l_m\}} m_k\right) + \alpha _1 \times \left( \sum _{k \in {\mathcal {C}}(\tilde{j}) \cap \{k > l_m\}}m_k\right) = 1, \end{aligned}$$

  which implies that $d_l(t) = \alpha _1$. Moreover, $d_k(t) = \alpha _1$ for all $k \ge l$. Under this case, we have proved the lemma for all $k \ge l$. From here, we may consider the third largest k so that $Q_k(t)>0$. By repeatedly applying the above argument, we iterate through a sequence of low-priority classes $\{l_m> l_1> \ldots > l_r\}$ where $l_r >k_0$ is the smallest nonempty low-priority class after $k_0$ so that $d_k(t) = \alpha _1$ for all $k \ge l_r$. From here, since $k_0$ is the largest nonempty class before $l_r$, the lemma is proved by applying the result from case (1).

$\square$