Stability and delay of distributed scheduling algorithms for networks of conflicting queues

Abstract

This paper explains recent results on distributed algorithms for networks of conflicting queues. At any given time, only specific subsets of queues can be served simultaneously. The challenge is to select the subsets in a distributed way to stabilize the queues whenever the arrival rates are feasible.

One key idea is to formulate the subset selection as an optimization problem where the objective function includes the entropy of the distribution of the selected subsets. The dual algorithm for solving this optimization problem provides a distributed scheduling algorithm that requires only local queue-length information. The algorithm is based on the CSMA (Carrier Sense Multiple Access) protocol in wireless networks.

We also explain recent results, some of them unpublished so far, on the delay properties of these algorithms. In particular, we present a framework for queuing stability under bounded CSMA parameters, and show how the expected queue lengths depend on the throughput region to be supported. When the arrival rates are within a fraction of the capacity region, queue lengths that are polynomial (or even logarithmic) in the number of queues can be achieved.

Appendices

Appendix A: Proof of Theorem 2

Proof

The dynamics of r[j], by (12) and (11), is

$$r_{k}[j+1]= \bigl\{r_{k}[j]+\alpha\bigl(\hat{\lambda}_{k}[j]-\hat{s}_{k}[j] \bigr) \bigr \}_{+}, \quad \forall k.$$

Define $d[j]:=\frac{1}{2}\|\mathbf{r}[j]-\mathbf{r}^{*}\|^{2}\ge0$. Note that r ^∗≥0 element-wise. Then

where the last step follows from the fact that $\hat{\lambda }_{k}[j]\le1,\ \forall k,j$ since the arrivals are assumed Bernoulli. Therefore,

$$E_{j} \bigl\{d[j+1] \bigr\}-d[j]\le\alpha\bigl(\boldsymbol {\lambda}-\mathbf{s} \bigl(\mathbf{r}[j] \bigr) \bigr)^{T} \bigl(\mathbf{r}[j]-\mathbf{r}^{*} \bigr)+\frac{1}{2}\alpha^{2}K,$$

where we have used the assumptions of Theorem 2.

Since the dual function g(r) is convex and s(r[j])−λ is a subgradient at the point r[j], we have

$$\bigl(\mathbf{s} \bigl(\mathbf{r}[j] \bigr)-\boldsymbol{\lambda}\bigr)^{T} \bigl(\mathbf{r}^{*}-\mathbf{r}[j] \bigr)\le g\bigl(\mathbf{r}^{*} \bigr)-g \bigl(\mathbf{r}[j] \bigr).$$

Define the region

$$\mathcal{A}_{\alpha K}:= \bigl\{\mathbf{r}\ge\mathbf{0}|g(\mathbf {r})\le g\bigl(\mathbf{r}^{*} \bigr)+\alpha K \bigr\}.$$

We will show later that $\mathcal{A}_{\alpha K}$ is bounded.

Then, if $\mathbf{r}[j]\notin\mathcal{A}_{\alpha K}$, we have g(r[j])>g(r ^∗)+αK, and therefore

(21)

Also,

(22)

By (21) and (22), we conclude that the set $\mathcal{A}_{\alpha K}$ is recurrent for r[j]. This establish the stability of r[j].³ Finally, since r[j] and the queue lengths X(t) are proportional by (12), the queues are also stable.

It remains to be shown that the set $\mathcal{A}_{\alpha K}$ is bounded. Since λ is strictly feasible, there exist δ>0 and a distribution $\tilde{\mathbf{u}}$ such that $\sum_{\sigma }\tilde{u}_{\sigma}\sigma_{k}=\lambda_{k}+\delta,\ \forall k$. So, $\mathcal{L}(\tilde{\mathbf{u}};\mathbf{r})=-\sum_{\sigma\in \varOmega}[\tilde{u}_{\sigma}\log(\tilde{u}_{\sigma})]+\sum _{k}[r_{k}(\sum_{\sigma}\tilde{u}_{\sigma}\sigma_{k}-\lambda _{k})]=-\sum_{\sigma\in\varOmega}[\tilde{u}_{\sigma}\log(\tilde {u}_{\sigma})]+\delta\sum_{k}r_{k}\ge\delta\sum_{k}r_{k}$. By (9), we have $g(\mathbf{r})\ge\mathcal {L}(\tilde{\mathbf{u}};\mathbf{r})\ge\delta\sum_{k}r_{k}$. Therefore, any $\mathbf{r}\in\mathcal{A}_{\alpha K}$ must satisfy that $\sum_{k}r_{k}\le\frac{\alpha K+g(\mathbf{r}^{*})}{\delta}$ and r _k ≥0, ∀k. This proves that $\mathcal{A}_{\alpha K}$ is bounded. □

Appendix B: Proof of Lemma 2

Proof

In view of the queue dynamics (11), at time instance (j+1)T, $\bar{\mathbf{r}}$ is updated as

$$\bar{r}_{k}[j+1]=\max\bigl\{\bar{r}_{k}[j]+\alpha\cdot \bigl[\hat{\lambda}_{k}[j]-\hat{s}_{k}[j] \bigr],r_{\min} \bigr\}, \quad \forall k. $$

(23)

We use E _j [⋅] to denote the conditional expectation E[⋅|X[j],σ[j]] where σ[j]:=σ(jT) is the transmission schedule just before time instance jT. And Pr _j [⋅] denotes the conditional probability Pr{⋅|X[j],σ[j]}.

Now we compute $E_{j}[\hat{s}_{k}[j]]$ for every link k:

where μ _σ[j],τ is the distribution of the CSMA Markov chain after τ slots if the Markov chain starts with schedule σ[j]. Remember that $s_{k}(\mathbf{r}) =\sum_{\sigma\in\varOmega:\sigma_{k}=1}\pi(\sigma;\mathbf{r})$, so

Therefore, by (16), we have

$$\bigl|E_{j} \bigl[\hat{s}_{k}[j] \bigr]-s_{k} \bigl(\mathbf{r}[j] \bigr)\bigr|\le\delta/(4K\cdot D),\quad \forall k,j. $$

(24)

Next we show that $\bar{\mathbf{r}}[j]$ is stable. Define the Lyapunov function

$$L(\bar{\mathbf{r}}):=\sum_{k}L_{k}(\bar{r}_{k})$$

where

Note that the minimal value of $L(\bar{\mathbf{r}})$ is achieved at the point $\tilde{\mathbf{r}}$. (So $\tilde{\mathbf{r}}$ serves as the “attraction point” of our algorithm.) Unlike the usual quadratic Lyapunov function used in Sects. 2.2 and 2.3, this Lyapunov function is partially quadratic and partially linear.

We have

(25)

where $r_{k}=\min\{\bar{r}_{k},r_{\max}\}$ as defined.

Next we prove the following useful inequality:

$$L_{k} \bigl(\bar{r}_{k}[j+1] \bigr)-L_{k}\bigl( \bar{r}_{k}[j] \bigr)\le\alpha\bigl(\hat{\lambda}_{k}[j]-\hat{s}_{k}[j] \bigr)L_{k}^{\prime} \bigl(\bar{r}_{k}[j] \bigr)+\alpha^{2}/2. $$

(26)

For this purpose, first note that $L_{k}(\bar{r}_{k}[j+1])\le L_{k}(\hat{\bar{r}}_{k}[j+1])$ where $\hat{\bar{r}}_{k}[j+1]:=\bar{r}_{k}[j]+\alpha\cdot[\hat {\lambda}_{k}[j]-\hat{s}_{k}[j]]$. (This is clearly true if $\hat{\bar{r}}_{k}[j+1]\ge r_{\min}$. If $\hat{\bar{r}}_{k}[j+1]<r_{\min}$, then $L_{k}(\bar {r}_{k}[j+1])=L_{k}(r_{\min})\le L_{k}(\hat{\bar{r}}_{k}[j+1])$.) Therefore, we only need to prove

$$L_{k} \bigl(\hat{\bar{r}}_{k}[j+1] \bigr)-L_{k}\bigl(\bar{r}_{k}[j] \bigr)\le\alpha\bigl(\hat{\lambda}_{k}[j]-\hat{s}_{k}[j] \bigr)L_{k}^{\prime} \bigl(\bar{r}_{k}[j] \bigr)+\alpha^{2}/2. $$

(27)

Consider several cases:

• (i) If $\hat{\bar{r}}_{k}[j+1]$, $\bar {r}_{k}[j]\ge r_{\max}$, (27) is true.

• (ii) If $\hat{\bar {r}}_{k}[j+1]$, $\bar{r}_{k}[j]<r_{\max}$, then $L_{k}(\hat{\bar{r}}_{k}[j+1])-L_{k}(\bar{r}_{k}[j])=\frac {1}{2}(\hat{\bar{r}}_{k}[j+1]-\tilde{r}_{k})^{2}-\frac{1}{2}(\bar {r}_{k}[j]-\tilde{r}_{k})^{2}=\alpha\cdot(\hat{\lambda }_{k}[j]-\hat{s}_{k}[j])(\bar{r}_{k}[j]-\tilde{r}_{k})+\frac {1}{2}\alpha^{2}(\hat{\lambda}_{k}[j]-\hat{s}_{k}[j])^{2}$. Due to the assumption of Bernoulli arrivals, we have $|\hat{\lambda }_{k}[j]-\hat{s}_{k}[j]|\le1$. Then (27) follows.

• (iii) Assume that $\bar {r}_{k}[j]\ge r_{\max}$ and $\hat{\bar{r}}_{k}[j+1]<r_{\max}$. Then $L_{k}(r_{\max })-L_{k}(\bar{r}_{k}[j])=(r_{\max}-\tilde{r}_{k})(r_{\max}-\bar {r}_{k}[j])$, and similar to case (ii), $L_{k}(\hat{\bar {r}}_{k}[j+1])-L_{k}(r_{\max})\le(r_{\max}-\tilde{r}_{k})(\hat {\bar{r}}_{k}[j+1]-r_{\max})+\frac{1}{2}\alpha^{2}$. Therefore, $L_{k}(\hat{\bar{r}}_{k}[j+1])-L_{k}(\bar{r}_{k}[j])\le (r_{\max}-\tilde{r}_{k})(\hat{\bar{r}}_{k}[j+1]-\bar {r}_{k}[j])+\frac{1}{2}\alpha^{2}$, proving (27).

• (iv) Assume that $\bar {r}_{k}[j]<r_{\max}$ and $\hat{\bar{r}}_{k}[j+1]\ge r_{\max}$. Note that $L_{k}(\hat {\bar{r}}_{k}[j+1])\le\frac{1}{2}[(\hat{\bar{r}}_{k}[j+1]-\tilde {r}_{k})^{2}+(r_{\max}-\tilde{r}_{k})^{2}]$. So $L_{k}(\hat{\bar{r}}_{k}[j+1])-L_{k}(\bar{r}_{k}[j])\le\frac {1}{2}(\hat{\bar{r}}_{k}[j+\nobreak 1]-\tilde{r}_{k})^{2}-\frac{1}{2}(\bar {r}_{k}[j]-\tilde{r}_{k})^{2}\le\alpha(\hat{\lambda}_{k}[j]-\hat {s}_{k}[j])L_{k}^{\prime}(\bar{r}_{k}[j])+\alpha^{2}/2$ similar to case (ii). This completes the proof of (26).

Given a $\bar{\mathbf{r}}[j]\notin\mathcal{D}$, at least one element of r[j] is equal to r _max (i.e., r _k′[j]=r _max for some k′). Using (26), (25), (24), and (15), we obtain the following:

which establishes the negative drift of $L(\bar{\mathbf{r}}[j])$ if $\bar{\mathbf{r}}[j]\notin\mathcal{D}$. Also, it is clear that for any $\bar{\mathbf{r}}[j]\in\mathcal{D}$, we have Δ[j]<∞ (since $\bar{\mathbf{r}}[j+1]-\bar{\mathbf{r}}[j]$ is bounded). By the Foster–Lyapunov criteria, $\bar{\mathbf{r}}[j]$ is stable. By (14), X[j] is also stable.

The proof of (18) is similar to that of Theorem 8 in [27]. For completeness, we present the proof in the following.

Denote $\bar{L}:=\max_{\bar{\mathbf{r}}\in\mathcal{D}}L(\bar {\mathbf{r}})$. Then if $L(\bar{\mathbf{r}})>\bar{L}$, we have $\bar{\mathbf {r}}\notin\mathcal{D}$. Define

$$G(\bar{\mathbf{r}}):= \bigl[L(\bar{\mathbf{r}})-\bar{L} \bigr]_{+}.$$

Note that $|\bar{r}_{k}[j+1]-\bar{r}_{k}[j]|\le\alpha,\ \forall k,j$. Also, (25) implies that $|\frac{\partial L_{k}(\bar{r}_{k})}{\partial\bar{r}_{k}}|\le D$, $\forall\bar {r}_{k}\ge r_{\min}$. Therefore,

Consider the following two cases.

Case 1: If $L(\bar{\mathbf{r}}[j])-\bar{L}>0$, then $G(\bar{\mathbf {r}}[j])=L(\bar{\mathbf{r}}[j])-\bar{L}>0$, and $\bar{\mathbf{r}}[j]\notin\mathcal{D}$. Therefore,

Therefore,

Case 2: If $G(\bar{\mathbf{r}}[j])=0$, then $0\le G(\bar{\mathbf {r}}[j+1])\le c$. Therefore,

$$E_{j} \bigl[G^{2} \bigl(\bar{\mathbf{r}}[j+1]\bigr)-G^{2} \bigl(\bar{\mathbf{r}}[j] \bigr) \bigr]\le c^{2}=-G \bigl(\bar{\mathbf{r}}[j] \bigr)\alpha \delta/2+c^{2}.$$

Combining Cases 1 and 2, we have

$$E_{j} \bigl[G^{2} \bigl(\bar{\mathbf{r}}[j+1]\bigr)-G^{2} \bigl(\bar{\mathbf{r}}[j] \bigr) \bigr]\le-G \bigl (\bar{\mathbf{r}}[j] \bigr)\alpha\delta/2+c^{2}.$$

Taking expectations on both sides yields

$$E \bigl[G^{2} \bigl(\bar{\mathbf{r}}[j+1] \bigr)-G^{2}\bigl(\bar{\mathbf{r}}[j] \bigr) \bigr]\le-E \bigl[G \bigl(\bar {\mathbf{r}}[j]\bigr) \bigr]\alpha\delta/2+c^{2}.$$

Summing the above inequality from j=0 to j=J−1, and dividing both sides by J, we have

$$E \bigl[G^{2} \bigl(\bar{\mathbf{r}}[J] \bigr)-G^{2}\bigl( \bar{\mathbf{r}}[0] \bigr) \bigr]/J\le -(\alpha\delta/2)\sum_{j=0}^{J-1}E\bigl[G \bigl(\bar{\mathbf{r}}[j] \bigr) \bigr]/J+c^{2}.$$

Therefore,

$$\limsup_{J\rightarrow\infty}\sum_{j=0}^{J-1}E\bigl(G \bigl(\bar{\mathbf{r}}[j] \bigr) \bigr)/J\le\frac {c^{2}}{\alpha\delta/2}=2K\frac{c^{2}}{\delta^{2}}=2KD^{2}.$$

Note that

$$W(\bar{\mathbf{r}}):=\sum_{k}(r_{\max}-\tilde{r}_{k}) (\bar{r}_{k}-\tilde{r}_{k})\le L(\bar{\mathbf{r}})\le G(\bar{\mathbf{r}})+\bar{L}.$$

So

$$\limsup_{J\rightarrow\infty}\sum_{j=0}^{J-1}E\bigl(W \bigl(\bar{\mathbf{r}}[j] \bigr) \bigr)/J\le2KD^{2}+\bar{L}\le3KD^{2}.$$

In view of (14), we then have

Since $r_{\max}-\tilde{r}_{k}\ge\epsilon_{0},\ \forall k$, we have

$$\limsup_{J\rightarrow\infty}\sum_{j=0}^{J-1}\sum _{k}E \bigl(X_{k}[j] \bigr)/J\le4\frac{T}{\alpha}\frac{1}{\epsilon_{0}}KD^{2}.$$

Since in a slot each queue is increased at most by 1, we have

$$\sum_{k}\bar{X}_{k}\le \limsup_{J\rightarrow\infty}\sum_{j=0}^{J-1}\sum _{k}E \bigl(X_{k}[j] \bigr)/J+KT.$$

So,

$$\frac{1}{K}\sum_{k}\bar{X}_{k}\le T \biggl\{\frac{4}{\alpha}\frac{1}{\epsilon_{0}}D^{2}+1 \biggr \}.$$

 □