Optimal control of a single server in a finite-population queueing network

Abstract

We study the optimal dynamic assignment of a single server to multiple stations in a finite-population queueing network. The objective is to maximize the long-run average reward/throughput. We use sample-path comparisons to identify conditions on the network structure and service time distributions under which the optimal policy is an index policy. This index policy assigns the server to the non-empty station where it takes the shortest amount of time (in some stochastic sense) to complete a job. For example, in a network of multiple parallel stations, the optimal policy assigns the highest priority to the fastest station if service times can be ordered in likelihood ratios. Finally, by means of a numerical study, we test the shortest-expected-remaining-service-time policy on parallel-series networks with three stations and find that this index policy either coincides with the optimal policy or provides a near-optimal performance.

Appendix

In this appendix, we prove the analytical results presented in the main paper and provide supplemental material.

Proof of Proposition 1

Let $D_{0,j}^{\pi }(t)$ be the number of customers who request service from the jth branch after leaving station 0 during [0, t] under policy $\pi$. This means that $\sum _{j=1}^JD_{0,j}^{\pi }(t)=D_0^{\pi }(t)$. Let also $C_{i,j}^{\pi }(t)$ denote the number of customers in station (i, j) at time t ($1\le i\le i_j$ and $1\le j\le J$). Then, for $1\le i\le i_j$ and $1\le j\le J$, we have

$$\begin{aligned} C_{i,j}^{\pi }(t)=C_{i,j}^{\pi }(0)+D_{i-1,j}^{\pi }(t)-D_{i,j}^{\pi }(t), \end{aligned}$$

which implies

$$\begin{aligned} D_{i,j}^{\pi }(t)=D_{0,j}^{\pi }(t)+\sum _{k=1}^i\left( C_{k,j}^{\pi }(0)-C_{k,j}^{\pi }(t)\right) . \end{aligned}$$

(4)

Hence, we have

$$\begin{aligned} R^{\pi }=\liminf _{t\rightarrow \infty }\sum _{j=1}^J\sum _{i=1}^{i_j}\frac{r_{i,j}\mathrm {E}\left[ D_{0,j}^{\pi }(t)\right] }{t}+\liminf _{t\rightarrow \infty }\sum _{j=1}^J\sum _{i=1} ^{i_j}r_{i,j}\sum _{k=1}^i\frac{\mathrm {E}\left[ C_{k,j}^{\pi }(0)-C_{k,j}^{\pi }(t)\right] }{t}. \end{aligned}$$

Since $0\le C_{k,j}^{\pi }(t)\le B < \infty$ for all $t\ge 0$, $j=1,\ldots ,J$, and $k=1,\ldots ,i_j$, the last term in the above equation is zero, which leads to

$$\begin{aligned} R^{\pi }&=\liminf _{t\rightarrow \infty }\sum _{j=1}^J\sum _{i=1}^{i_j}\frac{r_{i,j}\mathrm {E}\left[ D_{0,j}^{\pi }(t)\right] }{t}\\&=\liminf _{t\rightarrow \infty }\sum _{j=1}^J\sum _{i=1}^{i_j}\frac{r_{i,j}p_j\mathrm {E}\left[ D_{0}^{\pi }(t)\right] }{t}\\&=T^{\pi }\sum _{j=1}^J\sum _{i=1}^{i_j}p_j r_{i,j}, \end{aligned}$$

where the second equation follows from the independent Bernoulli splitting of customers departing from station 0. $\square$

Proof of Proposition 4

We will prove the result by means of a sample-path argument. As is commonly used in proofs for characterizing optimal policies within a set of preemptive policies, we consider a discrete-time version of the problem, where the discretization can be arbitrarily small. Let T be the number of periods left in this discrete-time problem. We will use induction on T to prove the result. When $T=1$, all policies will result in the same number of departures from station 0, and hence the policy given in the statement of the proposition is trivially optimal. Now, suppose that the same policy is optimal for a finite-horizon problem with $T-1$ periods. We will next show that it is also optimal for a problem with T periods.

Let $\pi$ be a policy that does not follow the optimal policy defined in the proposition at time zero under the problem with T periods. From the inductive hypothesis, we can assume that $\pi$ follows the optimal policy from time 1 on. We next construct a new policy $\gamma$ that follows the optimal policy at time 0 and generates at least the same number of departures from station 0 as policy $\pi$ along the entire sample path.

Let $C_1$ be the job that policy $\pi$ takes into service at time 0 and $C_2$ be the job that policy $\gamma$ takes at time 0. (We here consider only the case where policy $\pi$ serves a job at time 0; a similar argument can be used when $\pi$ idles at time 0.) Suppose that job $C_i$ has $k_i$ tasks remaining (including the task at the current station at time 0) and $t_i$ units of service time attained at the current station for $i=1,2$. We need to consider two cases:

Case (i) We first consider the case with $k_1=k_2$ and $t_1< t_2$. (The case with $k_1=k_2$ and $t_1= t_2$ will trivially hold.) Since $\pi$ follows the optimal policy at time 1 and on, it will not serve $C_1$ again until $C_2$ is served. Let $\tau$ be the first time $\pi$ takes $C_2$ into service. (We assume that $\tau \le T$ but the results would trivially hold if the server never takes $C_2$ into service within the horizon.) We now construct $\gamma$ such that it agrees with $\pi$ at all decisions except that it serves $C_2$ at time 0 and $C_1$ at time $\tau$, and thereafter serves $C_1$ or $C_2$ with priority to $C_2$ whenever $\pi$ serves $C_1$ or $C_2$. Let $S_i$ be the remaining service time of job $C_i$ at the current station at time 0 under policy $\pi$ and $S_i^{\prime }$ be the corresponding service time under policy $\gamma$. Suppose that we directly couple the service times of all jobs other than $C_1$ and $C_2$ under both policies, and couple service time of job $C_1[C_2]$ at station 0 under policy $\pi$ and service time of job $C_2[C_1]$ under policy $\gamma$. Since service times are i.i.d. and ILR, and $t_1< t_2$, we have $S_2\le _\mathrm{lr} S_1$ and $S_2^{\prime }\le _\mathrm{lr} S_1^{\prime }$. Hence, we generate $m=\min \{S_1,S_2\}$ and $M=\max \{S_1,S_2\}$ from the appropriate distribution and use them for sample paths of both policies. If $m=M=1$, then we have $S_1=S_1^{\prime }=S_2=S_2^{\prime }=1$, and the two sample paths will couple at time $\tau +1$, i.e., both sample paths will reach the same state at this time. Furthermore, we will have $D_0^{\pi }(t)= D_0^{\gamma }(t)$ for all $t=0,1,\ldots ,T$. If $1<m\le M$, then we let $S_1=S_1^{\prime }=M$ and $S_2=S_2^{\prime }=m$ with probability $p=\Pr \{S_1=M, S_2=m|\min \{S_1,S_2\}=m,\max \{S_1,S_2\}=M\}$ and we let $S_1=S_1^{\prime }=m$ and $S_2=S_2^{\prime }=M$ with probability $1-p$. In each case, the sample paths will couple at $\tau +1$. Furthermore, we will again have $D_0^{\pi }(t)= D_0^{\gamma }(t)$ for all $t=0,1,\ldots ,T$. If $1=m< M$, then we let $S_1=S_2^{\prime }=M$ and $S_2=S_1^{\prime }=1$ with probability $1-p$, we let $S_1=S_2^{\prime }=1$ and $S_2=S_1^{\prime }=M$ with probability $1-p$, and we let $S_1=S_1^{\prime }=M$ and $S_2=S_2^{\prime }=1$ with probability $2p-1$. This generation works because $p\ge 1-p$ by Theorem 1.C.24 of Shaked and Shanthikumar [17] and the likelihood ratio ordering between $S_1$ and $S_2$, and it will yield the correct probabilities, i.e., $\Pr \{S_1=M, S_2=1|\min \{S_1,S_2\}=1,\max \{S_1,S_2\}=M\}=\Pr \{S_1^{\prime }=M, S_2^{\prime }=1|\min \{S_1^{\prime },S_2^{\prime }\}=1,\max \{S_1^{\prime },S_2^{\prime }\}=M\}=p$. In the first two subcases, the interchange has no effect and the sample paths again couple at $\tau +1$ with $D_0^{\pi }(t)= D_0^{\gamma }(t)$ for all $t=0,1,\ldots ,T$. In the third subcase, if $k_1>1$, then the sample paths under both policies will again couple at $\tau +1$ with no differences in the number of departures from station 0. However, if $k_1=1$, then job $C_2$ will enter station 0 at time 1 under policy $\gamma$ and at time $\tau +1$ under policy $\pi$. Let $S_0$ be the service time of $C_2$ at station 0 at this entrance to that station under both policies. Then, the two sample paths will couple at time $\tau +1+S_0$ and we will have

$$\begin{aligned} D_0^{\gamma }(t)= \left\{ \begin{array}{ll} D_0^{\pi }(t),&{}\quad 0\le t<S_0+1,\\ D_0^{\pi }(t)+1,&{}\quad S_0+1\le t< \tau +1+S_0,\\ D_0^{\pi }(t),&{}\quad t\ge \tau +1+S_0. \end{array} \right. \end{aligned}$$

Case (ii) We now consider the case with $k_1>k_2$. Let ${\overline{S}}_i$ be the total service time remaining at time 0 for job $C_i$ until it enters station 0 for $i=1,2$ under policy $\pi$. Thus, we can write ${\overline{S}}_i=\sum _{j=1}^{k_i}S_j(i)$, where $S_j(i)$ is the service time of job $C_i$’s jth task remaining for $i=1,2$. Note that the $S_j(i)$ are i.i.d. except for $S_1(1)$ and $S_1(2)$. Furthermore, since service times at all stations are ILR, by Theorem 1.C.9 in Shaked and Shanthikumar [17] (with the discussion that follows the theorem) and the facts that $k_1>k_2$ and the $S_j(i)$ are non-negative, we have ${\overline{S}}_2\le _\mathrm{lr}{\overline{S}}_1$. Now, in the proof of Case (i), setting $k_1=k_2=1$ and replacing $S_i$ with ${\overline{S}}_i$ completes the proof. $\square$

We defer the proof of Proposition 5 as it is based on an argument used in the proof of Proposition 6.

Proof of Proposition 6

We will use an inductive sample-path argument to prove the result. Suppose at time zero the server can only perform a total of $v\ge 1$ more tasks, at which point the server is shut off and the problem stops. Applying induction on v and letting $v\rightarrow \infty$ will complete the proof.

Suppose $\pi$ is a policy under which the server takes a job into service in station k with $k\ne (i_{j^*},j^*)$ at a decision epoch where $v=1$ and there exists a job available in station $(i_{j^*},j^*)$. (For convenience, we here abuse the notation and instead of using a 2-tuple to refer to a station we simply call it station k.) Since $v=1$, such an event could only happen once in a sample path. Without loss of generality, we assume that this event takes place at time zero. Now, consider another policy $\gamma$ that serves a job in station $(i_{j^*},j^*)$ at time zero. Note that once $\pi$ and $\gamma$ complete the respective tasks, there will be no other decisions left under either policy since $v=1$. To generate the sample paths under policies $\pi$ and $\gamma$, we directly couple the sojourn times of all jobs at station 0. If k is an intermediate station, i.e., it is not an end station that will immediately lead to an arrival to station 0, then it is easy to see that $D_0^{\gamma }(t)\ge D_0^{\pi }(t)$ for all $t\ge 0$ as $(i_{j^*},j^*)$ is an end station. Now consider the case where k is another end station. Let $S_1^{\prime }$ and $S_2$ be the service time of the job taken into service at time zero under policies $\gamma$ and $\pi$, respectively. Since $(i_{j^*},j^*)$ is the fastest end station in likelihood ratio ordering (and thus in usual stochastic ordering), we can couple the service times such that $S_1^{\prime }\le S_2$ with probability one. This means that policy $\gamma$ will have an earlier arrival at station 0 than policy $\pi$, and directly coupling the sojourn times of these two arrivals at station 0 will yield $D_0^{\gamma }(t)\ge D_0^{\pi }(t)$ for all $t\ge 0$. This concludes the proof that at a decision epoch where $v=1$ and there exists a job at station $(i_{j^*},j^*)$, it is better to take this job into service than to take any other job. Having this result, a simple sample-path argument would also immediately lead to the suboptimality of idling at such a decision epoch. This completes the proof of the proposition for $v=1$.

Now, suppose that the proposition holds when the server can serve at most $v-1$ more tasks. We will next show that a policy, say policy $\pi$, which assigns the server to a station other than station $(i_{j^*},j^*)$ at a decision epoch where there is a job at station $(i_{j^*},j^*)$ and the maximum number of tasks that could be served is v, can be improved by serving the job in station $(i_{j^*},j^*)$ at that decision epoch. Again, without loss of generality, assume that time zero is the first time policy $\pi$ starts working in station k, where $k\ne (i_{j^*},j^*)$, despite the fact that there is a job in station $(i_{j^*},j^*)$. At the time of completion of service of this job in station k, the total number of tasks that the server can take on will drop to $v-1$, and hence, by the inductive argument, we assume that policy $\pi$ starts serving a job in station $(i_{j^*},j^*)$ at this time. Let $S_2$ be the service time of the job that the server picks from station k right before the server moves to station $(i_{j^*},j^*)$ to serve a job there with service time $S_1$ under policy $\pi$. We construct an alternative policy $\gamma$ as follows: $\gamma$ serves a job in station $(i_{j^*},j^*)$ with service time $S_1^{\prime }$ at time zero, switches to station k, and serves a job there with service time $S_2^{\prime }$.

Case 1 (k is an intermediate station) We directly couple the service times of all jobs taken into service during $[0, \infty )$ under both policies, which yields $S_1^{\prime } = S_1$ and $S_2^{\prime } = S_2$. Let $S_0$ be the service time at station 0 for the job entering station 0 at time $S_2+S_1$ under policy $\pi$ and at time $S_1^{\prime }$ under policy $\gamma$. We then let $\gamma$ follow $\pi$ during $[S_1+S_2,\infty )$. This is possible because at least the same number of jobs are available to policy $\gamma$ compared to policy $\pi$ and idling is allowed. The system states for the two policies will become identical after time $S_2+S_1+S_0$. With this construction, it is then easy to see that $D_0^{\gamma }(t) \ge D_0^{\pi }(t)$ for all $t\ge 0$.

Case 2 (k is an end station) We cross couple $S_1$, $S_2$, $S_1^{\prime }$, and $S_2^{\prime }$ as follows. We first generate the minimum and maximum of $S_1$ and $S_2$, namely m and M, respectively, condition on their values, and use these values in both sample paths. Let $p=\Pr \{S_1=M|m,M\}=\Pr \{S_2=m|m,M\}$ and $q=\Pr \{S_1=m|m,M\}=\Pr \{S_2=M|m,M\}=1-p$. By Lemma 13.D.1(i) of Righter [14] (or Theorem 1.C.24 of Shaked and Shanthikumar [17]) and the likelihood ratio ordering between $S_1$ and $S_2$, we have $p<q$. Thus, we can let

• (i) $S_1=m$, $S_2=M$, $S_1^{\prime }=M$, $S_2^{\prime }=m$, with probability p,

• (ii) $S_1=M$, $S_2=m$, $S_1^{\prime }=m$, $S_2^{\prime }=M$, with probability p,

• (iii) $S_1=m$, $S_2=M$, $S_1^{\prime }=m$, $S_2^{\prime }=M$, with probability $1-2p$.

The coupling yields $S_1^{\prime } \le S_2$ (and $S_1 \le S_2^{\prime }$) in all three cases. See Fig. 6 for a visual depiction of this coupling. In the first two cases all arrival times to station 0 under policies $\pi$ and $\gamma$ are identical. Furthermore, the system states for the two policies become identical after time $m+M$. Hence, policy $\gamma$ can follow policy $\pi$ thereafter. By directly coupling the service times of all jobs taken into service after time $m+M$, we have $D_0^{\gamma }(t)=D_0^{\pi }(t)$ for all $t\ge 0$ under cases (i) and (ii).

Fig. 6

Visual depiction of sample-path couplings used in the proof of Proposition 6

In case (iii), let $S_0$ be the service time at station 0 for the job entering station 0 at time $S_2$ under policy $\pi$ and at time $S_1^{\prime }$ under policy $\gamma$. We directly couple the service times of all jobs taken into service after $m+M$. Note that the system states for the two policies will become identical after time $M+\max \{m,S_0\}$. However, policy $\gamma$ can follow $\pi$ after $m+M$ because the same jobs or more will be available to policy $\gamma$ compared to policy $\pi$ and idling is allowed. We then have $D_0^{\gamma }(t)\ge D_0^{\pi }(t)$ for all $t\ge 0$. This completes the proof that it is better to take a job in station $(i_{j^*},j^*)$ into service than to take any other job at a decision epoch where the maximum number of tasks that can be served is v.

We finally prove that idling is suboptimal at a decision epoch (say, at time zero, without loss of generality), where the maximum number of tasks that can be served is v and there exists a job in station $(i_{j^*},j^*)$. Consider policy $\pi$ that idles the server for I units of time at time zero. We already proved that if the server would be assigned to a job, then it is best to assign to one in station $(i_{j^*},j^*)$ (whenever there exists one) at a decision epoch where the maximum number of tasks that can be served is v. Hence, we assume that $\pi$ assigns the server to a job at station $(i_{j^*},j^*)$ with service time $S_1$ at time I. Consider an alternative policy $\gamma$ that serves a job in station $(i_{j^*},j^*)$ with service time $S_1^{\prime }$ at time zero and then starts idling for I units of time. We directly couple the service times of all jobs taken into service and sojourn times of all jobs at station 0 during $[0,\infty )$, which means that $S_1^{\prime }=S_1$. Let $S_0$ be the service time at station 0 for the job entering station 0 at time $S_1$ under policy $\gamma$ and at time $I+S_1$ under policy $\pi$. The system states under $\pi$ and $\gamma$ will become identical at $I+S_1+S_0$. However, policy $\gamma$ can follow $\pi$ after $I+S_1$ because the same jobs or more will be available to policy $\gamma$ compared to policy $\pi$ and idling is allowed. This construction will yield $D_0^{\gamma }(t)\ge D_0^{\pi }(t)$ for all $t\ge 0$, which completes the proof. $\square$

We need the following lemma to prove Proposition 5 and Corollary 1.

Lemma 1

Suppose that station $(i_{j^*},j^*)$ has the highest priority under an optimal policy in $\Pi _\mathrm{NP}$ where $j^*\in \{1,2,\ldots ,J\}$. If $i_{j^*}>1$, then, within $\Pi _\mathrm{NP}$, it suffices to consider policies that pool the two tasks at stations $(i_{j^*}-1,j^*)$ and $(i_{j^*},j^*)$ for all jobs that enter station $(i_{j^*}-1,j^*)$ in order to maximize $D_0^{\pi }(t)$ in the usual stochastic order for all $t\ge 0$.

Proof of Lemma 1

Note that the only source of jobs for station $(i_{j^*},j^*)$ is station $(i_{j^*}-1,j^*)$ when $i_{j^*}>1$. Hence, when seeking an optimal policy, it is sufficient to only consider policies that sequentially serve stations $(i_{j^*}-1,j^*)$ and $(i_{j^*},j^*)$, i.e., policies under which the tasks at these two stations are essentially pooled, given that station $(i_{j^*},j^*)$ has the highest priority under an optimal policy in $\Pi _\mathrm{NP}$. $\square$

Proof of Proposition 5

When $J=1$, using the arguments for the case where k is an intermediate station in the proof of Proposition 6, we can prove that the end station $(i_1,1)$ should receive the highest priority. Then, by Lemma 1, we can consider only policies that pool the two stations at the end of the branch. Repeating this argument several times proves Proposition 5. $\square$

Proof of Corollary 1

Applying Lemma 1 and Proposition 6 in Steps S.3a and S.3b of Algorithm A-1 proves Corollary 1. $\square$

We need Definition 1 and Lemma 2 to prove Proposition 7.

Definition 1

Y has a trapezoidal distribution with parameters $a\le b\le c \le d$ if its probability density function f(x) is given by

$$\begin{aligned} f(x)=\left\{ \begin{array}{ll} \frac{2(x-a)}{(b-a)(c+d-a-b)},&{}\quad \text{ if } a\le x\le b,\\ \frac{2}{c+d-a-b},&{}\quad \text{ if } b\le x\le c,\\ \frac{2(d-x)}{(d-c)(c+d-a-b)},&{}\quad \text{ if } c\le x\le d,\\ 0,&{}\quad \text{ otherwise. } \end{array}\right. \end{aligned}$$

Lemma 2

Let X and Y be two continuous random variables, where X is uniformly distributed over $[a_1,b_1]$ and Y has a trapezoidal distribution with parameters $a\le b\le c \le d$.

• (i) If $a_1\le a$ and $b_1\le c$, then $X\le _\mathrm{lr} Y$.

• (ii) If $a_1\ge b$ and $b_1\ge d$, then $Y\le _\mathrm{lr} X$.

Proof of Lemma 2

(i) Let f and g be the probability density functions for X and Y, respectively. We will show that $f(x)g(y)\ge f(y)g(x)$ for all $x\le y$, where $x,y\in [a_1,d]$. When $x\in [a_1,a)$, we have $g(x)=0$, which implies that $f(x)g(y)\ge f(y)g(x)$ for all $y\in [a_1,d]$. The proof will be then complete once we show that f(x) / g(x) is non-increasing for $x\in [a,d]$. For $x\in [a,b_1]$ (when $a\le b_1$), g(x) is non-decreasing by the condition that $b_1\le c$ and f(x) is a constant, and hence, f(x) / g(x) is non-increasing. For $x\in [b_1,d]$, f(x) / g(x) is trivially non-increasing as f(x) is zero, which completes the proof.

(ii) Follows directly from part (i), by noting that $-X$ is uniformly distributed on $[-b_1,-a_1]$ and $-Y$ has a trapezoidal distribution with parameters $-d\le -c\le -b \le -a$. $\square$

Proof of Proposition 7

(a) In Lemma 2, if $a=b$ and $c=d$, then Y has a uniform distribution on [a, c], from which the result follows immediately.

(b) The first stochastic inequality is already proved in part (a). For the ordering between $X_1$ and $X_2+X_3$, first note that $X_2+X_3$ has a trapezoidal distribution with parameters $a\le b\le c \le d$, where $a=a_2+a_3$, $b=a_2+a_3+\min \{b_2-a_2,b_3-a_3\}$, $c=b_2+b_3-\min \{b_2-a_2,b_3-a_3\}$, and $d=b_2+b_3$ by Proposition 2.1 in Korwar [11]. Since $a_1\le a_2+a_3$ and $b_1\le b_2+b_3-\min \{b_2-a_2,b_3-a_3\}$, part (i) of Lemma 2 yields that $X_1\le _\mathrm{lr} X_2+X_3$.

(c) Conditions imply that $a_1\ge a_3$ and $b_1\ge b_3$, and hence by part (a), $X_3\le _\mathrm{lr} X_1$. For the ordering between $X_1$ and $X_2+X_3$, we know from the proof of part (b) that $X_2+X_3$ has a trapezoidal distribution. Since $a_1\ge a_2+a_3+\min \{b_2-a_2,b_3-a_3\}$ and $b_1\ge b_2+b_3$, part (ii) of Lemma 2 yields that $X_2+X_3\le _\mathrm{lr} X_1$. $\square$

Proof of Proposition 8

We prove the result by a sample-path argument. Suppose that $\pi$ is a policy that chooses to serve a customer that has j tasks remaining (say, customer $C_1$) at time $\tau _1>0$, although there exists a customer (say, customer $C_2$) that has $k<j$ tasks remaining. We will next construct a new policy $\gamma$ that follows $\pi$ during $[0, \tau _1)$.

Let $t_i$ be the first time that job $C_i$ enters station 0 after $\tau _1$ for $i=1,2$. Let also ${\underline{\tau }}=\min \{t_1,t_2\}$ and ${\overline{\tau }}=\max \{t_1,t_2\}$, where $\tau _1<{\underline{\tau }}<{\overline{\tau }}$. During $[\tau _1,{\overline{\tau }})$, let $\gamma$ follow $\pi$ exactly except that $\gamma$ works on $C_1$ or $C_2$ with preference for $C_2$ whenever $\pi$ works on $C_1$ or $C_2$. We directly couple all service times at the service stations and station 0 that start at the same decision epoch under both policies during $[\tau _1,{\overline{\tau }}]$. Such a coupling would work because service times at all service stations are i.i.d. Let ${\underline{S}}_0$ be the service time (at station 0) of $C_1$ or $C_2$ that starts at time ${\underline{\tau }}$ and ${\overline{S}}_0$ be the service time (at station 0) of the other one that starts at time ${\overline{\tau }}$ under policy $\pi$. By the construction of policy $\gamma$, $C_2$ will enter station 0 earlier than $C_1$ under policy $\gamma$; denote this time by ${\underline{\tau }}^{\prime }$. Note that we must have ${\underline{\tau }}^{\prime }<{\underline{\tau }}$ due to the construction of policy $\gamma$ and the fact that $k<j$. We couple the two sample paths such that ${\underline{S}}_0$ and ${\overline{S}}_0$ are the service times of $C_2$ and $C_1$ at station 0 under policy $\gamma$, respectively. (The service times of all other jobs at station 0 are directly coupled under $\pi$ and $\gamma$.) At time ${\overline{\tau }}$, the same jobs or more will be available to the server under policy $\gamma$ when compared with policy $\pi$. Hence, policy $\gamma$ can follow policy $\pi$ after ${\overline{\tau }}$ given that idling is allowed. The two sample paths will eventually couple at time $\max \{{\underline{\tau }}+{\underline{S}}_0,{\overline{\tau }}\}$. For such a coupling, we have $D_0^{\gamma }(t)\ge D_0^{\pi }(t)$, for all $t\ge 0$. $\square$

Proof of Proposition 9

Suppose policy $\pi$ is a policy that idles at time $\tau _1>0$, when there exists at least one job that has fewer tasks remaining than the total number of tasks in the shortest branch of the network. Without loss of generality, assume that $\tau _1$ is the first such decision epoch. Let $C_1$ be the job that has the fewest number of tasks remaining at time $\tau _1$ and k be the number of tasks it has remaining, where $k\le \min _{j=1,\ldots ,J}{i_j}$. Let $\tau _2$ be the first time after $\tau _1$ at which the server starts working on a job. Based on Proposition 8, we need to only consider the case where policy $\pi$ starts working on job $C_1$ at time $\tau _2$. Note that job $C_1$ will still be the job with the fewest number of tasks remaining at time $\tau _2$ because any job that might have arrived during $[\tau _1,\tau _2)$ will have at least $\min _{j=1,\ldots ,J}{i_j}$ tasks remaining. Using similar arguments as in our earlier sample-path results, we can construct a new policy $\gamma$ that follows $\pi$ during $[0, \tau _1)$ but serves job $C_1$ at time $\tau _1$ such that $D_0^{\gamma }(t)\ge D_0^{\pi }(t)$ for all $t\ge 0$. $\square$

We finally provide a counter example to the conjecture that serving the job with the shortest service time in the likelihood ratio ordering maximizes the number of departures in a parallel network of more than two stations under non-preemptive and non-idling policies.

Example 1

Consider a parallel network of three service stations with deterministic service times of i units of time at service station i, for $i=1,2,3$, and $B\ge 3$. Suppose that at a decision epoch, say at time zero, there are no jobs at station 1 but a single job at each of stations 2 and 3, and the rest of the $B-2$ jobs are at station 0. Suppose also that under this sample path the first departure from station 0 among the $B-2$ jobs initially there takes place at time 3 and enters station 1. Under the non-idling and non-preemptive shortest-expected-service-time policy, the server will start serving at station 2 at time zero and then at station 3 at time 2. Hence, by time 4, only a single job would have entered station 0. Now consider another policy that serves the job at station 3 at time zero, and then takes the job at station 1 into service at time 3 right after it arrives. Under this alternative policy, two jobs enter station 0 by time 4, which implies that this policy could have a larger number of departures from station 0 than the shortest-expected-service-time policy for some $t\ge 4$.