Abstract
We analyze the tradeoff between efficiency and service quality in tandem systems with flexible servers and finite buffers. We reward efficiency by assuming that a revenue is earned each time a job is completed, and penalize poor service quality by incorporating positive holding costs. We study the dynamic assignment of servers to tasks with the objective of maximizing the long-run average profit. For systems of arbitrary size, structured service rates, and linear or nonlinear holding costs, we determine the server assignment policy that maximizes the profit. For systems with two stations, two servers with arbitrary service rates, and linear holding costs, we show that the optimal server assignment policy is of threshold type and determine the value of this threshold as a function of the revenue and holding cost. The threshold can be interpreted as the best possible buffer size, and hence our results prove the equivalence of addressing service quality via a holding cost and via limiting the buffer size. Furthermore, we identify the optimal buffer size when each buffer space comes at a cost. We provide numerical results that suggest that the optimal policy also has a threshold structure for nonlinear holding costs. Finally, for larger systems with arbitrary service rates, we propose effective server assignment heuristics.
Similar content being viewed by others
References
Ahn, H.-S., Duenyas, I., Lewis, M.E.: The optimal control of a two-stage tandem queueing system with flexible servers. Probab. Eng. Inf. Sci. 16, 453–469 (2002)
Ahn, H.-S., Duenyas, I., Zhang, R.: Optimal scheduling of a 2-stage tandem queue with parallel servers. Adv. Appl. Probab. 31, 1095–1117 (1999)
Ahn, H.-S., Duenyas, I., Zhang, R.: Optimal control of a flexible server. Adv. Appl. Probab. 36, 139–170 (2004)
Ahn, H.-S., Righter, R.: Dynamic load balancing with flexible workers. Adv. Appl. Probab. 38, 621–642 (2006)
Andradóttir, S., Ayhan, H.: Throughput maximization for tandem lines with two stations and flexible servers. Oper. Res. 53, 516–531 (2005)
Andradóttir, S., Ayhan, H., Down, D.G.: Server assignment policies for maximizing the steady-state throughput of finite queueing systems. Manag. Sci. 47, 1421–1439 (2001)
Andradóttir, S., Ayhan, H., Down, D.G.: Dynamic server allocation for queueing networks with flexible servers. Oper. Res. 51, 952–968 (2003)
Andradóttir, S., Ayhan, H., Down, D.G.: Dynamic assignment of dedicated and flexible servers in tandem lines. Probab. Eng. Inf. Sci. 21, 497–538 (2007)
Andradóttir, S., Ayhan, H., Kırkızlar, E.: Flexible servers in tandem lines with setup costs. Queueing Syst. 70, 165–186 (2012)
Bartholdi III, J.J., Bunimovich, L.A., Eisenstein, D.D.: Dynamics of two- and three-worker “bucket brigade” production lines. Oper. Res. 47, 488–491 (1999)
Bartholdi III, J.J., Eisenstein, D.D.: A production line that balances itself. Oper. Res. 44, 21–34 (1996)
Bartholdi III, J.J., Eisenstein, D.D., Foley, R.D.: Performance of bucket brigades when work is stochastic. Oper. Res. 49, 710–719 (2001)
Bell, S.L., Williams, R.J.: Dynamic scheduling of a system with two parallel servers in heavy traffic with complete resource pooling: asymptotic optimality of a threshold policy. Ann. Appl. Probab. 11, 608–649 (2001)
Bell, S.L., Williams, R.J.: Dynamic scheduling of a parallel server system in heavy traffic with complete resource pooling: asymptotic optimality of a threshold policy. Electron. J. Probab. 10, 1044–1115 (2005)
Bhulai, J., Koole, G.: A queueing model for call blending in call centers. IEEE Trans. Autom. Control 48, 1421–1426 (2003)
Billingsley, P.: Probability and Measure. Wiley, New York (1995)
Farrar, T.M.: Optimal use of an extra server in a two station tandem queueing network. IEEE Trans. Autom. Control 38, 1296–1299 (1993)
Gans, N., Zhou, Y.-P.: A call routing problem with service level constraints. Oper. Res. 51, 255–271 (2003)
Gel, E.S., Hopp, W.J., Van Oyen, M.P.: Factors affecting opportunity of worksharing as a dynamic line balancing mechanism. IIE Trans. 34, 847–863 (2002)
Harrison, J.M., López, M.J.: Heavy traffic resource pooling in parallel-server systems. Queueing Syst. 33, 339–368 (1999)
Hopp, W.J., Tekin, E., Van Oyen, M.P.: Benefits of skill chaining in serial production lines with cross-trained workers. Manag. Sci. 50, 83–98 (2004)
Iravani, S.M.R., Posner, M.J.M., Buzacott, J.M.: A two-stage tandem queue attended by a moving server with holding and switching costs. Queueing Syst. 26, 203–228 (1997)
Kaufman, D.L., Ahn, H.-S., Lewis, M.E.: On the introduction of an agile, temporary workforce into a tandem queueing system. Queueing Syst. 51, 135–171 (2005)
Kırkızlar, E., Andradóttir, S., Ayhan, H.: Robustness of efficient server assignment policies to service time distributions in finite-buffered lines. Nav. Res. Logist. 57, 563–582 (2010)
Kırkızlar, E., Andradóttir, S., Ayhan, H.: Flexible servers in understaffed tandem lines. Prod. Oper. Manag. 21, 761–777 (2012)
Kulkarni, V.G.: Modeling and Analysis of Stochastic Systems. Chapman & Hall, London (1995)
Lim, Y.F., Yang, K.K.: Maximizing throughput of bucket brigades on discrete work stations. Prod. Oper. Manag. 18, 48–59 (2009)
Lippman, S.A.: Applying a new device in the optimization of exponential queueing system. Oper. Res. 23, 687–710 (1975)
Mandelbaum, A., Stolyar, A.L.: Scheduling flexible servers with convex delay costs: heavy-traffic optimality of the generalized \(c \mu \)-rule. Oper. Res. 52, 836–855 (2004)
Mayorga, M.E., Taaffe, K.M., Arumugam, R.: Allocating flexible servers in serial systems with switching costs. Ann. Oper. Res. 172, 231–242 (2009)
McClain, J.O., Thomas, L.J., Sox, C.: On-the-fly line balancing with very little WIP. Int. J. Prod. Econ. 27, 283–289 (1992)
Pandelis, D.G.: Optimal use of excess capacity in two interconnected queues. Math. Methods Oper. Res. 65, 179–192 (2007)
Puterman, M.L.: Markov Decision Processes. Wiley, New York (1994)
Rosberg, Z., Varaiya, P.P., Walrand, J.C.: Optimal control of service in tandem queues. IEEE Trans. Autom. Control 27, 600–609 (1982)
Saghafian, S., Van Oyen, M.P., Kolfal, B.: The “W” network and the dynamic control of unreliable flexible servers. IIE Trans. 43, 893–907 (2011)
Schiefermayr, K., Weichbold, J.: A complete solution for the optimal stochastic scheduling of a two-stage tandem queue with two flexible servers. J. Appl. Probab. 42, 778–796 (2005)
Sennott, L.I., Van Oyen, M.P., Iravani, S.M.R.: Optimal dynamic assignment of a flexible server on an open production line with specialists. Eur. J. Oper. Res. 170, 541–566 (2006)
Tassiulas, L., Bhattacharya, P.P.: Allocation of interdependent resources for maximal throughput. Stoch. Models 16, 27–48 (2000)
Tassiulas, L., Ephrimedes, A.: Stability properties of constrained queueing systems and scheduling properties for maximum throughput in multihop radio networks. IEEE Trans. Autom. Control 37, 1936–1948 (1992)
Van Oyen, M.P., Gel, E.G.S., Hopp, W.J.: Performance opportunity for workforce agility in collaborative and noncollabortive work systems. IIE Trans. 33, 761–777 (2001)
Wolff, R.W.: Stochastic Modeling and the Theory of Queues. Prentice Hall, Englewood Cliffs (1989)
Wu, C.-H., Down, D.G., Lewis, M.E.: Heuristics for allocation of reconfigurable resources in a serial line with reliability considerations. IIE Trans. 40, 595–611 (2008)
Wu, C.-H., Lewis, M.E., Veatch, M.: Dynamic allocation of reconfigurable resources in a two-stage tandem queueing system with reliability considerations. IEEE Trans. Autom. Control 51, 309–314 (2006)
Zavadlav, E., McClain, J.O., Thomas, L.J.: Self-buffering, self-balancing, self-flushing production lines. Manag. Sci. 42, 1151–1164 (1996)
Acknowledgments
This research was supported by the National Science Foundation under Grant CMMI–0856600. The third author was also supported by the National Science Foundation under grant CMMI–0969747. The authors would like to thank the Associate Editor and two anonymous referees whose comments led to significant improvements to the paper.
Author information
Authors and Affiliations
Corresponding author
Appendix
Appendix
Details of the problem formulation (5):
The discrete time Markov chain \(\{ X_\pi ' (k) \}\) has state space \({\mathcal {S}}\) and transition probabilities \(p_d (x, x') = q_d (x, x') / q\) if \(x' \not = x\) and \(p_d (x, x) = 1 - \sum _{x' \in {\mathcal {S}}, x' \not = x} q_d (x, x') / q\) for all \(x \in {\mathcal {S}}\). We will generate a Poisson process \(\{ K (t) \}\) with rate \(q\) and the next state of the continuous time Markov chain \(\{ X_\pi (t) \}\) is generated (independently from \(\{ K (t) \}\)) using the transition probabilities of the discrete time Markov chain \(\{ X_\pi ' (k) \}\) at the times of the events of \(\{ K (t) \}\).
For all \(x, x' \in {\mathcal {S}}\) and \(\pi _d \in \Pi \), let
where \({\mathcal {D}} = \{ x \in {\mathcal {S}} : x_{N-1} >0 \}\), \({\mathcal {D}}_x = \{ (x_1, \ldots , x_{N-2}, x_{N-1}-1) \}\) for all \(x \in {\mathcal {D}}\), and \({\mathcal {D}}_x= \emptyset \) for all \(x \not \in {\mathcal {D}}\) (note that the set \({\mathcal {D}}_x\) has at most one state \(x'\)). As \(\sum _{j=2}^N h_j x_j\) is the holding cost per unit time in state \(x\) and \(\frac{1}{q}\) is the expected time between transitions, an expected holding cost of \(\frac{\sum _{j=2}^N h_j x_j}{q}\) is incurred for each transition into state \(x\). Moreover, a revenue of \(r=1\) is gained if there is a departure from the system. Hence, \(R_d'(x,x')\) equals the expected immediate reward associated with each transition from state \(x\) to state \(x'\). Consequently, for all \(\pi _d \in \Pi \),
It is clear that \(K (t) / t \rightarrow q\) almost surely as \(t \rightarrow \infty \) by the elementary renewal theorem. Moreover, it is clear that for all \(\pi _d \in \Pi \), the limit \(\lim _{K \rightarrow \infty } \sum _{k=1}^K R_d' \left( X_\pi ' (k-1), X_\pi ' (k)\right) / K\) exists almost surely by the strong law of large numbers for Markov chains (see, for example, [41], p. 164; and Sect. 3.8 of [26]). Uniform integrability shows that for all \(\pi _d \in \Pi \), we have
(see for example the corollary to Theorem 25.12 in [16]) since
for all \(K \ge 1\) and \(\sup _{t \ge 0} \mathrm{I}\!\mathrm{E}\{ [K (t) / t]^2 \} < \infty \) (because \(K (t)\) is a Poisson random variable with mean \(q t\)). Hence, the optimization problem (4) has the same solution as
Using the strong law of large numbers for Markov chains, we obtain for all \(\pi _d \in \Pi \) that
where
for all \(x \in {\mathcal {S}}\). Uniform integrability now gives that
Hence, the optimization problem (4) has the same solution as
As \(R_d(x)=qR_d''(x)\) for all \(x \in {\mathcal {S}}\) and \(\pi _d \in \Pi \), it follows that the optimization problem (4) has the same solution as the Markov decision problem (5).
Proof of Lemma 3.1
If all the servers are idle in a state \(x\), then \(x\) is an absorbing state and the long-run average profit is at most zero. Since \(\sum _{j=2}^N\frac{h_j}{\Sigma _j}<r\), the expedite policy \(\pi ^e\) is profitable, and hence at least one of the servers should not be idle in state \(x\). Let \(\pi \) be any policy that idles one server when the other server is assigned to a station \(j\) in state \(x\). Now compare \(\pi \) with the policy \(\pi '\) that assigns both servers to station \(j\) in state \(x\) and agrees with \(\pi \) otherwise. The next state visited, \(x'\), will be the same under both policies, and the transition time from state \(x\) to state \(x'\) is never longer for \(\pi '\) compared to \(\pi \). Hence, the number of departures is never smaller under \(\pi '\). Moreover, the holding cost per item produced is never bigger under \(\pi '\), because both policies go through the same sequence of states and the time between transitions is never longer under \(\pi '\). Hence, \(\frac{D_{\pi ,{\mathcal {B}}} (t)}{t} \ge \frac{D_{\pi ',{\mathcal {B}}} (t)}{t}\) and \(C_{\pi ,{\mathcal {B}}}(t) \le C_{\pi ',{\mathcal {B}}}(t)\) for all \(t \ge 0\). Equation (6) shows that \(P_{\pi ,{\mathcal {B}}}' \ge P_{\pi ',{\mathcal {B}}}'\), and consequently there exists an optimal policy that does not idle any servers in state \(x\). \(\square \)
Proof of Lemma 3.2
Let \(i,j \in \mathrm{I}\!\mathrm{N}^+\) and \(i<j\). We have \(f_1(i,j) \ge 0\) trivially because of our assumption that \(\mu _{11}\mu _{22} \ge \mu _{12}\mu _{21}\). Hence, we only need to show that \(f_2(i,j)>0\). Some algebra shows that when \(\mu _{11} \ne \mu _{22}\),
Hence, it is clear that \(f_2(i,j)>0\) because \(\mu _{11} \ne \mu _{22}\) and \(\mu _{11}\mu _{22} \ge \mu _{12}\mu _{21}\). Finally, when \(\mu _{11}=\mu _{22}\), it is clear that \(f_2(i,j)>0\) because \(i<j\) and \(\mu _{22}>0\). Hence the proof is complete. \(\square \)
Proof of Lemma 3.3
Note that when \(\mu _{11}\mu _{22}=\mu _{12}\mu _{21}\), we have \(\frac{f_1(i,i+1)}{f_2(i,i+1)}=0\) for all \(i \in \mathrm{I}\!\mathrm{N}^+\) and the result follows trivially. Hence, in the rest of the proof we assume that \(\mu _{11}\mu _{22}>\mu _{12}\mu _{21}\).
When \(\mu _{11} \ne \mu _{22}\), some algebra shows that
The last inequality is correct because \(\mu _{11}\ne \mu _{22}\), hence the lemma holds. When \(\mu _{11}=\mu _{22}\), we can show that the lemma holds because
Hence the proof is complete. \(\square \)
Proof of Lemma 3.4
First note that when \(\mu _{11}\mu _{22}=\mu _{12}\mu _{21}\), we have \(\frac{f_1(i,j)}{f_2(i,j)}=0\) for all \(i,j \in \mathrm{I}\!\mathrm{N}^+\) and \(i<j\) and the result follows trivially. Hence, in the rest of the proof we assume that \(\mu _{11}\mu _{22}>\mu _{12}\mu _{21}\).
It is sufficient to show that for all \(i,j \in \mathrm{I}\!\mathrm{N}^+\) with \(i<j-1\)
First assume that \(\mu _{11} \ne \mu _{22}\). Then, some algebra shows that inequality (9) holds if and only if
When \(\mu _{11}=\mu _{22}\), we observe that inequality (9) holds if and only if
Inequalities (10) and (11) hold because \(j>i\) and \(\mu _{22}>0\). Hence the proof is complete. \(\square \)
Proof of Lemma 3.5
First note that when \(\mu _{11}\mu _{22}=\mu _{12}\mu _{21}\), we have \(\frac{f_1(i,i+1)}{f_2(i,i+1)}=0\) for all \(i \in \mathrm{I}\!\mathrm{N}^+\) and the result follows trivially. Hence, in the rest of the proof we assume that \(\mu _{11}\mu _{22}>\mu _{12}\mu _{21}\).
When \(\mu _{11} \ne \mu _{22}\), using the \(f_2\) expression provided in the proof of Lemma 3.2 we obtain
Note that the numerator of this expression is positive, and that all the terms in the denominator are nonnegative and the last two terms are positive. Moreover, \(\sum _{k=1}^{i}k(\frac{\mu _{22}}{\mu _{11}})^{i-k} \ge i\), because the last term in the summation equals \(i\). Hence, it is clear that the denominator approaches infinity as \(i \rightarrow \infty \), and it follows that \(\frac{f_1(i,i+1)}{f_2(i,i+1)} \rightarrow 0\) as \(i \rightarrow \infty \). Finally, when \(\mu _{11}=\mu _{22}\), we observe that
It is clear that the numerator and denominator of this expression are positive and that the denominator approaches infinity as \(i \rightarrow \infty \). Hence the proof is complete. \(\square \)
Proof of Theorem 3.2
Lemma 3.1 shows that it suffices to consider the policies that are non-idling. Then, in each state \(i \in {\mathcal {S}}\), it is sufficient to consider the following set of allowable actions:
The policy described in the theorem corresponds to a Markov chain with a single recurrent class and possibly some transient states. Thus, we can use the policy iteration algorithm for weakly communicating models as described in Sect. 9.5.1 of [33].
Let \(P_d\) be the probability transition matrix corresponding to the policy \(\pi _d\), and \(r_d(i)\) denote the reward in state \(i\) when policy \(\pi _d\) is employed. It follows from Lemmas 3.2, 3.3, and 3.5 that \(s_1\) is well defined. Let us define \(\frac{f_1(0,1)}{f_2(0,1)}=\infty \), so that \(\frac{f_1(s_1,s_1+1)}{f_2(s_1,s_1+1)} < h \le \frac{f_1(s_1-1,s_1)}{f_2(s_1-1,s_1)}\). We start the policy iteration algorithm by considering the policy \(\pi _0=\pi _{d_0}\), where
Then we obtain
and
For all \(i,i' \in {\mathcal {S}}\) and \(a \in A_i\), we use \(r(i,a)\) to denote the immediate reward in state \(i\) when action \(a\) is taken and \(p(i' | i,a)\) to denote the one-step probability of going from state \(i\) to state \(i'\) when action \(a\) is chosen in state \(i\). Under our assumptions on the service rates (\(\mu _{11}\mu _{22} \ge \mu _{12}\mu _{21}\), \(\sum _{k=1}^M\mu _{kj}>0\) for \(j \in \{1,\ldots ,N\}\), and \(\sum _{j=1}^N\mu _{kj}>0\) for \(k \in \{1,\ldots ,M\}\)), it is clear that \(\mu _{11}>0\) and \(\mu _{22}>0\). Hence, the discrete time Markov chain \(\{ X_{\pi _0}' (t) \}\) found by uniformization of \(\{ X_{\pi _0} (t) \}\) is unichain, and we can solve the following set of equations to find a scalar \(g_0\) and a vector \(h_0\), letting \(h_0(0)=0\),
where \(e\) is the column vector of ones and \(I\) is the identity matrix. We can show that
and
is a solution to (12), with the convention that summation over an empty set is equal to zero.
For the remainder of the proof, we assume that \(\mu _{11} \ne \mu _{22}\) since the proof for the case with \(\mu _{11}=\mu _{22}\) is similar. When \(\mu _{11}\mu _{22}=\mu _{12}\mu _{21}\), the service rates are structured (with \(\mu _1=\mu _{11}\), \(\mu _2=\mu _{21}\), \(\gamma _1=1\), and \(\gamma _2=\frac{\mu _{12}}{\mu _{11}}\)). Consequently, Theorem 3.1 shows that the expedite policy maximizes the profit. Moreover, when \(\mu _{11}\mu _{22}=\mu _{12}\mu _{21}\), we observe that \(\frac{f_1(1,2)}{f_2(1,2)}=0\), and hence \(s_B^*=s_1=1\) in Theorem 3.2, corresponding to the expedite policy. Hence, in the rest of the proof, we only consider the case where \(\mu _{11}\mu _{22}>\mu _{12}\mu _{21}\).
Next, we compute \(d_1(i)\), where
and set \(d_1(i) = d_{0}(i)\) whenever possible. If one can show \(d_1(i)=d_{0}(i)\) for all \(i \in {\mathcal {S}}\), then the policy \(\pi _0\) is optimal according to Theorem 9.5.1 of [33]. Consequently, for all \(i \in {\mathcal {S}}\) and \(a \in A_i\), we want to show that the following inequality holds:
It is clear that the action \(d_0(i)\) is optimal for \(i \in \{0,B+2\}\) because the action space consists of only one action in these states. Hence, it is sufficient to check if the inequality (14) is satisfied in states \(1 \le i \le B+1\). For \(i \in \mathrm{I}\!\mathrm{N}^+\), let us define \(f_j(i)\) for \(j \in \{3,4,5,6\}\) as follows:
Moreover, let
First, let us consider the states \(i \in \{1,\ldots , s_B^*-1\}\). Recall that \(d_{0}(i)=a_{12}\) for \(i \in \{1,\ldots , s_B^*-1\}\). With some algebra we have
The denominator of this expression is always positive. We observe that \(f_3(i) > 0\) for \(i \in \{1,\ldots ,s_B^*-1\}\). If \(f_4(i) \le 0\), then \(\Delta (i,a_{11}) > 0\). Next, we prove that \(\Delta (i,a_{11}) > 0\) when \(f_4(i) >0\), by showing that \(h < \frac{f_3(i)}{f_4(i)}\) (note that we do not have to show this when \(f_4(i) \le 0\)). Extensive algebra, together with Lemma 3.2, shows that for \(i \in \{1,\ldots ,s_B^*-1\}\), we have
We observe that inequality (15) holds trivially because \(\mu _{11} \ne \mu _{22}\). Our assumption that \(h \le \frac{f_1(s_1-1,s_1)}{f_2(s_1-1,s_1)}\), together with Lemmas 3.3 and 3.4, implies that \(h \le \frac{f_1(i,s_B^*)}{f_2(i,s_B^*)}\) for \(i \in \{1,\ldots ,s_B^*-1\}\). Consequently, the inequality (15) shows that \(h < \frac{f_3(i)}{f_4(i)}\), and hence \(\Delta (i,a_{11}) > 0\) for \(i \in \{1,\ldots ,s_B^*-1\}\).
Similarly, we can show that
The denominator of this expression is always positive. We see that \(f_5(i) > 0\) for \(i \in \{1,\ldots ,s_B^*-1\}\). If \(f_6(i) \le 0\), then \(\Delta (i,a_{21}) \ge 0\). Next, we prove that \(\Delta (i,a_{21}) > 0\) when \(f_6(i) >0\), by showing that \(h < \frac{f_5(i)}{f_6(i)}\) (note that we do not have to show this when \(f_6(i) \le 0\)). Extensive algebra, together with Lemma 3.2, shows that for \(i \in \{1,\ldots ,s_B^*-1\}\), we have
We observe that the inequality (16) holds trivially because \(\mu _{11} \ne \mu _{22}\). Our assumption that \(h \le \frac{f_1(s_1-1,s_1)}{f_2(s_1-1,s_1)}\), together with Lemmas 3.3 and 3.4, implies that \(h \le \frac{f_1(i,s_B^*)}{f_2(i,s_B^*)}\) for \(i \in \{1,\ldots ,s_B^*-1\}\). Consequently, the inequality (16) shows that \(h < \frac{f_5(i)}{f_6(i)}\), and hence \(\Delta (i,a_{21}) > 0\) for \(i \in \{1,\ldots ,s_B^*-1\}\).
Finally, some algebra shows that
The denominator of this expression is always positive, and our assumption that \(h \le \frac{f_1(s_1-1,s_1)}{f_2(s_1-1,s_1)}\), together with Lemmas 3.2, 3.3, and 3.4, shows that \(\Delta (i,a_{22}) \ge 0\) for \(i \in \{1,\ldots ,s_B^*-1\}\) (note that \(\Delta (i,a_{22}) > 0\) when \(h < \frac{f_1(s_B^*-1,s_B^*)}{f_2(s_B^*-1,s_B^*)}\)).
For \(i \in \mathrm{I}\!\mathrm{N}^+\) and \(i \ge s\), let us define \(\bar{f}_j(i)\) for \(j \in \{1,2,3\}\) as follows:
Now, consider the states \(i \in \{ s_B^*,\ldots ,B+1 \}\), whenever this set is nonempty. Recall that \(d_{0}(i)=a_{22}\) for \(i \in \{ s_B^*,\ldots ,B+1 \}\). With some algebra we have
The denominator and the first term in the numerator are nonnegative. Moreover, \(\bar{f}_1(i) \ge 0\) is nondecreasing in \(i\), and hence we only need to show that \(\Delta (s_B^*,a_{11}) \ge 0\). Some algebra shows that
It is clear that \(\bar{f}_1(s_B^*) >0\) because \(\mu _{11} \ne \mu _{22}\). Hence we can conclude that \(\Delta (s_B^*,a_{11}) > 0\) and consequently \(\Delta (i,a_{11}) > 0\) for \(i \in \{s_B^*,\ldots ,B+1\}\).
Similarly, we can show that
The denominator of this expression is always positive. We observe that \(\bar{f}_2(s_B^*)=f_2(s_B^*,s_B^*+1)\). Moreover, \(\bar{f}_2(i)\) is positive (by Lemma 3.2) and increasing in \(i\). Hence, it is sufficient to show that \(-f_1(s_B^*,s_B^*+1)+h\bar{f}_2(s_B^*)=-f_1(s_B^*,s_B^*+1)+hf_2(s_B^*,s_B^*+1) \ge 0\). This follows from Lemma 3.3 since \(h>\frac{f_1(s_1,s_1+1)}{f_2(s_1,s_1+1)}\), and thus \(\Delta (i,a_{12}) > 0\) for \(i \in \{s_B^*,\ldots ,B+1\}\).
Finally, some algebra shows that
The denominator and the first term in the numerator are nonnegative. Moreover, \(\bar{f}_3(i)\ge 0\) is increasing in \(i\), and hence we only need to show that \(\Delta (s_B^*,a_{21}) \ge 0\). Some algebra shows that
It is clear that \(\bar{f}_3(s_B^*) > 0\) because \(\mu _{11}\ne \mu _{22}\). Hence, we can conclude that \(\Delta (s_B^*,a_{21}) > 0\) and consequently \(\Delta (i,a_{21}) > 0\) for \(i \in \{s_B^*,\ldots ,B+1\}\).
Note that any action that takes the process to one of the recurrent states can be used in states \(\{s_B^*+1,\ldots ,B+2\}\) because they are transient. For the reasons explained in the proof of Lemma 3.1, it is strictly suboptimal to idle both servers in a state. Similarly, when \(\mu _{ij}>0\) for \(i,j \in \{1,2\}\), idling one server in a recurrent state is strictly suboptimal compared to assigning the server to the station where the other server is working in that state, because in the problem formulation (5) the reward term is smaller in the idling policy and every recurrent state has a positive steady-state probability. The uniqueness of the optimal actions in states \(\{0,\ldots ,s_B^*\}\) when \(\mu _{11}\mu _{22} > \mu _{12}\mu _{21}>0\) and \(h\ne \frac{f_1(s_B^*-1,s_B^*)}{f_2(s_B^*-1,s_B^*)}\) now follows from the proof of Theorem 3.1 of [5] and the discussion in Sect. 9.5.2 of [33]. \(\square \)
Proof of Lemma 3.6
Note that if \(s_1 \le B+2\), then we have \(P'(B+1) = P'(B)\) (see Theorem 3.2 and Remark 3.3). Moreover, \(P'(B) \ge P'(B-1)\) (because the optimal policy for \(B-1\) is feasible for \(B\)). Hence, the inequality \(P'(B+1) - P'(B) \le P'(B) - P'(B-1)\) is satisfied trivially when \(s_1 \le B + 2\).
In the remainder of the proof, we prove the lemma for \(s_1 > B+2\) (so that \(s_{B-1}^* = B+1\), \(s_B^* = B+2\), and \(s_{B+1}^* = B+3\)). The expression for \(P' (B)\) is provided in equation (13). Note that we have \(P'(B) \ge 0\) for all \(B \ge 0\) because of our assumption that \(\sum _{j=2}^N\frac{h_j}{\Sigma _j} < r\).
First, consider the case with \(\mu _{11} \ne \mu _{22}\). It follows from Eq. (13) that \(P'(B) \ge 0\) if and only if \(\alpha _1 - h \alpha _2 \ge 0\), where
Note that
This implies that \(\alpha _2 \ge 0\), and hence we have that \(P'(B) \ge 0\) if and only if \(h \le \frac{\alpha _1}{\alpha _2}\). Some algebra shows that the inequality \(P'(B+1)- P'(B) \le P'(B) - P'(B-1)\) holds if and only if \(\beta _1- h \beta _2 \ge 0\), where
If \(\beta _2 \le 0\), the result follows trivially. Otherwise, if \(\beta _2 > 0\), some algebra shows that
where \(\gamma (t) = \sum _{k=0}^{B+1} (\mu _{11}^{t-k} -\mu _{22}^{t-k})(\mu _{11}^{k} -\mu _{22}^{k})\) for \(t \ge 0 \) . The assumption that \(\mu _{11}\mu _{22} \ge \mu _{12}\mu _{21}\) clearly implies that inequality (17) holds, and hence \(\frac{\beta _1}{\beta _2} \ge \frac{\alpha _1}{\alpha _2}\). The fact that \(h \le \frac{\alpha _1}{\alpha _2}\) now implies that \(\beta _1- h \beta _2 \ge 0\), and hence \(P'(B+1)- P'(B) \le P'(B) - P'(B-1)\).
Finally, consider the case with \(\mu _{11} = \mu _{22}\). It follows from Eq. (13) that \(P'(B) \ge 0\) if and only if
Some algebra shows that
If \(\mu _{12} \ge \mu _{21}\), the result follows trivially. Otherwise, we have
The last inequality holds because the assumptions that \(\mu _{11} = \mu _{22}\) and \(\mu _{11}\mu _{22} \ge \mu _{12}\mu _{21}\) together imply that \(\mu _{22}^2 \ge \mu _{12} \mu _{21}\). Hence, the fact that \(P'(B) \ge 0\) now implies that \(P'(B+1)- P'(B) \le P'(B) - P'(B-1)\). The proof is complete. \(\square \)
Rights and permissions
About this article
Cite this article
Kırkızlar, E., Andradóttir, S. & Ayhan, H. Profit maximization in flexible serial queueing networks. Queueing Syst 77, 427–464 (2014). https://doi.org/10.1007/s11134-014-9393-y
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11134-014-9393-y
Keywords
- Flexible servers
- Finite buffers
- Profit maximization
- Holding cost
- Queueing control
- Buffer size optimization