Abstract
The purpose of this paper is to investigate situations of non-cooperative dynamic control of queueing systems by two agents, having different objectives. The main part of the paper is devoted to analyzing a problem of an admission and a service (vacation) control. The admission controller has to decide whether to allow arrivals to occur. Once the queue empties, the server goes on vacation, and controls the vacations duration (according to the state and past history of the queue). The immediate costs for each controller are increasing in the number of customers, but no convexity assumptions are made. The controllers are shown to have a stationary equilibrium policy pair, for which each controller uses a stationary threshold type policy with randomization in at most one state. We then investigate a problem of a non-zero sum stochastic game between a router into several queues, and a second controller that allocates some extra service capacity to one of the queues. We establish the equilibrium of a policy pair for which the router uses the intuitive “Join the shortest queue” policy.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
E. Altman, Flow control using the theory of zero-sum Markov games,IEEE Trans. Automat. Control 39(4) (1994) 814–818.
E. Altman, Monotonicity of optimal policies in a zero sum game: a flow control model,Advances of Dynamic Games and Applications 1 (1994) 269–286.
E. Altman, A Markov game approach for optimal routing into a queueing network, INRIA Report No. 2178, submitted (1994).
E. Altman and A. Hordijk, Zero-sum Markov games and worst-case optimal control of queueing systems, invited paper,Queueing Systems 21, special issue on Optimization of Queueing Systems, ed. S. Stidham (1995) 415–447.
E. Altman, A. Hordijk and F.M. Spieksma, Contraction conditions for average andα-discount optimality in countable state Markov games with unbounded rewards, Research Report No. TW-93-16, University of Leiden, to appear inMOR.
E. Altman and G. Koole, Stochastic scheduling games with Markov decision arrival processes,Journal Computers and Mathematics with Appl., 3rd. special issue on “Differential Games” (1993) 141–148.
E. Altman and N. Shimkin, Individually optimal dynamic routing in a processor sharing system: Stochastic game analysis, EE Pub. No. 849 (August, 1992). Submitted toOperations Research.
E. Altman and N. Shimkin, Worst-case and Nash routing policies in parallel queues with uncertain service allocations, IMA Preprint Series No. 1120 (Institute for Mathematics and Applications, University of Minnesota, Minneapolis, USA, 1993).
R. Cavazos-Cadena, Weak conditions for the existence of optimal stationary policies in average decision chains with unbounded costs,Kybernetika 25(3) (1989) 145–156.
K. Fan, Fixed-point and minimax theorems in locally convex topological linear spaces,Proc. Nat. Acad. Sci. 38 (1952) 121–126.
A. Federgruen, OnN-person stochastic games with denumerable state space,Advances in Applied Probability 10 (1978) 452–471.
A. Glazer and R. Hassin, Stable priority purchasing in queues,Operations Research Letters 6 (1986) 285–288.
I. Glicksberg, A further generalization of the Kakutani fixed point theorem with application to Nash equilibrium points,Proceedings of the American Mathematical Society 3 (1952) 170–174.
R. Hassin and M. Haviv, Equilibrium strategies and the value of information in a two line queueing system with threshold jockeying,Communications in Statistics Stochastic Models 10(2) (1994) 415–435.
M. Haviv, Stable strategies for processor sharing systems,European J. of Operations Research 52 (1991) 103–106.
D.P. Heyman, Optimal operating policies for M/G/1 queueing systems,Operations Research 16 (1968) 362–382.
A. Hordijk,Dynamic Programming and Markov Potential Theory, 2nd edition, Mathematical Centre Tracts 51 (Mathematisch Centrum, Amsterdam, 1977).
A. Hordijk and F. Spieksma, Constrained admission control to a queueing system,Advances in Applied Probability 21 (1989) 409–431.
M.T. Hsiao and A.A. Lazar, A game theoretic approach to decentralized flow control of Markovian queueing Networks, in:Performance '87, eds. Courtois and Latouche (1988) pp. 55–73.
M.M. Jaiswal,Priority Queues (Academic Press, 1968).
H.-U. Küenle,Stochastiche Spiele und Entscheidungesmodelle, Teubner-Texte, Band 89 (1986).
H.-U. Küenle, On the optimality of (s, S)-strategies in a minimax inventory model with average cost criterion,Optimization 22(1) (1991) 123–138.
S.A. Lippman and J.W. Mamer, Preemptive innovation,J. of Economic Theory 61(1) (1993) 104–119.
J.M. McNamara, S. Merad and E.J. Collins, The Hawk-Dove game as an average-cost problem,Advances in Applied Probability 23 (1991) 667–682.
P. Naor, On the regulation of queueing size by levying tolls,Econometrica 37 (1969) 15–24.
S. Ross,Applied Probability Models with Optimization Applications (Holden-Day, San Francisco, 1970).
N. Shimkin, A sample path approach to admission and flow control, private communication.
T.E.S. Raghavan and J.A. Filar, Algorithms for stochastic games — a survey,ZOR 35 (1991) 437–472.
F.M. Spieksma, Geometrically ergodic Markov chains and the optimal control of queues, PhD Thesis, University of Leiden (1990).
M.J. Sobel, Optimal average-cost policy for a queue with start-up and shut down costs,Operations Research 17 (1969) 145–162.
S. Stidham, Socially and individually optimal control of arrivals to a GI/M/1 queue,Management Science 24 (1970) 1598–1610.
S. Stidham and R.R. Weber, Monotonic and insensitive optimal policies for control of queues with undiscounted cost,Operations Research 87(4) (1989) 611–625.
M. Tidball and E. Altman, Approximations in dynamic zero-sum games, I,SIAM J. Control and Optimization 34(1) (1996) 311–328.
R.R. Weber and S. Stidham, Optimal control of service rates in networks of queues,Advances in Applied Probability 19 (1987) 202–218.
M. Yadin and P. Naor, Queueing systems with removable service station,Operations Research Quaterly 14 (1963) 393–405.
D.D. Yao, S-modular games, with queueing applications, invited paper,Queueing Systems 21, special issue on Optimization of Queueing Systems, ed. S. Stidham (1995) 449–475.
U. Yechiali, On optimal balking rules and toll charges in a GI/M/1 queueing process,Operations Research 19 (1971) 349–370.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Altman, E. Non zero-sum stochastic games in admission, service and routing control in queueing systems. Queueing Syst 23, 259–279 (1996). https://doi.org/10.1007/BF01206560
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01206560