Level–phase independent stationary distributions for GI/M/1-type Markov chains with infinitely-many phases
Introduction
A two-dimensional discrete-time Markov chain on the state space is called a (level-independent) GI/M/1-type Markov chain if its transition probability matrix has the block representation where the s and s are matrices of order . We shall use the matrix functions and .
The first dimension is called the level, the second is called the phase, and , the number of phases, may be finite or infinite. A GI/M/1-type Markov chain is called a Quasi-Birth-And-Death process or QBD, when for .
It is known that an irreducible Markov chain with a transition probability matrix of the form (1) is positive recurrent if and only if element-wise and the system has a solution, where the rate matrix is the minimal nonnegative solution of the matrix equation and is the -dimensional column vector of ones.
In this case, partitioning the stationary distribution according to levels so that , with , the Markov chain has the matrix-geometric stationary distribution given by . This was proved in Neuts [1] for and is a consequence of Tweedie [2] for . In the case where , condition (2) is equivalent to the fact that the spectral radius of is strictly less than one.
The matrix is the transition matrix of the censored Markov chain observed only when it is in level zero. Therefore, by (3), this censored Markov chain is positive recurrent and is proportional to its stationary distribution.
It is obvious that if the transition probabilities into level zero are changed, then we can expect that the solution of (3) will change and so, consequently, will . However, when , it is straightforward to show that, for all , where , irrespective of , and we see that the stationary distribution decays at rate as the level is increased.
The situation is more complicated when . The first thing to note is that the stationary distribution may not exhibit geometric decay at all. Miyazawa and Zhao [3], He, Li and Zhao [4], and Miyazawa [5] discussed this phenomenon in the context of various structured models and went on to provide conditions for the stationary distribution to decay geometrically with the level. A very good general overview of this whole area was given in the survey paper by Miyazawa [6].
Let be the set of vectors such that . In the continuous-time context and for some tridiagonal special cases of the parameter matrices , Kroese, Scheinhardt and Taylor [7] and Motyer and Taylor [8], [9] showed that it is possible for there to be a continuum of values for which there is a positive vector such that Each of these values is a potential decay rate of the stationary distribution, compatible with the transition dynamics of the process away from the boundary. If it happens that also satisfies (3), then the stationary distribution will be such that, for all and we see that the stationary distribution has the level–phase independence property conveniently expressed by , where and denotes the Kronecker product.
For the special cases of the transition matrices that they considered, and all the relevant values of , Kroese, Scheinhardt and Taylor [7] and Motyer and Taylor [8] gave examples of sequences constructed so that the which satisfies (6) also satisfies (3), which demonstrated that all the decay rates compatible with Eq. (6) in their examples can be realised by suitably modifying the boundary transition matrices.
The existence of a continuum of values of and positive vectors compatible with Eq. (6) is a common characteristic of models with transition matrices of the form (1) and infinitely-many phases. However, it is not known in general whether each of these values of can actually be realised as the decay rate of the stationary distribution of a model with transition matrices . In this paper, we settle this question in the affirmative by using the scalar and vector from (6) to construct a sequence for which Eq. (3) is satisfied by . The resulting GI/M/1-type model will have a level–phase independent stationary distribution with decay rate .
Our construction adapts the method that Latouche and Taylor [10] used to create chains with the level–phase independence property for the case . Of course, in that case and under the irreducibility Assumption 2.1 that we make below, the only possible value of is the spectral radius of .
The paper is organised as follows. In the next section, we specify our basic assumptions on the interior blocks and give a useful necessary and sufficient condition for the independence of level and phase. Our construction of a sequence of that guarantees the level–phase independence property is established in Section 3, and in Sections 4 Example 1: a two-priority system, 5 Example 2: the tandem Jackson network we apply the general results to the M/M/1 priority queue with two classes of customers and to the two-node tandem Jackson network, respectively. Finally, we take the opportunity in Section 6 to make a few concluding remarks.
Section snippets
A condition for independence
In order to simplify our presentation, and not be side-tracked by marginal issues, we shall make an irreducibility assumption. Assumption 2.1 The transition matrix that is obtained from (1) by removing the reflection from level zero is sub-stochastic and irreducible.
For the case , Latouche and Taylor [10] also used Assumption 2.1. As discussed there, the irreducibility assumption has a number of useful consequences, and it is not restrictive from a modelling
Transition probabilities at the boundary
In this section we prove that, for a GI/M/1-type Markov chain with transition matrix of the form (1), and any scalar and positive vector for which (6) holds, it is possible to choose the transition matrices into level zero, so that the level is independent of the phase. First we prove the following lemma which is used for the main result. Lemma 3.1 For a Markov chain with transition matrix (1) such that (2) holds, we have
Proof Note that all matrices and
Example 1: a two-priority system
As an example, consider a single server Markovian priority queueing system with two types of customers. Customers of type , for , arrive according to a Poisson process with rate and are served at an exponential rate . Type 1 customers receive preemptive priority over Type 2 customers. Miller [16] modelled this system as a continuous-time QBD for which the level is the number of customers of Type 1 and the phase is the number of customers of Type 2, the transition rates of which are
Example 2: the tandem Jackson network
In [7], Kroese, Scheinhardt and Taylor modelled the two-node tandem Jackson network depicted in Fig. 2 as a continuous-time quasi-birth-and-death process with infinitely-many phases: the number in the second queue is the level and the number in the first queue is the phase. The arrival rate to queue 1 is and the service rates at the first and second queues are given by and respectively.
In the notation of Section 1, we have
Discussion
Theorem 3.3 answers the question that we posed in the Introduction. For any scalar and vector such that (10) holds, it shows how to construct boundary matrices so that the stationary distribution of the GI/M/1 Markov chain with transition matrix in (1) has the level–phase independence property, and the level has a geometric distribution with decay rate .
Since there can, in general, be a continuum of values of such that there exists a with (10) holding, there can be a
Acknowledgements
All three authors thank the Ministère de la Communauté française de Belgique for funding this research through the ARC grant AUWB-08/13-ULB 5. The first and third authors also acknowledge the Australian Research Council for funding part of the work through Discovery Grant number DP110101663.
Guy Latouche received the Ph.D. degree in Mathematics from the Université Libre de Bruxelles in 1976. He has mostly taught classes on stochastic processes and their applications at ULB, and he has extensively visited colleagues at the University of Adelaide, the University of Delaware, the University of Melbourne, the Tokyo Institute of Technology, and Bellcore.
His research activity includes various aspects of computational probability: he has participated in the development of matrix-analytic
References (16)
Matrix-Geometric Solutions in Stochastic Models
(1981)Operator-geometric stationary distributions for Markov chains, with application to queueing models
Advances in Applied Probability
(1982)- et al.
The stationary tail asymptotics in the GI/G/1 type queue with countably many background states
Advances in Applied Probability
(2004) - et al.
Light-tailed behavior in QBD processes with countably many phases
Stochastic Models
(2009) Tail decay rates in double QBD processes and related reflected random walks
Mathematics of Operations Research
(2009)Light tail asymptotics in multidimensional reflecting processes for queueing networks
TOP
(2011)- et al.
Spectral properties of the tandem Jackson network, seen as a quasi-birth-and-death process
The Annals of Applied Probability
(2004) - et al.
Decay rates for quasi-birth-and-death process with countably many phases and tri-diagonal block generators
Advances in Applied Probability
(2006)
Cited by (11)
Improving efficiency of service systems by performing a part of the service without the customer's presence
2022, European Journal of Operational ResearchCitation Excerpt :However, when the phase space is infinite (as well as the level space), the properties are more complex. Kapodistria and Shneer (2020) and Latouche, Mahmoodi, and Taylor (2013) provide further discussion. In the current study, we formulate such two unbounded dimensions system, investigate it by deriving a multivariate probability generating function (MPGF), and obtain closed-form expressions for the system's steady-state probabilities and various performance measures.
Poisson's equation for discrete-time quasi-birth-and-death processes
2013, Performance EvaluationA computational framework for two-dimensional random walks with restarts
2020, SIAM Journal on Scientific ComputingSOLVING QUADRATIC MATRIX EQUATIONS ARISING in RANDOM WALKS in the QUARTER PLANE
2020, SIAM Journal on Matrix Analysis and Applications
Guy Latouche received the Ph.D. degree in Mathematics from the Université Libre de Bruxelles in 1976. He has mostly taught classes on stochastic processes and their applications at ULB, and he has extensively visited colleagues at the University of Adelaide, the University of Delaware, the University of Melbourne, the Tokyo Institute of Technology, and Bellcore.
His research activity includes various aspects of computational probability: he has participated in the development of matrix-analytic methods for the analysis of Markov models, with applications in queueing theory, telecommunication systems, fluid queues, branching processes and ruin theory. He has also shown a minor, but recurrent, interest for nearly completely decomposable systems.
Safieh Mahmoodi received her B.Sc. degree in Statistics from Shiraz University (SU), the M.Sc. degree in Mathematical Statistics from Isfahan University of Technology (IUT) and her Ph.D. degree in Probability Theory and Stochastic Processes from SU, Iran. Since 2005 she has been with the Department of Mathematical Sciences, IUT, as an assistant professor. Her research interests include Stochastic Processes and Modelling, Stable Processes and Random Measures, Queueing Theory, Markov Processes, Matrix analytic methods, Large Deviations Theory, Statistical Inferences and Time Series Analysis.
Peter G. Taylor received a B. Sc.(Hons) and a Ph.D. in Applied Mathematics from the University of Adelaide in 1980 and 1987 respectively. In between, he spent time working for the Australian Public Service in Canberra. After periods at the Universities of Western Australia and Adelaide, he moved at the beginning of 2002 to the University of Melbourne. In January 2003, he took up a position as the inaugural Professor of Operations Research. He was Head of the Department of Mathematics and Statistics from 2005 until 2010.
Peter’s research interests lie in the fields of stochastic modelling and applied probability, with particular emphasis on applications in telecommunications, biological modelling, healthcare and disaster management. Recently he has become interested in the interaction of stochastic modelling with optimisation and optimal control under conditions of uncertainty. He is regularly invited to present plenary papers at international conferences. He has also acted on organising and programme committees for many conferences.
Peter is the Editor-in-Chief of ‘Stochastic Models’, and on the editorial boards of ‘Queueing Systems’, the ‘Journal of Applied Probability’ and ‘Advances in Applied Probability’. In 2008, Peter became one of the five trustees of the Applied Probability Trust. This trust, which is based in Sheffield, UK, is the body which publishes the Applied Probability journals plus The Mathematical Scientist and Spectrum.
From February 2006 to February 2008, Peter was Chair of the Australia and New Zealand Division of Industrial and Applied Mathematics (ANZIAM). He has recently completed a two-year term as President of the Australian Mathematical Society.