Elsevier

Performance Evaluation

Volume 70, Issue 9, September 2013, Pages 551-563
Performance Evaluation

Level–phase independent stationary distributions for GI/M/1-type Markov chains with infinitely-many phases

https://doi.org/10.1016/j.peva.2013.05.004Get rights and content

Abstract

GI/M/1-type Markov chains are two-dimensional processes, with one dimension called the level and the other the phase. When such a chain is positive-recurrent and the phase space is finite, much is known about the properties of the stationary distribution. In particular, it is known how to derive the decay rate as the level variable approaches infinity. However, when the phase space is infinite, the properties are more complicated. A range of decay rates may be achievable and the transition structure at the boundary has a strong influence on the stationary distribution.

In this paper, we show that, for given transition probabilities in the interior states, it is possible under mild conditions to change the boundary transition probabilities in such a way that the level and phase are independent under the stationary distribution and furthermore, one may choose the decay rate of the stationary distribution from the entire range compatible with the interior transition rates.

Introduction

A two-dimensional discrete-time Markov chain {(Xl,Yl),l=0,1,2,} on the state space {(k,j);k=0,1,,j=1,2,,m} is called a (level-independent) GI/M/1-type Markov chain if its transition probability matrix P has the block representation P=[B0A100B1A0A10B2A1A0A1B3A2A1A0], where the Ais and Bis are matrices of order m. We shall use the matrix functions A(z)=k=1zkAkI and B(z)=k=0zkBk.

The first dimension is called the level, the second is called the phase, and m, the number of phases, may be finite or infinite. A GI/M/1-type Markov chain is called a Quasi-Birth-And-Death process or QBD, when Ai=Bi=0 for i2.

It is known that an irreducible Markov chain with a transition probability matrix of the form (1) is positive recurrent if and only if k=0R̂k1< element-wise and the system π0=π0k=0R̂kBk,π0k=0R̂k1=1, has a solution, where the rate matrix R̂ is the minimal nonnegative solution of the matrix equation R=k=0RkAk+1, and 1 is the m-dimensional column vector of ones.

In this case, partitioning the stationary distribution according to levels so that π=(π0,π1,π2,), with πk=(πk,1,πk,2,), the Markov chain has the matrix-geometric stationary distribution given by πk=π0R̂k. This was proved in Neuts  [1] for m< and is a consequence of Tweedie  [2] for m=. In the case where m<, condition (2) is equivalent to the fact that the spectral radius sp(R̂) of R̂ is strictly less than one.

The matrix k=0R̂kBk is the transition matrix of the censored Markov chain observed only when it is in level zero. Therefore, by (3), this censored Markov chain is positive recurrent and π0 is proportional to its stationary distribution.

It is obvious that if the transition probabilities Bk into level zero are changed, then we can expect that the solution π0 of  (3) will change and so, consequently, will πk. However, when m<, it is straightforward to show that, for all j=1,,m, limkπk+1,jπk,j=η where η=sp(R̂), irrespective of π0, and we see that the stationary distribution decays at rate η as the level is increased.

The situation is more complicated when m=. The first thing to note is that the stationary distribution may not exhibit geometric decay at all. Miyazawa and Zhao  [3], He, Li and Zhao  [4], and Miyazawa  [5] discussed this phenomenon in the context of various structured models and went on to provide conditions for the stationary distribution to decay geometrically with the level. A very good general overview of this whole area was given in the survey paper by Miyazawa  [6].

Let 1 be the set of vectors x={xi}R such that |xi|<. In the continuous-time context and for some tridiagonal special cases of the parameter matrices Ak, Kroese, Scheinhardt and Taylor  [7] and Motyer and Taylor  [8], [9] showed that it is possible for there to be a continuum of values η(0,1) for which there is a positive vector uη1 such that uηR̂=ηuη. Each of these values is a potential decay rate of the stationary distribution, compatible with the transition dynamics of the process away from the boundary. If it happens that uη also satisfies (3), then the stationary distribution will be such that, for all k,πk=(1η)ηkuη and we see that the stationary distribution has the level–phase independence property conveniently expressed by π=μuη, where μk=(1η)ηk and denotes the Kronecker product.

For the special cases of the transition matrices Ak that they considered, and all the relevant values of η, Kroese, Scheinhardt and Taylor  [7] and Motyer and Taylor  [8] gave examples of sequences Bk constructed so that the uη which satisfies (6) also satisfies (3), which demonstrated that all the decay rates compatible with Eq. (6) in their examples can be realised by suitably modifying the boundary transition matrices.

The existence of a continuum of values of η(0,1) and positive vectors uη1 compatible with Eq. (6) is a common characteristic of models with transition matrices of the form (1) and infinitely-many phases. However, it is not known in general whether each of these values of η can actually be realised as the decay rate of the stationary distribution of a model with transition matrices Ak. In this paper, we settle this question in the affirmative by using the scalar η and vector uη from (6) to construct a sequence Bk for which Eq. (3) is satisfied by uη. The resulting GI/M/1-type model will have a level–phase independent stationary distribution with decay rate η.

Our construction adapts the method that Latouche and Taylor  [10] used to create chains with the level–phase independence property for the case m<. Of course, in that case and under the irreducibility Assumption 2.1 that we make below, the only possible value of η is the spectral radius of R̂.

The paper is organised as follows. In the next section, we specify our basic assumptions on the interior blocks and give a useful necessary and sufficient condition for the independence of level and phase. Our construction of a sequence of Bk that guarantees the level–phase independence property is established in Section  3, and in Sections  4 Example 1: a two-priority system, 5 Example 2: the tandem Jackson network we apply the general results to the M/M/1 priority queue with two classes of customers and to the two-node tandem Jackson network, respectively. Finally, we take the opportunity in Section  6 to make a few concluding remarks.

Section snippets

A condition for independence

In order to simplify our presentation, and not be side-tracked by marginal issues, we shall make an irreducibility assumption.

Assumption 2.1

The transition matrix T=[A0A10A1A0A10A2A1A0A1] that is obtained from (1) by removing the reflection from level zero is sub-stochastic and irreducible.

For the case m<, Latouche and Taylor  [10] also used Assumption 2.1. As discussed there, the irreducibility assumption has a number of useful consequences, and it is not restrictive from a modelling

Transition probabilities at the boundary

In this section we prove that, for a GI/M/1-type Markov chain with transition matrix of the form (1), and any scalar η(0,1) and positive vector uη1 for which (6) holds, it is possible to choose the transition matrices Bk into level zero, so that the level is independent of the phase. First we prove the following lemma which is used for the main result.

Lemma 3.1

For a Markov chain with transition matrix   (1)   such that   (2)   holds, we havek=0R̂k(1n=1k1An1)=1.

Proof

Note that all matrices and

Example 1: a two-priority system

As an example, consider a single server Markovian priority queueing system with two types of customers. Customers of type i, for i=1,2, arrive according to a Poisson process with rate λi and are served at an exponential rate  μi. Type 1 customers receive preemptive priority over Type 2 customers. Miller  [16] modelled this system as a continuous-time QBD for which the level is the number of customers of Type 1 and the phase is the number of customers of Type 2, the transition rates of which are

Example 2: the tandem Jackson network

In  [7], Kroese, Scheinhardt and Taylor modelled the two-node tandem Jackson network depicted in Fig. 2 as a continuous-time quasi-birth-and-death process with infinitely-many phases: the number in the second queue is the level and the number in the first queue is the phase. The arrival rate to queue 1 is λ and the service rates at the first and second queues are given by μ1 and μ2 respectively.

In the notation of Section  1, we have Ā1=[000μ1000μ10],Ā1=μ2I,Ā0=[λμ2λ00λμ1μ2λ00λμ1

Discussion

Theorem 3.3 answers the question that we posed in the Introduction. For any scalar η(0,1) and vector uη such that (10) holds, it shows how to construct boundary matrices so that the stationary distribution of the GI/M/1 Markov chain with transition matrix P in (1) has the level–phase independence property, and the level has a geometric distribution with decay rate  η.

Since there can, in general, be a continuum of values of η(0,1) such that there exists a uη with (10) holding, there can be a

Acknowledgements

All three authors thank the Ministère de la Communauté française de Belgique for funding this research through the ARC grant AUWB-08/13-ULB 5. The first and third authors also acknowledge the Australian Research Council for funding part of the work through Discovery Grant number DP110101663.

Guy Latouche received the Ph.D. degree in Mathematics from the Université Libre de Bruxelles in 1976. He has mostly taught classes on stochastic processes and their applications at ULB, and he has extensively visited colleagues at the University of Adelaide, the University of Delaware, the University of Melbourne, the Tokyo Institute of Technology, and Bellcore.

His research activity includes various aspects of computational probability: he has participated in the development of matrix-analytic

References (16)

  • M.F. Neuts

    Matrix-Geometric Solutions in Stochastic Models

    (1981)
  • R.L. Tweedie

    Operator-geometric stationary distributions for Markov chains, with application to queueing models

    Advances in Applied Probability

    (1982)
  • M. Miyazawa et al.

    The stationary tail asymptotics in the GI/G/1 type queue with countably many background states

    Advances in Applied Probability

    (2004)
  • Q.-M. He et al.

    Light-tailed behavior in QBD processes with countably many phases

    Stochastic Models

    (2009)
  • M. Miyazawa

    Tail decay rates in double QBD processes and related reflected random walks

    Mathematics of Operations Research

    (2009)
  • M. Miyazawa

    Light tail asymptotics in multidimensional reflecting processes for queueing networks

    TOP

    (2011)
  • D.P. Kroese et al.

    Spectral properties of the tandem Jackson network, seen as a quasi-birth-and-death process

    The Annals of Applied Probability

    (2004)
  • A.J. Motyer et al.

    Decay rates for quasi-birth-and-death process with countably many phases and tri-diagonal block generators

    Advances in Applied Probability

    (2006)
There are more references available in the full text version of this article.

Cited by (11)

  • Improving efficiency of service systems by performing a part of the service without the customer's presence

    2022, European Journal of Operational Research
    Citation Excerpt :

    However, when the phase space is infinite (as well as the level space), the properties are more complex. Kapodistria and Shneer (2020) and Latouche, Mahmoodi, and Taylor (2013) provide further discussion. In the current study, we formulate such two unbounded dimensions system, investigate it by deriving a multivariate probability generating function (MPGF), and obtain closed-form expressions for the system's steady-state probabilities and various performance measures.

  • SOLVING QUADRATIC MATRIX EQUATIONS ARISING in RANDOM WALKS in the QUARTER PLANE

    2020, SIAM Journal on Matrix Analysis and Applications
View all citing articles on Scopus

Guy Latouche received the Ph.D. degree in Mathematics from the Université Libre de Bruxelles in 1976. He has mostly taught classes on stochastic processes and their applications at ULB, and he has extensively visited colleagues at the University of Adelaide, the University of Delaware, the University of Melbourne, the Tokyo Institute of Technology, and Bellcore.

His research activity includes various aspects of computational probability: he has participated in the development of matrix-analytic methods for the analysis of Markov models, with applications in queueing theory, telecommunication systems, fluid queues, branching processes and ruin theory. He has also shown a minor, but recurrent, interest for nearly completely decomposable systems.

Safieh Mahmoodi received her B.Sc. degree in Statistics from Shiraz University (SU), the M.Sc. degree in Mathematical Statistics from Isfahan University of Technology (IUT) and her Ph.D. degree in Probability Theory and Stochastic Processes from SU, Iran. Since 2005 she has been with the Department of Mathematical Sciences, IUT, as an assistant professor. Her research interests include Stochastic Processes and Modelling, Stable Processes and Random Measures, Queueing Theory, Markov Processes, Matrix analytic methods, Large Deviations Theory, Statistical Inferences and Time Series Analysis.

Peter G. Taylor received a B. Sc.(Hons) and a Ph.D. in Applied Mathematics from the University of Adelaide in 1980 and 1987 respectively. In between, he spent time working for the Australian Public Service in Canberra. After periods at the Universities of Western Australia and Adelaide, he moved at the beginning of 2002 to the University of Melbourne. In January 2003, he took up a position as the inaugural Professor of Operations Research. He was Head of the Department of Mathematics and Statistics from 2005 until 2010.

Peter’s research interests lie in the fields of stochastic modelling and applied probability, with particular emphasis on applications in telecommunications, biological modelling, healthcare and disaster management. Recently he has become interested in the interaction of stochastic modelling with optimisation and optimal control under conditions of uncertainty. He is regularly invited to present plenary papers at international conferences. He has also acted on organising and programme committees for many conferences.

Peter is the Editor-in-Chief of ‘Stochastic Models’, and on the editorial boards of ‘Queueing Systems’, the ‘Journal of Applied Probability’ and ‘Advances in Applied Probability’. In 2008, Peter became one of the five trustees of the Applied Probability Trust. This trust, which is based in Sheffield, UK, is the body which publishes the Applied Probability journals plus The Mathematical Scientist and Spectrum.

From February 2006 to February 2008, Peter was Chair of the Australia and New Zealand Division of Industrial and Applied Mathematics (ANZIAM). He has recently completed a two-year term as President of the Australian Mathematical Society.

View full text