Compositional reversed Markov processes, with applications to G-networks

https://doi.org/10.1016/j.peva.2004.02.002Get rights and content

Abstract

Stochastic networks defined by a collection of cooperating agents are solved for their equilibrium state probability distribution by a new compositional method. The agents are processes formalised in a Markovian Process Algebra, which enables the reversed stationary Markov process of a cooperation to be determined symbolically under appropriate conditions. From the reversed process, a separable (compositional) solution follows immediately for the equilibrium state probabilities. The well-known solutions for networks of queues (Jackson’s theorem) and G-networks (with both positive and negative customers) can be obtained simply by this method. Here, the reversed processes, and hence product-form solutions, are derived for more general cooperations, focussing on G-networks with chains of triggers and generalised resets, which have some quite distinct properties from those proposed recently. The methodology’s principal advantage is its potential for mechanisation and symbolic implementation; many equilibrium solutions, both new and derived elsewhere by customised methods, have emerged directly from the compositional approach. As further examples, we consider a known type of fork-join network and a queueing network with batch arrivals.

Introduction

The existence of separable (compositional) stationary state probability distributions in certain queueing networks has greatly facilitated tractable models of performance over the past three decades; see [1], [5], [7], [14], [18], [19], to mention a few of many sources. However, more general stochastic networks than just networks of queues are separable at equilibrium, as in [2], [3], [6], [16], [17], for example. These have typically been derived in a rather ad hoc way; guessing that such a solution exists, then verifying that the Kolmogorov equations of the defining Markov chain are satisfied and appealing to uniqueness. A more structured way uses properties of local balance or quasi-reversibility and verifies a set of traffic equations; see [3] for an excellent survey. We approach the problem in a different, hierarchical way, by seeking the reversed process of the Markov chain [19] in terms of the reversed processes of its sub-chains. From a reversed process, a separable solution for the equilibrium state probabilities follows immediately.

The formalism we use for this hierarchical analysis is PEPA [15], a Markovian Process Algebra (MPA), which has an appropriate recursive structure. The determination of the reversed process of a certain type of cooperation between two agents is based on the Reversed Compound Agent Theorem (RCAT) of [11]. This methodology is extended in this paper by considering multiple cooperations to facilitate the modelling of chains of immediate transitions in a set of two or more cooperating processes. It is also applied to approximating non-separable solutions, by finding perturbations to a cooperation’s specifications that render it compositional.

In Section 2, the salient properties of reversed processes, our MPA-based formulation and RCAT are reviewed. The methodology is applied to G-networks with triggers and resets in Section 3 and these results are extended with the introduction of negative, propagating triggers [3] and more general resets that can also propagate. These resets are quite distinct from those of [8]. Section 4 considers further applications, including a type of fork-join network (leading to the result on ‘positive triggers’ of [3]) and queueing networks with batches. The paper concludes in Section 5, where we assess the significance of this work and outline some directions for further research. The full statement of RCAT is given in Appendix A.

The methodology unifies many existing product-forms, derived elsewhere over many years in various, customised ways. Moreover, it generates new ones (to the author’s best knowledge), such as the aforementioned G-networks with generalised resets. The RCAT-based proofs of product-forms given here are entirely novel to this paper, providing a serious alternative, or complement, to current teaching and understanding of separable Markov processes at equilibrium. The advantages for mechanisation and symbolic evaluation are clear, potentially leading to the automatic generation of steady state theorems by computer.

Section snippets

Process algebraic formalism

We consider (continuous time) Markov chains that are composed of simpler chains in a particular way. In order to formulate these chains and the way they interact rigorously, we use process algebraic concepts. The agents of a Markovian Process Algebra, in which all time delays are exponential random variables, describe syntactically Markov chains with generic patterns in their generator matrices, given by the rates at which an agent’s actions occur. For example, a single ‘arrival’ action can

Gelenbe networks and extensions

A simple G-queue is a single M/M/1 queue with negative customers, which may be represented by a normal M/M/1 queue with arrival rate λ+ (that of the positive arrivals) and service rate μ+λ, where μ is the usual rate of service and λ the negative arrival rate. The reversed queue, with aggregated arrival streams, is then the same M/M/1 queue, with the same arrival and service rates, since it is reversible. In a G-network (network of G-queues), each G-queue may have multiple positive and

A fork-join network

Consider a fork-join network of M>1 parallel servers, where tasks arrive and spawn one sub-task into each of the servers’ queues, i.e. arrivals are synchronised. The sub-tasks are processed independently and stored in a buffer associated with their own server. Tasks are then reconstituted at certain instants by an asynchronous process that collects n sub-tasks from each of the M buffers, where n is equal to the minimum buffer occupancy, i.e. at least one buffer is left empty. Such a system is

Conclusion

The use of Markovian Process Algebra and the Reversed Compound Agent Theorem of [11] introduces a new, compositional methodology for deriving the equilibrium state probabilities in separable Markov processes. This approach does not require balance equations to be solved but instead determines the reversed process whence a simple, separable solution ensues. The origins of RCAT and the methodology based on it lie in a combination of MPA and the theory of reversed stationary Markov processes. In

Peter Harrison is currently a Professor of Computing Science at Imperial College, London where he became a lecturer in 1983. He graduated at Christ’s College, Cambridge as a Wrangler in Mathematics in 1972 and went on to gain Distinction in Part III of the Mathematical Tripos in 1973, winning the Mayhew prize for Applied Mathematics. He obtained his Ph.D. in Computing Science at Imperial College in 1979. He has researched into stochastic performance modelling and algebraic program

References (19)

There are more references available in the full text version of this article.

Cited by (51)

  • Whittle-networks with signals

    2017, Performance Evaluation
    Citation Excerpt :

    As negative customers lead to customer deletions, the original description of quasi-reversibility by arrivals and departures does not hold anymore and a new version had been proposed by Chao and his co-authors in [13]. A different approach, based on Stochastic Process Algebra, was proposed by Harrison [14,15]. The main results (CAT and RCAT theorems and their extensions [14–16]) give some sufficient conditions for product form stationary distributions.

  • Separable solutions for Markov processes in random environments

    2013, European Journal of Operational Research
  • Product-form results for two-station networks with shared resources

    2012, Performance Evaluation
    Citation Excerpt :

    Reversed Compound Agent Theorem (RCAT) The RCAT theorem as presented in [39,33,40] generate the reversed process and the product form. In the most general conditions in [41], an element of local state-dependency is allowed for the active shared rates in the systems.

View all citing articles on Scopus

Peter Harrison is currently a Professor of Computing Science at Imperial College, London where he became a lecturer in 1983. He graduated at Christ’s College, Cambridge as a Wrangler in Mathematics in 1972 and went on to gain Distinction in Part III of the Mathematical Tripos in 1973, winning the Mayhew prize for Applied Mathematics. He obtained his Ph.D. in Computing Science at Imperial College in 1979. He has researched into stochastic performance modelling and algebraic program transformation for some 20 years, visiting IBM Research Centers during two summers. He has written two books, over 150 research papers published in his research areas and held a series of research grants, both national and international. The results of his research have been exploited extensively in industry, forming an integral part of commercial products such as Metron’s Athene client-server capacity planning tool. He has taught a range of subjects at undergraduate and graduate level, including Operating Systems: Theory and Practice, Functional Programming, Parallel Algorithms and Performance Analysis.

View full text