Skip to main content
SpringerLink
Log in

Random walk, birth-and-death process and their fluid approximations: absorbing case

  • Original Article
  • Published:
Mathematical Methods of Operations Research Aims and scope Submit manuscript

Abstract

Fluid models are used to study functionals of the underlying random processes. Instead of analysing the trajectories, we investigate algebraic equations of the dynamic programming type which turn out to be discrete analogs of the corresponding differential equations. This analysis makes it possible to estimate the accuracy of approximation. Since the algebraic equations are the same for random walks and continuous time birth-and-death processes, we study the two cases in parallel. Several illustrative examples are also presented.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Altman E (1999) Constrained Markov decision processes. Chapman and Hall/CRC, Boca Raton

    MATH  Google Scholar 

  • Altman E, Jimenez T, Koole G (2001) On the comparison of queueing systems with their fluid limits. Prob Eng Inform Sci 15: 165–178

    Article  MATH  MathSciNet  Google Scholar 

  • Avrachenkov K, Ayesta U, Piunovskiy A (2005) Optimal choice of the buffer size in the Internet routers. In: Proc. of CDC’05, 44-th IEEE conf. on Decision and Control, Spain

  • Bäuerle N (2002) Optimal control of queueing networks: an approach via fluid models. Adv Appl Prob 34: 313–328

    Article  MATH  Google Scholar 

  • Bremaud P (1999) Markov chains: Gibbs fields, Monte Carlo simulation, and Queues. Springer, New York

    MATH  Google Scholar 

  • Clancy D, Piunovskiy A (2005) An explicit optimal isolation policy for a deterministic epidemic model. Appl Math Comput 163: 1109–1121

    Article  MATH  MathSciNet  Google Scholar 

  • Dai JG (1995) On positive Harris recurrence of multiclass queueing networks: a unified approach via fluid limit models. Ann Appl Prob 5: 49–77

    Article  MATH  Google Scholar 

  • Dai JG, Meyn SP (1995) Stability and convergence of moments for multiclass queueing networks via fluid limit models. IEEE Trans Autom Control 40: 1889–1904

    Article  MATH  MathSciNet  Google Scholar 

  • Ethier SN, Kurtz TG (1986) Markov processes: characterization and convergence. Wiley, New York

    MATH  Google Scholar 

  • Foss S, Kovalevskii A (1999) A stability criterion via fluid limits and its application to a Polling system. Queueing Syst 32: 131–168

    Article  MATH  MathSciNet  Google Scholar 

  • Gajrat AS, Hordijk A, Malyshev VA, Spieksma FM (1997) Fluid approximations of Markov decision chains. Markov Process Related Fields 3: 129–150

    MATH  MathSciNet  Google Scholar 

  • Gajrat AS, Hordijk A (2005) On the structure of the optimal server control for fluid networks. Math Meth Oper Res 62: 55–75

    Article  MATH  MathSciNet  Google Scholar 

  • Hespanha JP, Bohacek S, Obraczka K, Lee J (2001) Hybrid modeling of TCP congestion control. In: Di Benedetto MD, Sangiovanni-Vincentelli A (eds) In hybrid systems: computation and control. Springer, Berlin, pp 291–304

    Chapter  Google Scholar 

  • Hordijk A, Spieksma F (1992) On ergodicity and recurrence properties of a Markov chain with an application to an open Jackson network. Adv Appl Prob 24: 343–376

    Article  MATH  MathSciNet  Google Scholar 

  • Kemeny JG, Snell JL, Knapp AW (1976) Denumerable Markov chains. Springer, New York

    MATH  Google Scholar 

  • Knessl Ch, Tier Ch (2001) A simple fluid model for servicing priority traffic. IEEE Trans Autom Control 46: 909–914

    Article  MATH  MathSciNet  Google Scholar 

  • Korobeinikov A (2007) Global properties of infectious disease models with nonlinear incidence. Bull Math Biol 69: 1871–1886

    Article  MATH  MathSciNet  Google Scholar 

  • Li QL, Zhao YQ (2005) Block-structured fluid queues driven by QBD processes. Stoch Anal Appl 23: 1087–1112

    Article  MATH  MathSciNet  Google Scholar 

  • Maglaras C (2006) Revenue management for a multiclass single-server queue via a fluid model analysis. Oper Res 54: 914–932

    Article  MATH  MathSciNet  Google Scholar 

  • Mandelbaum A, Pats G (1994) State-dependent queues: approximations and applications. In: Kelly F, Williams R (eds) Stochastic Networks, Proc. of the IMA, vol 71. Springer, New York, pp 239–282

  • Mandelbaum A, Massey WA, Reiman MI (1998) Strong approximations for Markovian service networks. Queueing Syst 30: 149–201

    Article  MATH  MathSciNet  Google Scholar 

  • Meyn SP, Tweedie RL (1993) Stability of Markovian processes III: Foster-Lyapunov criteria for continuous-time processes. Adv Appl Prob 25: 518–548

    Article  MATH  MathSciNet  Google Scholar 

  • Mitra D (1988) Stochastic theory of a fluid model of producers and consumers coupled by a buffer. Adv Appl Prob 20: 646–676

    Article  MATH  Google Scholar 

  • Piunovskiy A (2004) Multicriteria impulsive control of jump Markov processes. Math Meth Oper Res 60: 125–144

    Article  MATH  MathSciNet  Google Scholar 

  • Puterman M (1994) Markov decision processes. Wiley, New York

    Book  MATH  Google Scholar 

  • Trenoguine V (1985) Analyse Fonctionnelle. Mir, Moscow (French)

  • Warburton RDH (2004) An exact analytical solution to the production inventory control problem. Int J Prod Econ 92: 81–96

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematical Sciences, The University of Liverpool, Liverpool, L69 7ZL, UK

    A. B. Piunovskiy

Authors
  1. A. B. Piunovskiy
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to A. B. Piunovskiy.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Piunovskiy, A.B. Random walk, birth-and-death process and their fluid approximations: absorbing case. Math Meth Oper Res 70, 285–312 (2009). https://doi.org/10.1007/s00186-008-0269-y

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00186-008-0269-y

Keywords

Search

Navigation