Skip to main content
SpringerLink
Log in

Numerical methods for controlled and uncontrolled multiplexing and queueing systems

  • Articles
  • Published:
Queueing Systems Aims and scope Submit manuscript

Abstract

We deal with a very useful numerical method for both controlled and uncontrolled queuing and multiplexing type systems. The basic idea starts with a heavy traffic approximation, but it is shown that the results are very good even when working far from the heavy traffic regime. The underlying numerical method is a version of what is known as the Markov chain approximation method. It is a powerful methodology for controlled and uncontrolled stochastic systems, which can be approximated by diffusion or reflected diffusion type systems, and has been used with success on many other problems in stochastic control. We give a complete development of the relevant details, with an emphasis on multiplexing and particular queueing systems. The approximating process is a controlled or uncontrolled Markov chain which retains certain essential features of the original problem. This problem is generally substantially simpler than the original physical problem, and there are associated convergence theorems. The non-classical associated ergodic cost problem is derived, and put into a form such that reliable and good numerical algorithms, based on multigrid type ideas, can be used. Data for both controlled and uncontrolled problems shows the value of the method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. M. Akian, Résolution numéricque d'équations d'Hamilton-Jacobi-Bellman au moyen d'algorithmes multigrilles et d'iterations sur les politiques, in:8th Conf. on Analysis and Optimization of Systems, Antibes, France (INRIA, 1988).

    Google Scholar 

  2. M. Akian, Méthodes multigrilles en controle stochastique, PhD thesis, University of Paris (1990).

  3. D. Anick, D. Mitra and M.M. Sondhi, Stochastic theory of a data handling system with multiple sources, Bell Syst. Techn. J. 61 (1982) 1971–1894.

    Google Scholar 

  4. D.P. Bertsekas,Dynamic Programming: Deterministic and Stochastic Models (Prentice-Hall, Englewood Cliffs, NJ, 1987).

    Google Scholar 

  5. D.P. Bertsekas and D.A. Castañon, Adaptive aggregation methods for infinite horizon dynamic programming, IEEE Trans. Auto. Contr. AC-34 (1989) 589–598.

    Google Scholar 

  6. K.-L. Chung,Markov Chains with Stationary Transition Probabilities (Springer, Berlin, 1960).

    Google Scholar 

  7. J. Dai, Steady state analysis of reflected Brownian motions: characterization, numerical methods and queueing applications, PhD thesis, Operations Research Dept., Stanford University (1990).

  8. A.I. Elwalid and D. Mitra, Fluid models for the analysis and design of statistical multiplexing with loss priorities on multiple classes of bursty traffic, in:Proc. IEEE INFOCOM'92 (IEEE Press, New York, 1992) pp. 415–425.

    Google Scholar 

  9. J.M. Harrison and V. Nguyen, The qnet method for two moment analysis of open queueing networks, Queueing Syst. 61 (1990) 1–32.

    Google Scholar 

  10. D.L. Iglehart and W. Whitt, Multiple channel queues in heavy traffic, Adv. Appl. Prob. 2 (1970) 150–177.

    Google Scholar 

  11. C. Knessl and J.A. Morrison, Heavy traffic analysis of a data handling system with multiple sources, SIAM J. Appl. Math. 51 (1991) 187–213.

    Google Scholar 

  12. H.J. Kushner,Probability Methods for Approximations in Stochastic Control and for Elliptic Equations (Academic Press, New York, 1977).

    Google Scholar 

  13. H.J. Kushner, Robustness and approximation of escape times and large deviations estimates for systems with small noise effects, SIAM J. Appl. Math. 44 (1984) 160–182.

    Google Scholar 

  14. H.J. Kushner, Numerical methods for stoachastic control problems in continuous time, SIAM J. Contr. Optim. 28 (1990) 999–1048.

    Google Scholar 

  15. H.J. Kushner and P. Dupuis,Numerical Methods for Stochastic Control Problems in Continuous Time (Springer, New York and Berlin, 1992).

    Google Scholar 

  16. H.J. Kushner and L.F. Martins, Heavy traffic analysis of a data transmission system with many independent sources, Technical report, Brown University, Lefschetz Center for Dynamical Systems, LCDS#91-12 (1991), SIAM J. Appl. Math., to appear.

  17. H.J. Kushner and L.F. Martins, Numerical methods for stochastic singular control problems, SIAM J. Contr. Optim. 29 (1991) 1443–1475.

    Google Scholar 

  18. H.J. Kushner and L.F. Martins, Numerical methods for controlled and uncontrolled multiplexing and queueing systems, Technical report, Brown University, Lefschetz Center for Dynamical Systems (1993).

  19. H.J. Kushner and K.M. Ramachandran, Optimal and approximately optimal control policies for queues in heavy traffic, SIAM J. Contr. Optim. 27 (1989) 1293–1318.

    Google Scholar 

  20. H.J. Kushner and J. Yang, Numerical methods for controlled routing in large trunk line systems, Technical report, Brown University, Lefschetz Center for Dynamical Systems (1992), to appear in ORSA J. Comp.

  21. J.P. Lehoczky and S.E. Shreve, Absolutely continuous and singular stochastic control, Stochastics 17 (1986) 91–110.

    Google Scholar 

  22. L.F. Martins and H.J. Kushner, Routing and singular control for queueing networks in heavy traffic, SIAM J. Contr. Optim. 28 (1990) 1209–1233.

    Google Scholar 

  23. D. Mitra, Stochastic theory of a fluid model of producers and consumers coupled by buffer, Adv. Appl. Prob. 20 (1988) 646–676.

    Google Scholar 

  24. M.L. Puterman, Markov decision processes, in:Stochastic models, Vol. 2, eds. D.P. Heyman and M.J. Sobel (North-Holland, Amsterdam, 1991) chapter 8.

    Google Scholar 

  25. M.R. Reiman, Asymptotically optimal trunk reservation for large trunk groups, in:Proc. 28th Conf. on Decision and Control, New York (1989).

  26. M.R. Reiman, Optimal trunk reservations for a critically loaded line, in:Teletraffic and Datatraffic in a Period of Change, eds. A. Jensen and V.B. Jensen (North-Holland Elsevier, 1991).

  27. M. Taksar, Average optimal singular control and a related stopping problem, Math. Oper. Res. 10 (1985) 63–81.

    Google Scholar 

  28. L.M. Wein, Optimal control of a two station Brownian network, Math. Oper. Res. 15 (1990) 215–242.

    Google Scholar 

  29. D.J. White, Dynamic programming, Markov chains and the method of successive approximations, J. Math. Anal. Appl. 6 (1963) 373–376.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Division of Applied Mathematics, Brown University, 02912, Providence, RI, USA

    Harold J. Kushner

  2. Mathematics Department, Worcester Polytechnic Institute, 01609, Worcester, MA, USA

    Luiz Felipe Martins

Authors
  1. Harold J. Kushner
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Luiz Felipe Martins
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

Supported by ARO contract DAAL-03-92-G-0115, AFOSR contract F49620-92-J-0081, and DARPA contract AFOSR-91-0375.

Formerly at Brown University. Supported by DARPA contract AFOSR-91-0375.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kushner, H.J., Martins, L.F. Numerical methods for controlled and uncontrolled multiplexing and queueing systems. Queueing Syst 16, 241–285 (1994). https://doi.org/10.1007/BF01158957

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01158957

Keywords

Search

Navigation