Skip to main content
SpringerLink
Log in

Numerical Method for Bounds Computations of Discrete-Time Markov Chains with Different State Spaces

  • Conference paper
Book cover Analytical and Stochastic Modeling Techniques and Applications (ASMTA 2009)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 5513))

  • 507 Accesses

Abstract

In this paper, we propose a numerical method for bounds computations of discrete-time Markov chains with different state spaces. This method is based on the necessary and sufficient conditions for the comparison of one-dimensional (also known as the point-wise comparison) of discrete-time Markov chains given in our previous work [3]. For achieving our objective, we proceed as follows. Firstly, we transform the comparison criterion under the form of a complete linear system of inequalities. Secondly, we use our implementation on Scilab software of Gamma-algorithm to determine the set of all possible bounds of a given Markov chain.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abu-Amsha, O., Vincent, J.M.: An Algorithm to Bound Functionals of Markov Chains with Large State Space. In: 4th INFORMS Telecommunications Conference, Boca Raton, March 8-11 (1998)

    Google Scholar 

  2. Ahmane, M., Ledoux, J., Truffet, L.: Criteria for the Comparison of Discrete-Time Markov Chains. In: The Thirteenth International Workshop on Matrices and Statistics in celebration of Ingram Olkin’s 80th Birthday, IWMS 2004, Bedlewo, Poland, August 18-21 (2004)

    Google Scholar 

  3. Ahmane, M., Ledoux, J., Truffet, L.: Positive invariance of polyherons and comparison of Markov reward models with different state spaces. In: POsitive Systems: Theory and Applications (POSTA 2006). LNCIS, vol. 341, pp. 153–160. Springer, Heidelberg (2006) (invited paper)

    Chapter  Google Scholar 

  4. Castillo, E., Jubete, F.: The -algorithm and some applications. University of Cantabria, Santander, Spain (March 2003)

    Google Scholar 

  5. Castillo, E., Jubete, F., Pruneda, E., Solares, C.: Obtaining simultaneous solutions of linear subsystems of inequalities and duals. Linear Algebra and its Applications 346, 131–154 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  6. Doisy, M.: A Coupling Technique For Comparison of Functions of Markov Processes. Appl. Math. Decis. Sci. 4, 39–64 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  7. Jubete, F.: El cono polidrico convexo. Su indenciaen el algebra lineal y la programacion no lineal. Editorial CIS, Santander, Spain (1991)

    Google Scholar 

  8. Kester, A.J.M.: Preservation of Cone Characterizing Properties in Markov Chains. Phd thesis, Univ. of Rochester, New York (1977)

    Google Scholar 

  9. Keilson, J., Kester, A.: Monotone Matrices and Monotone Markov Processes. Stochastic Process. Appl. 5, 231–241 (1977)

    Article  MathSciNet  MATH  Google Scholar 

  10. Kemeny, J.G., Snell, J.L.: Finite Markov Chains. Springer, Heidelberg (1976)

    MATH  Google Scholar 

  11. Kijima, M.: Markov Processes for Stochastic Modeling. Chapman-Hall, Boca Raton (1997)

    Book  MATH  Google Scholar 

  12. Ledoux, J., Truffet, L.: Markovian Bounds on Functions of Finite Markov Chains. Adv. in Appl. Probab. 33, 505–519 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  13. Li, H., Shaked, M.: Stochastic Convexity and Concavity of Markov Processes. Math. Oper. Res. 29, 477–493 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  14. Massey, W.A.: Stochastics Orderings for Markov Processes on Partially Ordered Spaces. Math. Oper. Res. 11, 350–367 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  15. Muller, A., Stoyan, D.: Comparison methods for stochastic models and risks. J. Wiley and Sons, Chichester (2002)

    MATH  Google Scholar 

  16. Pekergin, N.: Stochastic Performance Bounds by State Space Reduction. Performance Eval. 117, 36–37 (1999)

    MATH  Google Scholar 

  17. Truffet, L.: Geometrical Bounds on an Output Stream of a Queue in ATM Switch: Application to the Dimensioning Problem. In: Kouvatsos, D. (ed.) ATM Networks: Performance Modelling and Analysis, vol. 2. Chapman-Hall, London (1996)

    Google Scholar 

  18. Whitt, W.: Stochastic comparisons for non-Markov processes. Math. Oper. Res. 11, 608–618 (1986)

    Article  MathSciNet  MATH  Google Scholar 

  19. Ziegler, G.M.: Lectures notes en Polytopes. Graduate Texts in Mathematics. Springer, Berlin (1995)

    Book  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. SET Laboratory, University of Technology of Belfort-Montbéliard, 90010, Belfort, France

    Mourad Ahmane

  2. Ecole des Mines de Nantes, 4 rue Alfred Kastler, BP.20722, 44307, Nantes Cedex3, France

    Laurent Truffet

Authors
  1. Mourad Ahmane
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Laurent Truffet
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Faculty of Advanced Technology, University of Glamorgan, CF37 1DL, Pontypridd, UK

    Khalid Al-Begain

  2. Department TELIN, Ghent University, Sint-Pietersnieuwstraat 41, 9000, Gent, Belgium

    Dieter Fiems

  3. Budapest University of Technology and Economics, P.O. Box 91, 1521, Budapest, Hungary

    Gábor Horváth

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Ahmane, M., Truffet, L. (2009). Numerical Method for Bounds Computations of Discrete-Time Markov Chains with Different State Spaces. In: Al-Begain, K., Fiems, D., Horváth, G. (eds) Analytical and Stochastic Modeling Techniques and Applications. ASMTA 2009. Lecture Notes in Computer Science, vol 5513. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02205-0_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-02205-0_22

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-02204-3

  • Online ISBN: 978-3-642-02205-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation