Skip to main content
SpringerLink
Log in
Book cover

International Colloquium on Theoretical Aspects of Computing

ICTAC 2017: Theoretical Aspects of Computing – ICTAC 2017 pp 192–209Cite as

Trace Relations and Logical Preservation for Continuous-Time Markov Decision Processes

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10580))

Abstract

Equivalence relations are widely used for comparing the behavior of stochastic systems. This paper introduces several variants of trace equivalence for continuous-time Markov decision processes (CTMDPs). These trace equivalences are obtained as a result of button pushing experiments with a black box model of CTMDP. For every class of CTMDP scheduler, a corresponding variant of trace equivalence has been introduced. We investigate the relationship among these trace equivalences and also compare them with bisimulation for CTMDPs. Finally, we prove that the properties specified using deterministic timed automaton (DTA) specifications and metric temporal logic (MTL) formulas are preserved under some of these trace equivalences.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    We only consider schedulers that make a decision as soon as a state is entered. Such schedulers are called early schedulers.

  2. 2.

    A CTMDP in which the delay time distribution per state visit is the same for all states.

  3. 3.

    Note that the exit rate of \(s_{3}\) for both \(\alpha \) and \(\beta \) is the same. This is in accordance with the assumption that the exit rates need to be the same for any state s with \(|Act(s)|>1\).

References

  1. Alur, R., Dill, D.L.: A theory of timed automata. Theor. Comput. Sci. 126(2), 183–235 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  2. Ash, R.B., Doleans-Dade, C.A.: Probability and Measure Theory. Academic Press, Cambridge (2000)

    MATH  Google Scholar 

  3. Baier, C., Haverkort, B.R., Hermanns, H., Katoen, J.-P.: Model-checking algorithms for continuous-time Markov chains. IEEE Trans. Softw. Eng. 29(6), 524–541 (2003)

    Article  MATH  Google Scholar 

  4. Baier, C., Hermanns, H.: Weak bisimulation for fully probabilistic processes. In: Grumberg, O. (ed.) CAV 1997. LNCS, vol. 1254, pp. 119–130. Springer, Heidelberg (1997). doi:10.1007/3-540-63166-6_14

    Chapter  Google Scholar 

  5. Baier, C., Katoen, J.-P.: Principles of Model Checking. MIT Press, Cambridge (2008)

    Google Scholar 

  6. Baier, C., Katoen, J.-P., Hermanns, H., Wolf, V.: Comparative branching-time semantics for Markov chains. Inf. Comput. 200(2), 149–214 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  7. Bernardo, M.: Non-bisimulation-based Markovian behavioral equivalences. J. Log. Algebr. Program. 72(1), 3–49 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  8. Bernardo, M.: Towards state space reduction based on t-lumpability-consistent relations. In: Thomas, N., Juiz, C. (eds.) EPEW 2008. LNCS, vol. 5261, pp. 64–78. Springer, Heidelberg (2008). doi:10.1007/978-3-540-87412-6_6

    Chapter  Google Scholar 

  9. Bernardo, M., De Nicola, R., Loreti, M.: Revisiting trace and testing equivalences for nondeterministic and probabilistic processes. LMCS, 10(1) (2014)

    Google Scholar 

  10. Bouyer, P.: From Qualitative to Quantitative Analysis of Timed Systems. Mémoire d’habilitation. Université Paris 7, Paris, France, January 2009

    Google Scholar 

  11. Chen, T., Han, T., Katoen, J.-P., Mereacre, A.: Observing continuous-time MDPs by 1-Clock timed automata. In: Delzanno, G., Potapov, I. (eds.) RP 2011. LNCS, vol. 6945, pp. 2–25. Springer, Heidelberg (2011). doi:10.1007/978-3-642-24288-5_2

    Chapter  Google Scholar 

  12. Chen, T., Han, T., Katoen, J.-P., Mereacre, A.: Quantitative model checking of continuous-time Markov chains against timed automata specifications. In: LICS, pp. 309–318. IEEE Computer Society (2009)

    Google Scholar 

  13. Desharnais, J., Gupta, V., Jagadeesan, R., Panangaden, P.: Weak bisimulation is sound and complete for pCTL\({}^{\text{* }}\). Inf. Comput. 208(2), 203–219 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  14. Fu, H.: Approximating acceptance probabilities of CTMC-paths on multi-clock deterministic timed automata. In: HSCC, pp. 323–332. ACM (2013)

    Google Scholar 

  15. Hermanns, H. (ed.): Interactive Markov Chains: And the Quest for Quantified Quality. LNCS, vol. 2428. Springer, Heidelberg (2002). doi:10.1007/3-540-45804-2

    MATH  Google Scholar 

  16. Huynh, D.T., Tian, L.: On some equivalence relations for probabilistic processes. Fundam. Inf. 17(3), 211–234 (1992)

    MathSciNet  MATH  Google Scholar 

  17. Jonsson, B., Larsen, K.G.: Specification and refinement of probabilistic processes. In: LICS, pp. 266–277. IEEE Computer Society (1991)

    Google Scholar 

  18. Jou, C.-C., Smolka, S.A.: Equivalences, congruences, and complete axiomatizations for probabilistic processes. In: Baeten, J.C.M., Klop, J.W. (eds.) CONCUR 1990. LNCS, vol. 458, pp. 367–383. Springer, Heidelberg (1990). doi:10.1007/BFb0039071

    Chapter  Google Scholar 

  19. Larsen, K.G., Skou, A.: Bisimulation through probabilistic testing. In: POPL, pp. 344–352 (1989)

    Google Scholar 

  20. Marsan, M.A., Balbo, G., Conte, G., Donatelli, S., Franceschinis, G.: Modelling with Generalized Stochastic Petri Nets, 1st edn. Wiley, Hoboken (1994)

    MATH  Google Scholar 

  21. Neuhäußer, M.R., Katoen, J.-P.: Bisimulation and logical preservation for continuous-time Markov decision processes. In: Caires, L., Vasconcelos, V.T. (eds.) CONCUR 2007. LNCS, vol. 4703, pp. 412–427. Springer, Heidelberg (2007). doi:10.1007/978-3-540-74407-8_28

    Chapter  Google Scholar 

  22. Neuhäußer, M.R., Stoelinga, M., Katoen, J.-P.: Delayed nondeterminism in continuous-time Markov decision processes. In: Alfaro, L. (ed.) FoSSaCS 2009. LNCS, vol. 5504, pp. 364–379. Springer, Heidelberg (2009). doi:10.1007/978-3-642-00596-1_26

    Chapter  Google Scholar 

  23. Ouaknine, J., Worrell, J.: Some recent results in metric temporal logic. In: Cassez, F., Jard, C. (eds.) FORMATS 2008. LNCS, vol. 5215, pp. 1–13. Springer, Heidelberg (2008). doi:10.1007/978-3-540-85778-5_1

    Chapter  Google Scholar 

  24. Parma, A., Segala, R.: Axiomatization of trace semantics for stochastic nondeterministic processes. In: QEST, pp. 294–303 (2004)

    Google Scholar 

  25. Philippou, A., Lee, I., Sokolsky, O.: Weak bisimulation for probabilistic systems. In: Palamidessi, C. (ed.) CONCUR 2000. LNCS, vol. 1877, pp. 334–349. Springer, Heidelberg (2000). doi:10.1007/3-540-44618-4_25

    Chapter  Google Scholar 

  26. Segala, R.: Modelling and verification of randomized distributed real time systems. Ph.D. thesis. MIT (1995)

    Google Scholar 

  27. Segala, R.: Testing probabilistic automata. In: Montanari, U., Sassone, V. (eds.) CONCUR 1996. LNCS, vol. 1119, pp. 299–314. Springer, Heidelberg (1996). doi:10.1007/3-540-61604-7_62

    Google Scholar 

  28. Segala, R., Lynch, N.A.: Probabilistic simulations for probabilistic processes. Nord. J. Comput. 2(2), 250–273 (1995)

    MathSciNet  MATH  Google Scholar 

  29. Sharma, A.: Weighted probabilistic equivalence preserves \(\upomega \)-regular properties. In: Schmitt, J.B. (ed.) MMB&DFT. LNCS. Springer, Heidelberg (2012). doi:10.1007/978-3-642-28540-0_9

    Google Scholar 

  30. Sharma, A., Katoen, J.-P.: Weighted lumpability on Markov chains. In: Clarke, E., Virbitskaite, I., Voronkov, A. (eds.) PSI 2011. LNCS, vol. 7162, pp. 322–339. Springer, Heidelberg (2012). doi:10.1007/978-3-642-29709-0_28

    Chapter  Google Scholar 

  31. Stoelinga, M., Vaandrager, F.: A testing scenario for probabilistic automata. In: Baeten, J.C.M., Lenstra, J.K., Parrow, J., Woeginger, G.J. (eds.) ICALP 2003. LNCS, vol. 2719, pp. 464–477. Springer, Heidelberg (2003). doi:10.1007/3-540-45061-0_38

    Chapter  Google Scholar 

  32. Wolf, V., Baier, C., Majster-Cederbaum, M.E.: Trace machines for observing continuous-time Markov chains. ENTCS 153(2), 259–277 (2006)

    Google Scholar 

  33. Wolf, V., Baier, C., Majster-Cederbaum, M.E.: Trace semantics for stochastic systems with nondeterminism. Electr. Notes Theo. Comput. Sci. 164(3), 187–204 (2006)

    Article  Google Scholar 

Download references

Acknowledgments

A special thanks goes to Michele Loreti for valuable discussions and suggestions.

Author information

Authors and Affiliations

  1. Department of Electrical Engineering and Computer Science, Indian Institute of Science Education and Research Bhopal, Bhopal, India

    Arpit Sharma

Authors
  1. Arpit Sharma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Arpit Sharma .

Editor information

Editors and Affiliations

  1. Vietnam National University, Hanoi, Vietnam

    Dang Van Hung

  2. University of New Mexico, Albuquerque, New Mexico, USA

    Deepak Kapur

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Sharma, A. (2017). Trace Relations and Logical Preservation for Continuous-Time Markov Decision Processes. In: Hung, D., Kapur, D. (eds) Theoretical Aspects of Computing – ICTAC 2017. ICTAC 2017. Lecture Notes in Computer Science(), vol 10580. Springer, Cham. https://doi.org/10.1007/978-3-319-67729-3_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-67729-3_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-67728-6

  • Online ISBN: 978-3-319-67729-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation