Hassan Hatefi
Skip Abstract Section
Abstract
Abstract
Costs and rewards are important tools for analysing quantitative aspects of models like energy consumption and costs of maintenance and repair. Under the assumption of transient costs, this paper considers the computation of expected cost-bounded rewards and cost-bounded reachability for Markov automata and Markov games. We provide a fixed point characterization of this class of properties under early schedulers. Additionally, we give a transformation to expected time-bounded rewards and time-bounded reachability, which can be computed by available algorithms. We prove the correctness of the transformation and show its effectiveness on a number of Markov automata case studies.
- ADD99 Ash RB, Doléans-Dade CA (1999) Probability & measure theory, 2nd edn. Academic Press, New YorkGoogle Scholar
- AHK03 Andova S, Hermanns H, Katoen JP (2003) Discrete-time rewards model-checked. In: Int’l conf. on formal modeling and analysis of timed systems (FORMATS). Lecture notes in computer science. vol 2791. Springer, Berlin, pp 88–104Google Scholar
- BCS10 A rigorous, compositional, and extensible framework for dynamic fault tree analysisIEEE Tran Depend Sec Comput20107212814310.1109/TDSC.2009.45Google ScholarDigital Library
- BDF81 Sequencing tasks with exponential service times to minimize the expected flow time or makespanJ ACM198128110011360318510.1145/322234.3222420454.68016Google ScholarDigital Library
- BFH+14 Braitling B, María Ferrer Fioriti L, Hatefi H, Wimmer R, Becker B, Hermanns H (2014) MeGARA: menu-based game abstraction refinement for Markov automata. In: Bertrand N, Bertolussi L (eds) Int’l workshop on quantitative aspects of programming languages and systems (QAPL). In: Electronic proceedings in theoretical computer science, vol 154. Open Publishing Association, Grenoble, pp 48–63Google Scholar
- BFH+15 Braitling B, María Ferrer FL, Hatefi H, Wimmer R, Hermanns H, Becker B (2015) Abstraction-based computation of reward measures for Markov automata. In: Int’l conf. on verification, model checking, and abstract interpretation (VMCAI). Lecture Notes in Computer Science, vol 8931. Springer, Berlin, pp 172–189Google Scholar
- BFK+13 Continuous-time stochastic games with time-bounded reachabilityInf Comput20132244670301645810.1016/j.ic.2013.01.0011264.91016Google ScholarDigital Library
- BHHK00 Baier C, Haverkort BR, Hermanns H, Katoen JP (2000) On the logical characterisation of performability properties. In: Int’l colloquium on automata, languages, and programming (ICALP). In: Lecture notes in computer science, vol 1853. Springer, Berlin, pp 780–792Google Scholar
- BHHK08 Baier C, Haverkort BR, Hermanns H, Katoen JP (2008) Reachability in continuous-time Markov reward decision processes. In: Logic and automata: history and perspectives. Honor of Wolfgang Thomas. Texts in logic and games, vol 2. Amsterdam University Press, Amsterdam, pp 53–72Google Scholar
- BHHK15 Butkova Y, Hatefi H, Hermanns H, Krcál J (2015) Optimal continuous time Markov decisions. In: Finkbeiner B, Pu G, Zhang L (eds) Int’l symp. on automated technology for verification and analysis (ATVA). Lecture notes in computer science, vol 9364. Springer, Shanghai, pp 166–182Google Scholar
- BK08 Baier C, Katoen J-P (2008) Principles of model checking. The MIT Press, MassachusettsGoogle Scholar
- BS11 Numerical analysis of continuous time Markov decision processes over finite horizonsComput Oper Res2011383651659272616110.1016/j.cor.2010.08.0111206.90206Google ScholarDigital Library
- CKKP05 Cloth L, Katoen J-P, Khattri M, Pulungan R (2005) Model checking Markov reward models with impulse rewards. In: Int’l conf. on dependable systems and networks (DSN). IEEE Computer Society, New York, pp 722–731Google Scholar
- EHKZ13 Eisentraut C, Hermanns H, Katoen J-P, Zhang L (2013) A semantics for every GSPN. In: Proc. of petri nets. Lecture notes in computer science, vol 7927. Springer, Berlin, pp 90–109Google Scholar
- EHZ10 Eisentraut C, Hermanns H, Zhang L (2010) On probabilistic automata in continuous time. In: Annual IEEE symp. on logic in computer science (LICS). IEEE Computer Society, New York, pp 342–351Google Scholar
- Fu14a Fu H (2014) Maximal cost-bounded reachability probability on continuous-time Markov decision processes. In: Int’l conf. on foundations of software science and computation structures (FoSSaCS). Lecture notes in computer science, vol 8412. Springer, Berlin, pp 73–87Google Scholar
- Fu14b Fu H (2014) Verifying probabilistic systems: new algorithms and complexity results. PhD thesis, RWTH Aachen UniversityGoogle Scholar
- GBK16 Gburek D, Baier C, Klüppelholz S (2016) Composition of stochastic transition systems based on spans and couplings. In: 43rd international colloquium on automata, languages, and programming, ICALP 2016, July 11–15, 2016, Rome, Italy, pp 102:1–102:15Google Scholar
- GHH+13 Guck D, Hatefi H, Hermanns H, Katoen J.-P. Timmer M (2013) Modelling, reduction and analysis of Markov automata. In: Int’l conf. on quantitative evaluation of systems (QEST). Lecture notes in computer science, vol 8054. Springer, Berlin, pp 55–71Google Scholar
- GHH+14 Guck D, Hatefi H, Hermanns H, Katoen JP, Timmer M (2014) Analysis of timed and long-run objectives for Markov automata. Logical Methods Comput Sci 10(3)Google Scholar
- GHKN12 Guck D, Han T, Katoen JP, Neuhäußer MR (2012) Quantitative timed analysis of interactive Markov chains. In: NASA formal methods symposium (NFM). Lecture notes in computer science, vol 7226. Springer, Berlin, pp 8–23Google Scholar
- GTH+14 Guck D, Timmer M, Hatefi H, Ruijters E, Stoelinga M (2014) Modelling and analysis of Markov reward automata. In: Int’l symp. on automated technology for verification and analysis (ATVA). Lecture notes in computer science, vol 8837. Springer, Berlin, pp 168–184Google Scholar
- HBW+15 Hatefi H, Braitling B, Wimmer R, María Ferrer Fioriti L, Hermanns H, Becker B (2015) Cost vs. time in stochastic games and Markov automata. In: Li X, Liu Z, Yi W (eds) int’l symp. on dependable software engineering: theory, tools and applications (SETTA). Lecture notes in computer science, vol 9409. Springer, Nanjing, pp 19–34Google Scholar
- Her02 Hermanns H (2002) Interactive Markov chains: the quest for quantified quality. In: Lecture notes in computer science, vol 2428. Springer, BerlinGoogle Scholar
- HH12 Hatefi H, Hermanns H (2012) Model checking algorithms for Markov automata. ECEASST 53Google Scholar
- Joh08 Johr S (2008) Model checking compositional Markov systems. PhD thesis, Saarland University, SaarbrückenGoogle Scholar
- Mil68 Finite state continuous time Markov decision processes with a finite planning horizonSIAM J Control19686226628024115310.1137/03060200162.23302Google ScholarCross Ref
- Neu10 Neuhäußer MR (2010) Model checking nondeterministic and randomly timed systems. PhD thesis, RWTH Aachen University and University of TwenteGoogle Scholar
- NS03 Stochastic games and applications2003BerlinSpringer1093.91005Google ScholarCross Ref
- NZ10 Neuhäußer MR, Zhang L (2010) Time-bounded reachability probabilities in continuous-time Markov decision processes. In: Int’l conf. on quantitative evaluation of systems (QEST). IEEE Computer Society, New York, pp 209–218Google Scholar
- QQP01 Stochastic modeling of a power-managed system-construction and optimizationIEEE Trans CAD Integrat Circ Syst200120101200121710.1109/43.952737Google ScholarDigital Library
- SBGM00 Simunic T, Benini L, Glynn P.W, De Micheli G (2000) Dynamic power management for portable systems. In: Annual int’l conf. on mobile computing and networking (MOBICOM), Boston, MA, USA, August, pp 11–19Google Scholar
- Seg95 Segala R (1995) A compositional trace-based semantics for probabilistic automata. In: Int’l conf. on concurrency theory (CONCUR). Lecture notes in computer science, vol 962. Springer, Berlin, pp 234–248Google Scholar
- Sha53 Shapley LS (1953) Stochastic games. Proc Natl Acad Sci USA 39(10):1095Google Scholar
- Tar55 A lattice-theoretical fixpoint theorem and its applicationsPac J Math1955522853097437610.2140/pjm.1955.5.2850064.26004Google ScholarCross Ref
- TKvdPS12 Timmer M, Katoen J-P van de Pol J, Stoelinga M (2012) Efficient modelling and generation of Markov automata. In: Int’l conf. on concurrency theory (CONCUR). Lecture notes in computer science, vol 7454. Springer, Berlin, pp 364–379Google Scholar
- TvdPS13 Timmer M, van de Pol J, Stoelinga M (2013) Confluence reduction for Markov automata. In: Int’l conf. on formal modeling and analysis of timed systems (FORMATS). Lecture notes in computer science, vol 8053. Springer, Berlin, pp 243–257Google Scholar
- WJ06 Wolovick N, Johr S (2006) A characterization of meaningful schedulers for continuous-time Markov decision processes. In: Asarin E, Bouyer P (eds) Int’l conf. on formal modeling and analysis of timed systems (FORMATS). Lecture notes in computer science, vol 4202. Springer, Paris, pp 352–367Google Scholar
Index Terms
- Cost vs. time in stochastic games and Markov automata
Index terms have been assigned to the content through auto-classification.
Recommendations
Cost vs. Time in Stochastic Games and Markov Automata
SETTA 2015: Proceedings of the First International Symposium on Dependable Software Engineering: Theories, Tools, and Applications - Volume 9409Costs and rewards are important tools for analysing quantitative aspects of models like energy consumption and costs of maintenance and repair. Under the assumption of transient costs, this paper considers the computation of expected cost-bounded ...
Recursive Markov Decision Processes and Recursive Stochastic Games
We introduce Recursive Markov Decision Processes (RMDPs) and Recursive Simple Stochastic Games (RSSGs), which are classes of (finitely presented) countable-state MDPs and zero-sum turn-based (perfect information) stochastic games. They extend standard ...
Total Reward Stochastic Games and Sensitive Average Reward Strategies
In this paper, total reward stochastic games are surveyed. Total reward games are motivated as a refinement of average reward games. The total reward is defined as the limiting average of the partial sums of the stream of payoffs. It is shown that total ...
Comments