ABSTRACT
This work studies computation tree logic (CTL) model checking for finite-state Markov decision processes (MDPs) over the space of their distributions. Instead of investigating properties over states of the MDP, as encoded by formulae in standard probabilistic CTL (PCTL), the focus of this work is on the associated transition system, which is induced by the MDP, and on its dynamics over the (transient) MDP distributions. CTL is thus used to specify properties over the space of distributions, and is shown to provide an alternative way to express probabilistic specifications or requirements over the given MDP. We discuss the distinctive semantics of CTL formulae over distribution spaces, compare them to existing non-branching logics that reason on probability distributions, and juxtapose them to traditional PCTL specifications. We then propose reachability-based CTL model checking algorithms over distribution spaces, as well as computationally tractable, sampling-based procedures for computing the relevant reachable sets: it is in particular shown that the satisfaction set of the CTL specification can be soundly under-approximated by the union of convex polytopes. Case studies display the scalability of these procedures to large MDPs.
- [n. d.]. MOSEK Software. https://www.mosek.com/Google Scholar
- Manindra Agrawal, Sundararaman Akshay, Blaise Genest, and PS Thiagarajan. 2015. Approximate verification of the symbolic dynamics of Markov chains. J. ACM 62, 1 (2015), 1–34.Google ScholarDigital Library
- S Akshay, Timos Antonopoulos, Joël Ouaknine, and James Worrell. 2015. Reachability problems for Markov chains. Inform. Process. Lett. 115, 2 (2015), 155–158.Google ScholarDigital Library
- S Akshay, Krishnendu Chatterjee, Tobias Meggendorfer, and Đorđe Žikelić. 2023. MDPs as distribution transformers: affine invariant synthesis for safety objectives. In International Conference on Computer Aided Verification. 86–112.Google Scholar
- S Akshay, Blaise Genest, and Nikhil Vyas. 2018. Distribution-based objectives for Markov Decision Processes. In 33rd Annual ACM/IEEE Symposium on Logic in Computer Science. 36–45.Google ScholarDigital Library
- Eitan Altman. 1999. Constrained Markov Decision Processes: Stochastic Modeling. Routledge.Google Scholar
- Christel Baier and Joost-Pieter Katoen. 2008. Principles of Model Checking. MIT press.Google ScholarDigital Library
- C Bradford Barber, David P Dobkin, and Hannu Huhdanpaa. 1996. The quickhull algorithm for convex hulls. ACM Transactions on Mathematical Software (TOMS) 22, 4 (1996), 469–483.Google ScholarDigital Library
- Daniele Beauquier, Alexander Rabinovich, and Anatol Slissenko. 2002. A logic of probability with decidable model-checking. In International Workshop on Computer Science Logic. 306–321.Google ScholarCross Ref
- Calin Belta, Boyan Yordanov, and Ebru Aydin Gol. 2017. Formal Methods for Discrete-time Dynamical Systems. Springer.Google Scholar
- Rohit Chadha, Vijay Anand Korthikanti, Mahesh Viswanathan, Gul Agha, and YoungMin Kwon. 2011. Model checking MDPs with a unique compact invariant set of distributions. In 8th International Conference on Quantitative Evaluation of Systems. 121–130.Google ScholarDigital Library
- Edmund M Clarke and E Allen Emerson. 1981. Design and synthesis of synchronization skeletons using branching time temporal logic. In Workshop on Logic of Programs. 52–71.Google ScholarDigital Library
- Edmund M. Clarke, E Allen Emerson, and A Prasad Sistla. 1986. Automatic verification of finite-state concurrent systems using temporal logic specifications. ACM Transactions on Programming Languages and Systems 8, 2 (1986), 244–263.Google ScholarDigital Library
- Giacomo Como and Fabio Fagnani. 2015. Robustness of large-scale stochastic matrices to localized perturbations. IEEE Transactions on Network Science and Engineering 2, 2 (2015), 53–64.Google ScholarCross Ref
- Christian Dehnert, Sebastian Junges, Joost-Pieter Katoen, and Matthias Volk. 2017. A STORM is coming: A modern probabilistic model checker. In International Conference on Computer Aided Verification. 592–600.Google ScholarCross Ref
- Ioannis Z Emiris and Vissarion Fisikopoulos. 2018. Practical polytope volume approximation. ACM Trans. Math. Software 44, 4 (2018), 1–21.Google ScholarDigital Library
- Yuan Feng and Lijun Zhang. 2014. When equivalence and bisimulation join forces in probabilistic automata. In International Symposium on Formal Methods. 247–262.Google ScholarDigital Library
- Vojtěch Forejt, Marta Kwiatkowska, Gethin Norman, and David Parker. 2011. Automated verification techniques for probabilistic systems. In International School on Formal Methods for the Design of Computer, Communication and Software Systems. 53–113.Google Scholar
- Yulong Gao, Karl Henrik Johansson, and Lihua Xie. 2020. Computing probabilistic controlled invariant sets. IEEE Trans. Automat. Control 66, 7 (2020), 3138–3151.Google ScholarCross Ref
- Hans Hansson and Bengt Jonsson. 1994. A logic for reasoning about time and reliability. Formal Aspects of Computing 6, 5 (1994), 512–535.Google ScholarDigital Library
- M. Herceg, M. Kvasnica, C.N. Jones, and M. Morari. 2013. Multi-Parametric Toolbox 3.0. In European Control Conference. 502–510.Google Scholar
- Holger Hermanns, Jan Krčál, and Jan Křetínskỳ. 2014. Probabilistic bisimulation: naturally on distributions. In International Conference on Concurrency Theory. 249–265.Google ScholarCross Ref
- Rui-Juan Jing, Marc Moreno-Maza, and Delaram Talaashrafi. 2020. Complexity estimates for Fourier-Motzkin elimination. In 22nd International Workshop on Computer Algebra in Scientific Computing. 282–306.Google ScholarDigital Library
- Austin Jones, Mac Schwager, and Calin Belta. 2013. Distribution temporal logic: Combining correctness with quality of estimation. In 52nd IEEE Conference on Decision and Control. 4719–4724.Google ScholarCross Ref
- Joost-Pieter Katoen. 2016. The probabilistic model checking landscape. In 31st Annual ACM/IEEE Symposium on Logic in Computer Science. 31–45.Google ScholarDigital Library
- Vijay Anand Korthikanti, Mahesh Viswanathan, Gul Agha, and YoungMin Kwon. 2010. Reasoning about MDPs as transformers of probability distributions. In 7th International Conference on the Quantitative Evaluation of Systems. 199–208.Google ScholarDigital Library
- Marta Kwiatkowska, Gethin Norman, and David Parker. 2007. Stochastic model checking. In International School on Formal Methods for the Design of Computer, Communication and Software Systems. 220–270.Google Scholar
- Marta Kwiatkowska, Gethin Norman, and David Parker. 2009. PRISM: probabilistic model checking for performance and reliability analysis. ACM SIGMETRICS Performance Evaluation Review 36, 4 (2009), 40–45.Google ScholarDigital Library
- Marta Kwiatkowska, Gethin Norman, and David Parker. 2018. Probabilistic model checking: advances and applications. In Formal System Verification. Springer, 73–121.Google Scholar
- YoungMin Kwon and Gul Agha. 2004. Linear inequality LTL (iLTL): A model checker for discrete time Markov chains. In International Conference on Formal Engineering Methods. 194–208.Google ScholarCross Ref
- YoungMin Kwon and Gul Agha. 2010. Verifying the evolution of probability distributions governed by a DTMC. IEEE Transactions on Software Engineering 37, 1 (2010), 126–141.Google ScholarDigital Library
- J. Löfberg. 2004. YALMIP : A Toolbox for Modeling and Optimization in MATLAB. In In Proceedings of the CACSD Conference.Google ScholarCross Ref
- Andreas Löhne and Benjamin Weißing. 2016. Equivalence between polyhedral projection, multiple objective linear programming and vector linear programming. Mathematical Methods of Operations Research 84 (2016), 411–426.Google ScholarCross Ref
- Kenneth L McMillan. 1993. Symbolic Model Checking. Springer.Google ScholarDigital Library
- R Tyrrell Rockafellar and Roger J-B Wets. 2009. Variational Analysis. Springer.Google Scholar
- Ilya Tkachev and Alessandro Abate. 2014. Characterization and computation of infinite-horizon specifications over Markov processes. Theoretical Computer Science 515 (2014), 1–18.Google ScholarDigital Library
- Petter Tøndel, Tor Arne Johansen, and Alberto Bemporad. 2003. An algorithm for multi-parametric quadratic programming and explicit MPC solutions. Automatica 39, 3 (2003), 489–497.Google ScholarDigital Library
- M. Y. Vardi and L. Stockmeyer. 1985. Improved upper and lower bounds for modal logics of programs. In ACM Symposium on Theory of Computing. 240–251.Google Scholar
- Yinyu Ye and Edison Tse. 1989. An extension of Karmarkar’s projective algorithm for convex quadratic programming. Mathematical programming 44 (1989), 157–179.Google Scholar
Recommendations
Model Checking CTL*[DC]
TACAS 2001: Proceedings of the 7th International Conference on Tools and Algorithms for the Construction and Analysis of SystemsWe define a logic called CTL*[DC] which extends CTL* with ability to specify past-time and quantitative timing properties using the formulae of Quantified Discrete-time Duration Calculus (QDDC). Alternately, we can consider CTL*[DC] as extending logic ...
Logics and translations for hierarchical model checking
In this study, logics and translations for hierarchical model checking are developed based on linear-time temporal logic (LTL) and computation-tree logic (CTL). Hierarchical model checking is a model checking paradigm that can appropriately verify ...
CTL model checking for boolean program
ICCSA'06: Proceedings of the 2006 international conference on Computational Science and Its Applications - Volume Part IVNowadays, there are some subtle errors in a software system. So verification technique is very important. The one of important verification technique is model checking technique. Model checking is a technique to verify behavior of system with desired ...
Comments