Abstract
We describe an approach to Statistical Model Checking (SMC) that produces not only an estimate of the probability that specified properties (a.k.a. predicates) are satisfied, but also an “input attribution” for those predicates. We use logistic regression to generate the input attribution as a set of linear and non-linear functions of the inputs that explain conditions under which a predicate is satisfied. These functions provide quantitative insight into factors that influence the predicate outcome. We have implemented our approach on a distributed SMC infrastructure, demeter, that uses Linux Docker containers to isolate simulations (a.k.a. trials) from each other. Currently, demeter is deployed on six 20-core blade servers, and can perform tens of thousands of trials in a few hours. We demonstrate our approach on examples involving robotic agents interacting in a simulated physical environment. Our approach synthesizes input attributions that are both meaningful to the investigator and have predictive value on the predicate outcomes.
This material is based upon work funded and supported by the Department of Defense under Contract No. FA8721-05-C-0003 with Carnegie Mellon University for the operation of the Software Engineering Institute, a federally funded research and development center, DM-0003895.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
In this paper we use \(\mathcal {U}(a,b)\) for the uniform distribution between two real numbers \(a \le b\), and \(\mathcal {U}\{a,b\}\) for the uniform integer distribution between a and b, inclusive.
- 2.
This leaves a few CPU threads for host processing independent of simulation activities.
References
A platform for operating docker in production. http://github.com/rancher/rancher
Dukeman, A., Adams, J.A., Edmondson, J.: Extensible collaborative autonomy using GAMS. In: Proceedings of IRMAS (2016)
Chaki, S., Kyle, D.: DMPL: programming and verifying distributed mixed-synchrony and mixed-critical software. Technical report CMU/SEI-2016-TR-005, Software Engineering Institute, Carnegie Mellon University, Pittsburgh (2016). http://resources.sei.cmu.edu/library/asset-view.cfm?assetid=464254
Clarke, E.M., Zuliani, P.: Statistical model checking for cyber-physical systems. In: Bultan, T., Hsiung, P.-A. (eds.) ATVA 2011. LNCS, vol. 6996, pp. 1–12. Springer, Heidelberg (2011). doi:10.1007/978-3-642-24372-1_1
Cousot, P., Cousot, R.: Abstract interpretation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints. In: Proceedings of POPL (1977)
Cousot, P., Cousot, R., Fähndrich, M., Logozzo, F.: Automatic inference of necessary preconditions. In: Giacobazzi, R., Berdine, J., Mastroeni, I. (eds.) VMCAI 2013. LNCS, vol. 7737, pp. 128–148. Springer, Heidelberg (2013). doi:10.1007/978-3-642-35873-9_10
David, A., Du, D., Guldstrand Larsen, K., Legay, A., Mikučionis, M.: Optimizing control strategy using statistical model checking. In: Brat, G., Rungta, N., Venet, A. (eds.) NFM 2013. LNCS, vol. 7871, pp. 352–367. Springer, Heidelberg (2013). doi:10.1007/978-3-642-38088-4_24
David, A., Larsen, K.G., Legay, A., Mikučionis, M., Wang, Z.: Time for statistical model checking of real-time systems. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 349–355. Springer, Heidelberg (2011). doi:10.1007/978-3-642-22110-1_27
Rohmer, E., Signgh, S.P.N., Freese, M.: V-REP: a versatile and scalable robot simulation framework. In: Proceedings of IROS (2013)
Edmondson, J., Gokhale, A.: Design of a scalable reasoning engine for distributed, real-time and embedded systems. In: Xiong, H., Lee, W.B. (eds.) KSEM 2011. LNCS (LNAI), vol. 7091, pp. 221–232. Springer, Heidelberg (2011). doi:10.1007/978-3-642-25975-3_20
Ernst, M.D., Cockrell, J., Griswold, W.G., Notkin, D.: Dynamically discovering likely program invariants to support program evolution. In: Proceedings of ICSE (1999)
James, G., Witten, D., Hastie, T., Tibshirani, R.: An Introduction to Statistical Learning, 6th edn. Springer, New York (2015)
Hanley, J., McNeil, B.: The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143(1), 29–36 (1982)
Hosmer, D., Lemeshow, S.: Applied Logistic Regression, 3rd edn. Wiley, Hoboken (2013)
Kwiatkowska, M., Norman, G., Parker, D.: PRISM 4.0: verification of probabilistic real-time systems. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 585–591. Springer, Heidelberg (2011). doi:10.1007/978-3-642-22110-1_47
Kyle, D., Hansen, J., Chaki, S.: Statistical model checking of distributed adaptive real-time software. In: Bartocci, E., Majumdar, R. (eds.) RV 2015. LNCS, vol. 9333, pp. 269–274. Springer, Heidelberg (2015). doi:10.1007/978-3-319-23820-3_17
Merkel, D.: Docker: lightweight Linux containers for consistent development and deployment. Linux J. http://dl.acm.org/citation.cfm?id=2600239.2600241
Moreno, G.A., Cámara, J., Garlan, D., Schmerl, B.: Efficient decision-making under uncertainty for proactive self-adaptation. In: Proceedings of ICAC (2016, to appear)
Musliner, D.J., Engstrom, E.: PRISMATIC: unified hierarchical probabilistic verification tool. Technical report AFRL-RZ-WP-TR-2011-2097 (2011)
R Development Core Team: R: A Language and Environment for Statistical Computing (2008). http://www.R-project.org
Seshachala, S.: Docker vs VMs. http://devops.com/2014/11/24/docker-vs-vms
Younes, H.L.S.: Ymer: a statistical model checker. In: Etessami, K., Rajamani, S.K. (eds.) CAV 2005. LNCS, vol. 3576, pp. 429–433. Springer, Heidelberg (2005). doi:10.1007/11513988_43
Younes, H.L.S.: Verification and planning for stochastic processes with asynchronous events. Ph.D. thesis, CMU, Technical report no. CMU-CS-05-105 (2005)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Hansen, J.P., Chaki, S., Hissam, S., Edmondson, J., Moreno, G.A., Kyle, D. (2016). Input Attribution for Statistical Model Checking Using Logistic Regression. In: Falcone, Y., Sánchez, C. (eds) Runtime Verification. RV 2016. Lecture Notes in Computer Science(), vol 10012. Springer, Cham. https://doi.org/10.1007/978-3-319-46982-9_12
Download citation
DOI: https://doi.org/10.1007/978-3-319-46982-9_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-46981-2
Online ISBN: 978-3-319-46982-9
eBook Packages: Computer ScienceComputer Science (R0)