Abstract
For developers of assisted or automated driving systems, gaining specific feedback and quantitative figures on the safety impact of the systems under development is crucial. However, obtaining such data from simulation of their design models is a complex and often time-consuming process. Especially when data of interest hinge on extremely rare events, an estimation of potential risks is highly desirable but a non-trivial task lacking easily applicable methods. In this paper we describe how a quantitative statement for a risk estimation involving extremely rare events can be obtained by guiding simulation based on reinforcement learning. The method draws on variance reduction and importance sampling, yet applies different optimization principles than related methods, like the cross-entropy methods against which we compare. Our rationale for optimizing differently is that in quantitative system verification, a sharper upper bound of the confidence interval is of higher relevance than the total width of the confidence interval.
Our application context is deduced from advanced driver assistance system (ADAS) development. In that context virtual driver simulations are performed with the objective to generate quantitative figures for the safety impact in pre-crash situations. In order to clarify the difference of our technique to variance reduction techniques, a comparative evaluation on a simple probabilistic benchmark system is also presented.
This research was supported by the Ministry of Science and Culture of Lower Saxony within the research center Critical Systems Engineering for Sociotechnical Systems.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Like in formal verification we have to assume the model used for simulation is correct.
- 2.
Note that hitherto unseen paths in the tree can arise during simulation due to the probabilistic nature of the model being simulated. Therefore, the set of nodes in the tree grows incrementally.
- 3.
Note that, due to the finite amount of samples used, this is only an approximation.
References
Clopper, C.J., Pearson, E.S.: The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika 26(4), 404–413 (1934)
Donzé, A., Maler, O.: Robust satisfaction of temporal logic over real-valued signals. In: Chatterjee, K., Henzinger, T.A. (eds.) FORMATS 2010. LNCS, vol. 6246, pp. 92–106. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15297-9_9
Efron, B., Tibshirani, R.J.: An Introduction to the Bootstrap. Monographs on Statistics and Applied Probability, no. 57. Chapman & Hall/CRC, London (1993)
European Commission: Towards a European road safety area: policy orientations on road safety 2011–2020 (2010). http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52010DC0389
Eurostat: Slightly over 26 000 victims of road accidents in the EU in 2015. Eurostat Press Office Vincent (2016). http://ec.europa.eu/eurostat/documents/2995521/7734698/7-18112016-BP-EN.pdf
Fränzle, M., Hansen, M.R.: A robust interpretation of duration calculus. In: Van Hung, D., Wirsing, M. (eds.) ICTAC 2005. LNCS, vol. 3722, pp. 257–271. Springer, Heidelberg (2005). https://doi.org/10.1007/11560647_17
Gietelink, O., De Schutter, B., Verhaegen, M.: Adaptive importance sampling for probabilistic validation of advanced driver assistance systems. In: 2006 American Control Conference, vol. 19, 6 pp. (2006)
Gietelink, O., De Schutter, B., Verhaegen, M.: Probabilistic validation of advanced driver assistance systems. In: Proceedings of the 16th IFAC World Congress, vol. 19 (2005)
Jegourel, C., Larsen, K.G., Legay, A., Mikučionis, M., Poulsen, D.B., Sedwards, S.: Importance sampling for stochastic timed automata. In: Fränzle, M., Kapur, D., Zhan, N. (eds.) SETTA 2016. LNCS, vol. 9984, pp. 163–178. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-47677-3_11
Kahn, H.: Use of different Monte Carlo sampling techniques, p. 766 (1955)
Page, Y., et al.: A comprehensive and harmonized method for assessing the effectiveness of advanced driver assistance systems by virtual simulation: the P.E.A.R.S. initiative. In: The 24th International Technical Conference on the Enhanced Safety of Vehicles (ESV). NHTSA, Gothenburg (2015)
Puch, S., Wortelen, B., Fränzle, M., Peikenkamp, T.: Using guided simulation to improve a model-based design process of complex human machine systems. In: Modelling and Simulation, ESM 2012, pp. 159–164. EUROSIS-ETI, Essen (2012)
Puch, S., Wortelen, B., Fränzle, M., Peikenkamp, T.: Evaluation of drivers interaction with assistant systems using criticality driven guided simulation. In: Duffy, V.G. (ed.) DHM 2013. LNCS, vol. 8025, pp. 108–117. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39173-6_13
Rubinstein, R.: The cross-entropy method for combinatorial and continuous optimization. Methodol. Comput. Appl. Probab. 1, 127–190 (1999)
Vogel, K.: A comparison of headway and time to collision as safety indicators. Accid. Anal. Prev. 35(3), 427–433 (2003)
Vorndran, I.: Unfallstatistik - Verkehrsmittel im Risikovergleich. DESTATIS (2010). https://www.destatis.de/DE/Publikationen/WirtschaftStatistik/Monatsausgaben/WistaDezember10.pdf?__blob=publicationFile
WIVW GmbH: Fahrsimulationssoftware SILAB. https://wivw.de/de/silab
Wortelen, B., Baumann, M., Lüdtke, A.: Dynamic simulation and prediction of drivers’ attention distribution. Transp. Res. Part F Traffic Psychol. Behav. 21, 278–294 (2013)
Wortelen, B., Lüdtke, A., Baumann, M.: Integrated simulation of attention distribution and driving behavior. In: Proceedings of the 22nd Annual Conference on Behavior Representation in Modeling & Simulation, pp. 69–76. BRIMS Society, Ottawa (2013)
Zuliani, P., Baier, C., Clarke, E.M.: Rare-event verification for stochastic hybrid systems. In: Proceedings of the 15th ACM International Conference on Hybrid Systems: Computation and Control, pp. 217–226. ACM, New York (2012)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Puch, S., Fränzle, M., Gerwinn, S. (2018). Quantitative Risk Assessment of Safety-Critical Systems via Guided Simulation for Rare Events. In: Margaria, T., Steffen, B. (eds) Leveraging Applications of Formal Methods, Verification and Validation. Verification. ISoLA 2018. Lecture Notes in Computer Science(), vol 11245. Springer, Cham. https://doi.org/10.1007/978-3-030-03421-4_20
Download citation
DOI: https://doi.org/10.1007/978-3-030-03421-4_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-03420-7
Online ISBN: 978-3-030-03421-4
eBook Packages: Computer ScienceComputer Science (R0)