Abstract
With the increasing complexity of automated driving systems, formal verification as well as statistical verification that solely relies on real-world testing methods, become infeasible. Virtual testing seems like a promising alternative to traditional methods, especially as part of a scenario-based verification and validation methodology. But in order to transfer the test results of a system from a simulation to the real world, we need to argue the validity of the virtual tests. Our proposed method enables this validity argumentation by comparing the virtual test traces against traces that have sufficiently similar recorded real-world traces. To reduce the amount of required real-world data, the method involves two mechanisms to generalize the validity statement of a single real-world trace to a set of virtual traces. The reduction of required data is showcased in a proof of concept that compares the needed amounts of data with a “naive” validation method and here presented enhancements in an ablation study.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
ISO 19364 - Passenger cars - Vehicle dynamic simulation and validation-Steady-state circular driving behaviour. Technical report, International Organization for Standardization
The world factbook, Washington, DC 20505 (2019)
Bendale, A., Boult, T.: Towards open world recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)
Danquah, B., Riedmaier, S., Meral, Y., Lienkamp, M.: Statistical validation framework for automotive vehicle simulations using uncertainty learning. Appl. Sci. 11(5), 1983 (2021)
Danquah, B., Riedmaier, S., Rühm, J., Kalt, S., Lienkamp, M.: Statistical model verification and validation concept in automotive vehicle design. Procedia CIRP 91, 261–270 (2020). enhancing design through the 4th Industrial Revolution Thinking
Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., Koltun, V.: CARLA: an open urban driving simulator. In: Proceedings of the 1st Annual Conference on Robot Learning (2017)
ERTRAC Working Group "Connectivity and Automated Driving": Connected automated driving roadmap (2019)
Kalra, N., Paddock, S.M.: Driving to safety: how many miles of driving would it take to demonstrate autonomous vehicle reliability? (2016)
Neurohr, C., Westhofen, L., Henning, T., de Graaff, T., Möhlmann, E., Böde, E.: Fundamental considerations around scenario-based testing for automated driving. In: 2020 IEEE Intelligent Vehicles Symposium (IV), pp. 121–127 (2020)
Oberkampf, W.L., Trucano, T.G., Hirsch, C.: Verification, validation, and predictive capability in computational engineering and physics. Appl. Mech. Rev. 57(5), 345–384 (2004)
Poddey, A., Brade, T., Stellet, J.E., Branz, W.: On the validation of complex systems operating in open contexts (2019)
Riedmaier, S., Danquah, B., Schick, B., Diermeyer, F.: Unified framework and survey for model verification, validation and uncertainty quantification. Arch. Comput. Methods Eng. 28, 2655–2688 (2021)
Riedmaier, S., Schneider, J., Danquah, B., Schick, B., Diermeyer, F.: Non-deterministic model validation methodology for simulation-based safety assessment of automated vehicles. Simul. Model. Pract. Theory 109, 102274 (2021)
Rosenberger, P., Schunk, G., Ikemeyer, F., Duong, Q.T.: Validation of Test Infrastructure - from cause trees to a validated system simulation
Rosenberger, P., et al.: Towards a generally accepted validation methodology for sensor models - challenges, metrics, and first results. In: Graz Symposium Virtual Vehicle (GSVF) (2019)
SAE International: Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles. Technical report
Sargent, R.G.: Verification and validation of simulation models. In: Proceedings of the 2011 Winter Simulation Conference (2011)
Schaermann, A.: Systematische Bedatung und Bewertung umfelderfassender Sensormodelle. Ph.D. thesis
Schlesinger, S., et al.: Terminology for model credibility. SIMULATION 32(3), 103–104 (1979)
Viehof, M.: Objektive Qualitätsbewertung von Fahrdynamiksimulationen durch statistische Validierung. Ph.D. thesis
Wachenfeld, W., Winner, H.: Die Freigabe des autonomen Fahrens. In: Maurer, M., Gerdes, J., Lenz, B., Winner, H. (eds.) Autonomes Fahren, pp. 439–464. Springer Vieweg, Berlin, Heidelberg (2015). https://doi.org/10.1007/978-3-662-45854-9_21
Acknowledgments
This work has received funding by the German ministries BMWK and BMBF within the projects “KI Wissen”and “ASIMOV”.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Neurohr, B., de Graaff, T., Eggers, A., Bienmüller, T., Möhlmann, E. (2024). Providing Evidence for the Validity of the Virtual Verification of Automated Driving Systems. In: Sangchoolie, B., Adler, R., Hawkins, R., Schleiss, P., Arteconi, A., Mancini, A. (eds) Dependable Computing – EDCC 2024 Workshops. EDCC 2024. Communications in Computer and Information Science, vol 2078. Springer, Cham. https://doi.org/10.1007/978-3-031-56776-6_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-56776-6_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-56775-9
Online ISBN: 978-3-031-56776-6
eBook Packages: Computer ScienceComputer Science (R0)