Skip to main content

Providing Evidence for the Validity of the Virtual Verification of Automated Driving Systems

  • Conference paper
  • First Online:
Dependable Computing – EDCC 2024 Workshops (EDCC 2024)

Abstract

With the increasing complexity of automated driving systems, formal verification as well as statistical verification that solely relies on real-world testing methods, become infeasible. Virtual testing seems like a promising alternative to traditional methods, especially as part of a scenario-based verification and validation methodology. But in order to transfer the test results of a system from a simulation to the real world, we need to argue the validity of the virtual tests. Our proposed method enables this validity argumentation by comparing the virtual test traces against traces that have sufficiently similar recorded real-world traces. To reduce the amount of required real-world data, the method involves two mechanisms to generalize the validity statement of a single real-world trace to a set of virtual traces. The reduction of required data is showcased in a proof of concept that compares the needed amounts of data with a “naive” validation method and here presented enhancements in an ablation study.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. ISO 19364 - Passenger cars - Vehicle dynamic simulation and validation-Steady-state circular driving behaviour. Technical report, International Organization for Standardization

    Google Scholar 

  2. The world factbook, Washington, DC 20505 (2019)

    Google Scholar 

  3. Bendale, A., Boult, T.: Towards open world recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)

    Google Scholar 

  4. Danquah, B., Riedmaier, S., Meral, Y., Lienkamp, M.: Statistical validation framework for automotive vehicle simulations using uncertainty learning. Appl. Sci. 11(5), 1983 (2021)

    Article  Google Scholar 

  5. Danquah, B., Riedmaier, S., Rühm, J., Kalt, S., Lienkamp, M.: Statistical model verification and validation concept in automotive vehicle design. Procedia CIRP 91, 261–270 (2020). enhancing design through the 4th Industrial Revolution Thinking

    Google Scholar 

  6. Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., Koltun, V.: CARLA: an open urban driving simulator. In: Proceedings of the 1st Annual Conference on Robot Learning (2017)

    Google Scholar 

  7. ERTRAC Working Group "Connectivity and Automated Driving": Connected automated driving roadmap (2019)

    Google Scholar 

  8. Kalra, N., Paddock, S.M.: Driving to safety: how many miles of driving would it take to demonstrate autonomous vehicle reliability? (2016)

    Google Scholar 

  9. Neurohr, C., Westhofen, L., Henning, T., de Graaff, T., Möhlmann, E., Böde, E.: Fundamental considerations around scenario-based testing for automated driving. In: 2020 IEEE Intelligent Vehicles Symposium (IV), pp. 121–127 (2020)

    Google Scholar 

  10. Oberkampf, W.L., Trucano, T.G., Hirsch, C.: Verification, validation, and predictive capability in computational engineering and physics. Appl. Mech. Rev. 57(5), 345–384 (2004)

    Article  Google Scholar 

  11. Poddey, A., Brade, T., Stellet, J.E., Branz, W.: On the validation of complex systems operating in open contexts (2019)

    Google Scholar 

  12. Riedmaier, S., Danquah, B., Schick, B., Diermeyer, F.: Unified framework and survey for model verification, validation and uncertainty quantification. Arch. Comput. Methods Eng. 28, 2655–2688 (2021)

    Article  MathSciNet  Google Scholar 

  13. Riedmaier, S., Schneider, J., Danquah, B., Schick, B., Diermeyer, F.: Non-deterministic model validation methodology for simulation-based safety assessment of automated vehicles. Simul. Model. Pract. Theory 109, 102274 (2021)

    Article  Google Scholar 

  14. Rosenberger, P., Schunk, G., Ikemeyer, F., Duong, Q.T.: Validation of Test Infrastructure - from cause trees to a validated system simulation

    Google Scholar 

  15. Rosenberger, P., et al.: Towards a generally accepted validation methodology for sensor models - challenges, metrics, and first results. In: Graz Symposium Virtual Vehicle (GSVF) (2019)

    Google Scholar 

  16. SAE International: Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles. Technical report

    Google Scholar 

  17. Sargent, R.G.: Verification and validation of simulation models. In: Proceedings of the 2011 Winter Simulation Conference (2011)

    Google Scholar 

  18. Schaermann, A.: Systematische Bedatung und Bewertung umfelderfassender Sensormodelle. Ph.D. thesis

    Google Scholar 

  19. Schlesinger, S., et al.: Terminology for model credibility. SIMULATION 32(3), 103–104 (1979)

    Article  Google Scholar 

  20. Viehof, M.: Objektive Qualitätsbewertung von Fahrdynamiksimulationen durch statistische Validierung. Ph.D. thesis

    Google Scholar 

  21. Wachenfeld, W., Winner, H.: Die Freigabe des autonomen Fahrens. In: Maurer, M., Gerdes, J., Lenz, B., Winner, H. (eds.) Autonomes Fahren, pp. 439–464. Springer Vieweg, Berlin, Heidelberg (2015). https://doi.org/10.1007/978-3-662-45854-9_21

    Chapter  Google Scholar 

Download references

Acknowledgments

This work has received funding by the German ministries BMWK and BMBF within the projects “KI Wissen”and “ASIMOV”.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Birte Neurohr .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Neurohr, B., de Graaff, T., Eggers, A., Bienmüller, T., Möhlmann, E. (2024). Providing Evidence for the Validity of the Virtual Verification of Automated Driving Systems. In: Sangchoolie, B., Adler, R., Hawkins, R., Schleiss, P., Arteconi, A., Mancini, A. (eds) Dependable Computing – EDCC 2024 Workshops. EDCC 2024. Communications in Computer and Information Science, vol 2078. Springer, Cham. https://doi.org/10.1007/978-3-031-56776-6_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-56776-6_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-56775-9

  • Online ISBN: 978-3-031-56776-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics