Abstract
[Context and Motivation] The automotive industry is moving towards increased automation, where features such as automated driving systems typically include machine learning (ML), e.g. in the perception system. [Question/Problem] Ensuring safety for systems partly relying on ML is challenging. Different approaches and frameworks have been proposed, typically where the developer must define quantitative and/or qualitative acceptance criteria, and ensure the criteria are fulfilled using different methods to improve e.g., design, robustness and error detection. However, there is still a knowledge gap between quality methods and metrics employed in the ML domain and how such methods can contribute to satisfying the vehicle level safety requirements. [Principal Ideas/Results] In this paper, we argue the need for connecting available ML quality methods and metrics to the safety lifecycle and explicitly show their contribution to safety. In particular, we analyse Out-of-Distribution (OoD) detection, e.g., the frequency of novelty detection, and show its potential for multiple safety-related purposes. I.e., as (a) an acceptance criterion contributing to the decision if the software fulfills the safety requirements and hence is ready-for-release, (b) in operational design domain selection and expansion by including novelty samples into the training/development loop, and (c) as a run-time measure, e.g., if there is a sequence of novel samples, the vehicle should consider reaching a minimal risk condition. [Contribution] This paper describes the possibility to use OoD detection as a safety measure, and the potential contributions in different stages of the safety lifecycle.
This research has been supported by the Strategic vehicle research and innovation (FFI) programme in Sweden, via the project SALIENCE4CAV (ref. 2020-02946) and by the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by Knut and Alice Wallenberg Foundation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Borg, M., et al.: Ergo, smirk is safe: A safety case for a machine learning component in a pedestrian automatic emergency brake system. arXiv preprint arXiv:2204.07874 (2022)
Burton, S.: A causal model of safety assurance for machine learning. arXiv preprint arXiv:2201.05451 (2022). https://doi.org/10.48550/arXiv.2201.05451
Burton, S., Hellert, C., Hüger, F., Mock, M., Rohatschek, A.: Safety Assurance of Machine Learning for Perception Functions. In: Deep Neural Networks and Data for Automated Driving: Robustness, Uncertainty Quantification, and Insights Towards Safety, pp. 335–358. Springer International Publishing (2022)
Cen, J., Yun, P., Cai, J., Wang, M.Y., Liu, M.: Deep metric learning for open world semantic segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 15333–15342 (2021)
Du, X., Wang, X., Gozum, G., Li, Y.: Unknown-aware object detection: Learning what you don’t know from videos in the wild. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13678–13688 (2022)
Gyllenhammar, M., et al.: Towards an operational design domain that supports the safety argumentation of an automated driving system. In: Proceedings of ERTS 2020. Toulouse, France (2020)
Hendrycks, D., Basart, S., Mazeika, M., Mostajabi, M., Steinhardt, J., Song, D.: Scaling out-of-distribution detection for real-world settings. arXiv preprint arXiv:1911.11132 (2019)
Hoss, M., Scholtes, M., Eckstein, L.: A Review of Testing Object-Based Environment Perception for Safe Automated Driving. Autom. Innov. 5(3), 223–250 (2022). https://doi.org/10.1007/s42154-021-00172-y
Huang, C., et al.: Out-of-distribution detection for lidar-based 3d object detection. arXiv preprint arXiv:2209.14435 (2022)
ISO: 26262:2018 Road Vehicles - Functional Safety. ISO (2018)
ISO: ISO/TR 4804:2020 Road Vehicles - Safety and Cybersecurity for Automated Driving Systems - Design, Verification and Validation. ISO (2020)
ISO: 21448:2022 Road Vehicles - Safety of the Intended Functionality. ISO (2022)
Mohseni, S., Wang, H., Yu, Z., Xiao, C., Wang, Z., Yadawa, J.: Taxonomy of Machine Learning Safety: A Survey and Primer. arXiv:2106.04823 [cs] (Mar 2022)
Ramachandra, B., Jones, M., Vatsavai, R.R.: A survey of single-scene video anomaly detection. IEEE Trans. Pattern Analysis Mach. Intell. 44, 2293–2312 (2020)
SAE: J3016 Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles. Tech. Rep. J3016:2021, SAE Int. (Apr 2021)
Salay, R., Czarnecki, K.: Using Machine Learning Safely in Automotive Software: An Assessment and Adaption of Software Process Requirements in ISO 26262. arXiv:1808.01614 [cs, stat] (Aug 2018)
Salay, R., Queiroz, R., Czarnecki, K.: An Analysis of ISO 26262: Using Machine Learning Safely in Automotive Software. Arxiv preprint 1709.02435. (2017)
Tencent Keen Security Lab: Experimental Security Research of Tesla Autopilot. Tech. rep., (Mar 2019), https://keenlab.tencent.com/en/whitepapers/Experimental_Security_Research_of_Tesla_Autopilot.pdf
Yang, J., Zhou, K., Li, Y., Liu, Z.: Generalized out-of-distribution detection: A survey. arXiv preprint arXiv:2110.11334 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Henriksson, J. et al. (2023). Out-of-Distribution Detection as Support for Autonomous Driving Safety Lifecycle. In: Ferrari, A., Penzenstadler, B. (eds) Requirements Engineering: Foundation for Software Quality. REFSQ 2023. Lecture Notes in Computer Science, vol 13975. Springer, Cham. https://doi.org/10.1007/978-3-031-29786-1_16
Download citation
DOI: https://doi.org/10.1007/978-3-031-29786-1_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-29785-4
Online ISBN: 978-3-031-29786-1
eBook Packages: Computer ScienceComputer Science (R0)