Abstract
We propose a new framework to facilitate dynamic assurance within a safety case approach by associating safety performance measurement with the core assurance artifacts of a safety case. The focus is mainly on the safety architecture, whose underlying risk assessment model gives the concrete link from safety measurement to operational risk. Using an aviation domain example of autonomous taxiing, we describe our approach to derive safety indicators and revise the risk assessment based on safety measurement. We then outline a notion of consistency between a collection of safety indicators and the safety case, as a formal basis for implementing the proposed framework in our tool, AdvoCATE.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Also known as an operational design domain (ODD) for systems integrating ML [14].
- 2.
Integrity is the probability that a barrier or control is not breached, i.e., it delivers its intended function for reducing risk in the specified operating context and scenario [8].
- 3.
Interested readers may refer to [3].
- 4.
Henceforth, identifiers with the prefix ‘\(\texttt{dev}\)’ refer to metrics used during system development, and the prefix ‘\(\texttt{op}\)’ indicates an operational safety metric.
- 5.
RR has also been used as a development safety metric, e.g., in designing aircraft collision avoidance systems [9].
References
Asaadi, E., Denney, E., Menzies, J., Pai, G., Petroff, D.: Dynamic assurance cases: a pathway to trusted autonomy. IEEE Comput. 53(12), 35–46 (2020)
Basili, V., Caldiera, G., Rombach, D.: Goal Question Metric Paradigm, pp. 528–532. Encyclopedia of Software Engineering, John Wiley & Sons, Inc., 2nd edn. (1994)
Busch, A.C.: Methodology for Establishing a Target Level of Safety. Technical Report DOT/FAA/CT-TN85/36, US DOT, FAA Technical Center (1985)
Calinescu, R., Weyns, D., Gerasimou, S., Iftikhar, M.U., Habli, I., Kelly, T.: Engineering trustworthy self-adaptive software with dynamic assurance cases. IEEE Trans. Softw. Eng. 44(11), 1039–1069 (2018)
Denney, E., Habli, I., Pai, G.: Dynamic safety cases for through-life safety assurance. In: 37th International Conference on Software Engineering - Vol. 2, pp. 587–590. (2015)
Denney, E., Johnson, M., Pai, G.: Towards a rigorous basis for specific operations risk assessment of UAS. In: 37th IEEE/AIAA Digital Avionics Systems Conference (2018)
Denney, E., Pai, G.: Tool Support for Assurance Case Development. J. Autom. Softw. Eng. 25(3), 435–499 (2018)
Denney, E., Pai, G., Whiteside, I.: The role of safety architectures in aviation safety cases. Reliab. Eng. Syst. Saf. 191, 106502 (2019)
Edwards, M., Mackay, J.: Determining required surveillance performance for unmanned aircraft sense and avoid. In: 17th AIAA Aviation Technology, Integration, and Operations (ATIO) Conference. AIAA 2017-4385 (2017)
FAA Air Traffic Organization: Safety Management System Manual (2022)
US Department of Transportation, FAA: Safety Risk Management Policy. Order 8040.4C (2023)
Hawkins, R., Ryan Conmy, P.: Identifying run-time monitoring requirements for autonomous systems through the analysis of safety arguments. In: Guiochet, J., Tonetta, S., Bitsch, F. (eds.) Computer Safety, Reliability, and Security. SAFECOMP 2023. LNCS, vol. 14181. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-40923-3_2
International Civil Aviation Organization (ICAO): Safety Management Manual (Doc 9859), 4 edn. (2018)
Kaakai, F., Adibhatla, S., Pai, G., Escorihuela, E.: Data-centric operational design domain characterization for machine learning-based aeronautical products. In: Guiochet, J., Tonetta, S., Bitsch, F. (eds.) Computer Safety, Reliability, and Security. SAFECOMP 2023. LNCS, vol. 14181. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-40923-3_17
Koopman, P.: How Safe is Safe Enough? Measuring and Predicting Autonomous Vehicle Safety. 1st edn. (2022)
Ladkin, P.: Evaluating software execution as a Bernoulli process. Saf. Crit. Syst. eJournal 1(2) (2022)
Reich, J., Trapp, M.: SINADRA: towards a framework for Assurable situation-aware dynamic risk assessment of autonomous vehicles. In: 16th European Dependable Computing Conference (EDCC), pp. 47–50 (2020)
Schleiss, P., Carella, F., Kurzidem, I.: Towards continuous safety assurance for autonomous systems. In: 6th International Conference on System Reliability and Safety (ICSRS 2022), pp. 457–462 (2022)
Strigini, L.: Trustworthy quantitative arguments for the safety of AVs: challenges and some modest proposals. In: 1st IFIP Workshop on Intelligent Vehicle Dependability and Security (IVDS) (2021)
The Assurance Case Working Group (ACWG): Goal Structuring Notation Community Standard Version 3. SCSC-141C (2021)
Trapp, M., Weiss, G.: Towards dynamic safety management for autonomous systems. In: 27th Safety-Critical Systems Symposium (SSS), pp. 193–204 (2019)
Acknowledgments
This work was performed under Contract No. 80ARC020D0010 with the National Aeronautics and Space Administration (NASA), with support from the System-wide Safety project, under the Airspace Operations and Safety Program of the NASA Aeronautics Research Mission Directorate. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, worldwide license to reproduce, prepare derivative works, distribute copies to the public, and perform publicly and display publicly, or allow others to do so, for United States Government purposes. All other rights are reserved by the copyright owner.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Denney, E., Pai, G. (2024). Reconciling Safety Measurement and Dynamic Assurance. In: Ceccarelli, A., Trapp, M., Bondavalli, A., Bitsch, F. (eds) Computer Safety, Reliability, and Security. SAFECOMP 2024. Lecture Notes in Computer Science, vol 14988. Springer, Cham. https://doi.org/10.1007/978-3-031-68606-1_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-68606-1_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-68605-4
Online ISBN: 978-3-031-68606-1
eBook Packages: Computer ScienceComputer Science (R0)