Skip to main content
SpringerLink
Log in

Model-based assurance evidence management for safety–critical systems

  • Theme Section Paper
  • Published:
Software and Systems Modeling Aims and scope Submit manuscript

Abstract

Most safety–critical systems are subject to rigorous assurance processes to justify that the systems satisfy given requirements and are dependable. These processes are typically conducted in compliance with standards and require the provision of assurance evidence in the form of system artifacts, such as system specifications and testing results. The management of assurance evidence is usually a complex process because of the large number of artifacts to deal with, the amount of information to gather about the artifacts, and the need to guarantee evidence quality, among other issues. Our aim is to facilitate assurance evidence management by means of a model-based approach. The approach is based on a metamodel that defines the information to be collected about evidence artifacts during their lifecycle. A process for assurance evidence management and usage guidance are also presented. The approach has been developed in the scope of several industry-academia projects, implemented in the OpenCert tool, and validated by practitioners in 10 industrial case studies. Based on the results of the validation, we argue that the approach is an effective means for assurance evidence management and that it could improve the state of the practice.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. Throughout the paper, we use the terms assurance evidence, evidence, artifact, and evidence artifact interchangeably to refer to the same concept.

  2. We have noticed that problems might exist with the access to some deliverables that contain information about the approach presented. If a deliverable is not available online, please contact the first author.

References

  1. de la Vara, J.L., Ruiz, A., Blondelle, G.: Assurance and certification of cyber-physical systems: the AMASS open source ecosystem. J. Syst. Softw. 171, 110812 (2021)

    Article  Google Scholar 

  2. Nair, S., de la Vara, J.L., Sabetzadeh, M., Briand, L.: An extended systematic literature review on provision of evidence for safety certification. Inf. Softw. Technol. 56(7), 689–717 (2014)

    Article  Google Scholar 

  3. de la Vara, J.L., Borg, M., Wnuk, K., Moonen, L.: An industrial survey on safety evidence change impact analysis practice. IEEE Trans. Software Eng. 42(12), 1095–1117 (2016)

    Article  Google Scholar 

  4. Nair, S., de la Vara, J.L., Sabetzadeh, M., Falessi, D.: Evidence management for compliance of critical systems with safety standards: a survey on the state of practice. Inf. Softw. Technol. 60, 1–15 (2015)

    Article  Google Scholar 

  5. OMG: Structured Assurance Case Metamodel (SACM), version 2.1. 2020

  6. de la Vara, J.L., Ruiz, A., Espinoza, H.: Recent Advances towards the Industrial Application of Model-Driven Engineering for Assurance of Safety-Critical Systems. 6th International Conference on Model-Driven Engineering and Software Development (MODELSWARD 2018)

  7. Lin, H., Wu, J., Yuan, C., Luo, Y., van den Brand, M., Engelen, L.: A systematic approach for safety evidence collection in the safety-critical domain. 9th Annual IEEE International Systems Conference (SysCon 2015)

  8. Nair, S., de la Vara, J.L., Melzi, A., Tagliaferri, G., de-la-Beaujardiere, L., Belmonte, F.: Safety Evidence Traceability: Problem Analysis and Model. 20th International Working Conference on Requirements Engineering: Foundation for Software Quality (REFSQ 2014)

  9. Panesar-Walawege, R.K., Sabetzadeh, M., Briand, L.: Supporting the verification of compliance to safety standards via model-driven engineering: approach, tool-support and empirical validation. Inf. Softw. Technol. 55(5), 836–864 (2013)

    Article  Google Scholar 

  10. Bucchiarone, A., Cabot, J., Paige, R.F., Pierantonio, A.: Grand challenges in model-driven engineering: an analysis of the state of the research. Softw. Syst. Model. 19(1), 5–13 (2020)

    Article  Google Scholar 

  11. Bakirtzis, G., Sherburne, T., Adams, S., Horowitz, B.M., Beling, P.A., Fleming, C.H.: An ontological metamodel for cyber-physical system safety, security, and resilience coengineering. Software and Systems Modeling (accepted paper). 2021

  12. Biggs, G., Sakamoto, T., Kotoku, T.: A profile and tool for modelling safety information with design information in SysML. Softw. Syst. Model. 15(1), 147–178 (2016)

    Article  Google Scholar 

  13. Voelter, M., Kolb, B., Birken, K., Tomassetti, F., Alff, P., Wiart, L., Wortmann, A., Nordmann, A.: Using language workbenches and domain-specific languages for safety-critical software development. Softw. Syst. Model. 18(4), 2507–2530 (2019)

    Article  Google Scholar 

  14. Munk, P., Nordmann, A.: Model-based safety assessment with SysML and component fault trees: application and lessons learned. Softw. Syst. Model. 19(4), 889–910 (2020)

    Article  Google Scholar 

  15. de la Vara, J.L., Ruiz, A., Attwood, K., Espinoza, H., Panesar-Walawege, R.K., Lopez, A., del Rio, I., Kelly, T.: Model-based specification of safety compliance needs: a holistic generic metamodel. Inf. Softw. Technol. 72, 16–30 (2016)

    Article  Google Scholar 

  16. OPENCOSS Project: https://cordis.europa.eu/project/id/289011 (accessed August 13, 2021)

  17. AMASS Project: https://www.amass-ecsel.eu/ (accessed August 13, 2021)

  18. OpenCert: https://www.eclipse.org/opencert/ (accessed August 13, 2021)

  19. de la Vara, J.L, Ruiz, A., Gallina, B., Blondelle, G., Alaña, E., Herrero, J., Warg, F., Skoglund, M., Bramberger, R.: The AMASS approach for assurance and certification of critical systems. embedded world Conference 2019

  20. de la Vara, J.L., Parra, E., Alonso, L., López, B., Álvarez-Rodríguez, J..M.: Integration of Tool Support for Assurance and Certification and for Knowledge-Centric Systems Engineering . 9th IEEE International Workshop on Software Certification (WoSoCer 2019)

  21. Nair, S., Walkinshaw, N., Kelly, T., de la Vara, J.L.: An Evidential Reasoning Approach for Assessing Confidence in Safety Evidence. 26th IEEE International Symposium on Software Reliability Engineering (ISSRE 2015)

  22. Ruiz, A., Juez, G., Espinoza, H., de la Vara, J.L., Larrucea, X.: Reuse of safety certification artefacts across standards and domains: A systematic approach. Reliab. Eng. Syst. Saf. 158, 153–171 (2017)

    Article  Google Scholar 

  23. OPENCOSS Project: Deliverable 6.2 - Detailed requirements for evidence management of the OPENCOSS platform, v1.0, 2012

  24. AMASS Project: Deliverable D2.9–AMASS platform validation, v1.1, 2019

  25. Kelly, T.: Safety Cases. In: Handbook of Safety Principles, 361–385. Wiley, 2017

  26. Alexander, R., Kelly, T., Gorry, B.: Safety Lifecycle Activities for Autonomous Systems Development. 5th SEAS DTC Technical Conference. 2010

  27. de la Vara, J.L., Marin, B., Ayora, C., Giachetti, G.: An empirical evaluation of the use of models to improve the understanding of safety compliance needs. Inf. Softw. Technol. 126, 106351 (2020)

    Article  Google Scholar 

  28. Ericson, C.A.: Hazard Analysis Techniques for System Safety, 2nd Edition. Wiley, 2015

  29. Leveson, N.: Engineering a safer world: systems thinking applied to safety. MIT press, Cambridge (2011)

    Google Scholar 

  30. Álvarez-Rodríguez, J.M., Mendieta, R., de la Vara, J.L., Fraga, A., Llorens, J.: Enabling system artefact exchange and selection through a Linked Data layer. J. Univ. Comput. Sci. 24(11), 1536–1560 (2018)

    Google Scholar 

  31. Rempel, P., Mäder, P., Kuschke, T., Cleland-Huang, J.: Mind the Gap: Assessing the Conformance of Software Traceability to Relevant Guidelines. 36th International Conference on Software Engineering (ICSE 2014)

  32. de la Vara, J.L.: Business process-based requirements specification and object-oriented conceptual modelling of information systems. Universidad Politecnica de Valencia. 2011

  33. Wolny, S., Mazak, A., Carpella, C., Geist, V., Wimmer, M.: Thirteen years of SysML: a systematic mapping study. Softw. Syst. Model. 19(1), 111–169 (2020)

    Article  Google Scholar 

  34. Ibrahim, Y., Törnlund, M.: Leveraging a traceability information model in order to enhance the maintenance of automotive safety assurance cases. Chalmers University of Technology, University of Gothenburg (2020)

    Google Scholar 

  35. Luo, Y., van den Brand, M., Engelen, L., Klabbers, M.: From Conceptual Models to Safety Assurance. 33rd International Conference on Conceptual Modeling (ER 2014)

  36. de la Vara, J.L.: Current and Necessary Insights into SACM: An Analysis Based on Past Publications. 7th International Workshop on Requirements Engineering and Law (RELAW 2014)

  37. de la Vara, J.L., Génova, G., Álvarez-Rodríguez, J.M., Llorens, J.: An analysis of safety evidence management with the structured assurance case metamodel. Comput. Stand. Interfac. 50, 179–198 (2017)

    Article  Google Scholar 

  38. Graydon, P.J.: The Simple Assurance Argument Interchange Format (SAAIF) Manual. NASA Technical Report NASA/TM–2018–219837. 2018

  39. Kokaly, S., Salay, R., Chechik, M., Lawford, M., Maibaum, T.: Safety Case Impact Assessment in Automotive Software Systems: An Improved Model-Based Approach. 36th International Conference on Computer Safety, Reliability, and Security (SAFECOMP 2017)

  40. OMG: Dependability Assurance Framework for Safety-Sensitive Consumer Devices (DAF), version 1.0. 2016

  41. Alanen, J., Tommila, T.: A reference model for the NPP I&C qualification process and safety demonstration data. VTT Research Report VTT-R-00478–16. 2016

  42. Alanen, J., Linnosmaa, J., Tommila, T.: Conformity assessment data model. VTT Research Report VTT-R-06743–17. 2017

  43. Vilela, J., Castro, J., Martins, L.E.G., Gorschek, T.: Safe-RE: a safety requirements metamodel based on industry safety standards. XXXII Brazilian Symposium on Software Engineering (SBES 2018)

  44. Larrucea, X., Gonzalez-Perez, C., McBride, T., Henderson-Sellers, B.: Standards-based metamodel for the management of goals, risks and evidences in critical systems development. Comput. Stand. Interfac. 48, 71–79 (2016)

    Article  Google Scholar 

  45. Metayer, N., Paz, A., El-Boussaidi, G.: Modelling DO-178C Assurance Needs: A Design Assurance Level-Sensitive DSL. 9th IEEE International Workshop on Software Certification (WoSoCer 2019)

  46. Habli, I, Kelly, T.: A Model-Driven Approach to Assuring Process Reliability. 19th International Symposium on Software Reliability Engineering (ISSRE 2008)

  47. Sun, L., Kelly, T.: Elaborating the Concept of Evidence in Safety Cases. 21st Safety-Critical Systems Symposium (SSS 2013)

  48. Ayora, C., Torres, V., de la Vara, J.L., Pelechano, V.: Variability management in process families through change patterns. Inf. Softw. Technol. 74, 86–104 (2016)

    Article  Google Scholar 

  49. Gallina, B., Gómez-Martínez, E., Benac-Earle, C.: Promoting MBA in the rail sector by deriving process-related evidence via MDSafeCer. Comput. Stand. Interfac. 54, 119–128 (2017)

    Article  Google Scholar 

  50. Nair, S., de la Vara, J.L., Sabetzadeh, M., Briand, L.: Classification, Structuring, and Assessment of Evidence For Safety: A Systematic Literature Review. 6th IEEE International Conference on Software Testing, Verification and Validation (ICST 2013)

  51. Nair, S., de la Vara, J.L., Sen, S.: A Review of Traceability Research at the Requirements Engineering Conference. 21st IEEE International Requirements Engineering Conference (RE’13)

  52. Borg, M., de la Vara, J.L., Wnuk, K.:Practitioners’ Perspectives on Change Impact Analysis for Safety-Critical Software - A Preliminary Analysis. 5th International Workshop on Next Generation of System Assurance Approaches for Safety-Critical Systems (SASSUR 2016)

  53. OPENCOSS Project: Deliverable 6.1 - Baseline for the evidence management needs of the OPENCOSS platform, v1.1, 2012

  54. Muram, F. U., Gallina, B., Kanwal, S.: A Tool-Supported Model-Based Method for Facilitating the EN50129-Compliant Safety Approval Process. 3rd International Conference on Reliability, Safety, and Security of Railway Systems (RSSRail 2019)

  55. Falessi, D., Sabetzadeh, M., Briand, L., Turella, E., Coq, T., Panesar-Walawege, R.K.: Planning for safety standards compliance: a model-based tool-supported approach. IEEE Softw. 29(3), 64–70 (2012)

    Article  Google Scholar 

  56. Bézivin, J.: On the unification power of models. Softw. Syst. Model. 4(2), 171–188 (2005)

    Article  Google Scholar 

  57. CHESS: https://www.eclipse.org/chess/ (accessed August 13, 2021)

  58. Eclipse Process Framework Project: https://www.eclipse.org/epf/ (accessed August 13, 2021)

  59. The REUSE Company: RQA - Quality Studio, https://www.reusecompany.com/rqa-quality-studio (accessed August 13, 2021)

  60. PTC: Windchill Process Director. https://www.ptc.com/en/products/windchill/process-director (accessed August 13, 2021)

  61. de la Vara, J.L., Parra, E., Ruiz, A., Gallina, B.: The AMASS Tool Platform: An Innovative Solution for Assurance and Certification of Cyber-Physical Systems. 26th International Working Conference on Requirements Engineering: Foundation for Software Quality (REFSQ 2020)

  62. AMASS Project: Deliverable 1.6–AMASS demonstrators (c), v1.1, 2019

  63. AMASS Project: Deliverable D5.6–Prototype for seamless interoperability (c), v1.0. 2018

  64. OPENCOSS Project: Deliverable 6.6 - Implementation of the evidence management service infrastructure, v1.4, 2015

  65. AMASS Project: Deliverable D5.8–Methodological Guide for Seamless Interoperability (b), v1.0. 2018

  66. OPENCOSS Project: Deliverable 6.7 - Evidence management service infrastructure: Methodological Guide, v2.0, 2015

  67. Espinoza, A., Alarcon, P.P., Garbajosa, J.: Analyzing and systematizing current traceability schemas. 30th Annual IEEE/NASA Software Engineering Workshop (SEW 2006)

  68. Pohl, K.: Requirements Engineering: Fundamentals, Principles, and Techniques. Springer, 2010

  69. Wiegers, K.E.: Software Requirements, 2nd ed. Microsoft Press, 2003

  70. SafeAdapt project: https://www.safeadapt.eu/ (accessed August 13, 2021)

  71. de la Vara, J.L., Nair, S., Verhulst, E., Studzizba, J., Pepek, P., Lambourg, J., Sabetzadeh, M.: Towards a Model-Based Evolutionary Chain of Evidence for Compliance with Safety Standards. 1st International Workshop on Next Generation of System Assurance Approaches for Safety-Critical Systems (SASSUR 2012)

  72. Runeson, P., Höst, M., Rainer, A., Regnell, B.: Case Study Research in Software Engineering - Guidelines and Examples. Wiley, 2012

  73. Opencert: Online training, https://www.eclipse.org/opencert/resources/training/ (accessed August 13, 2021)

  74. AMASS Project: Deliverable D2.5–AMASS user guidance and methodological framework, v1.0. 2018

  75. AQUAS project: http://aquas-project.eu/ (accessed August 13, 2021)

  76. EMC2 project: https://www.artemis-emc2.eu/ (accessed August 13, 2021)

  77. PDP4E project: https://www.pdp4e-project.eu/ (accessed August 13, 2021)

  78. RobMoSys project: https://robmosys.eu/ (accessed August 13, 2021)

  79. Hawkins, R., Richardson, T., Kelly, T.: Using Process Models in System Assurance. 35th International Conference on Computer Safety, Reliability, and Security (SAFECOMP 2016)

  80. OPENCOSS Project: Deliverable 1.2 - Industrial use cases: Description and business impact, v1.0, 2012

  81. OPENCOSS Project: Deliverable 1.4 - Implementation of use cases on top of OPENCOSS platform, v1.0, 2015

  82. AMASS Project: Deliverable D1.1–Case studies description and business impact, v1.3. 2018

  83. OSLC: https://open-services.net/ (accessed August 13, 2021)

  84. iRel4.0 project: https://www.irel40.eu/ (accessed August 13, 2021)

  85. VALU3S project: https://valu3s.eu/ (accessed August 13, 2021)

Download references

Acknowledgements

The work leading to this paper received funding from the AMASS (H2020-ECSEL grant agreement no 692474; Spain’s MINECO ref. PCIN-2015-262), iRel4.0 (H2020-ECSEL grant agreement no 876659; Spain’s MICINN ref. PCI2020-112240), OPENCOSS (FP7 grant agreement no 289011), VALU3S (H2020-ECSEL grant agreement no 876852; Spain’s MICINN ref. PCI2020-112001), ETHEREAL (Spain’s MICINN ref. PID2020-115220RB-C21 y PID2020-115220RB-C22), and Treasure (JCCM ref. SBPLY/19/180501/000270; EC’s European Regional Development Fund) projects and from the Ramon y Cajal Program (Spain’s MICINN ref. RYC-2017-22836; EC’s European Social Fund). We thank all the OPENCOSS and AMASS partners that contributed to envisioning, implementing, and validating the approach presented.

Funding

Electronic Components and Systems for European Leadership, 692474, 876659, 876852, European Social Fund, RYC-2017–22836, FP7 Information and Communication Technologies, 289011, Ministerio de Ciencia e Innovación, PCI2020-112001, PCI2020-112240, PID2020-115220RB-C21, PID2020-115220RB-C22, RYC-2017–22836, European Regional Development Fund, SBPLY/19/180501/000270, Secretaría de Estado de Investigación, Desarrollo e Innovación, PCIN-2015–262, Junta de Comunidades de Castilla-La Mancha, SBPLY/19/180501/000270

Author information

Authors and Affiliations

  1. Universidad de Castilla-La Mancha, Albacete, Spain

    Jose Luis de la Vara & Arturo S. García

  2. Madrid, Spain

    Jorge Valero

  3. Tree Technology S.A, Madrid, Spain

    Clara Ayora

Authors
  1. Jose Luis de la Vara
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arturo S. García
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jorge Valero
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Clara Ayora
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jose Luis de la Vara.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix A

Appendix A

1.1 Elements of the assurance evidence metamodel

1.1.1 Artifact Element (abstract)

This class (Fig. 

Fig. 12
figure 12

Assurance evidence metamodel fragment: Artifact Element

Full size image

12) generalises the rest of classes of the metamodel.


Attributes

  • id: String

    The ID of the Artifact Element.

  • name: String

    The name of the Artifact Element.

  • description: String

    A description of the Artifact Element.

Semantics


Artifact Element represents the fact that all the objects of an assurance evidence model can have an ID, a name, and a description.

1.1.2 Artifact definition

This class corresponds to a distinguishable abstract unit of data to manage in an assurance project (Fig. 

Fig. 13
figure 13

Assurance evidence metamodel fragment: Artifact Definition

Full size image

13). Artifact Definition depicts the complete lifecycle resulting from the evolution, in different versions, of Artifacts. An Artifact Definition would be specified for, e.g., a hazard log. Each requirement of a requirements specification could have its own Artifact Definition.


Superclass

  • Artifact Element

Relationships

  • artifact: Artifact [0..*]

    The Artifacts of the Artifact Definition.

Semantics


The evidence artifacts managed in an assurance project can evolve during the project. For example, a hazard log can be created at the beginning of a project, and it will be updated as safety requirements are specified, verification measures are defined, and verification actions are taken. In other words, the hazard log will have different versions during the project. Each individual version is represented by means of an Artifact (see next class description). In an assurance project, it is necessary not only to record the information about the individual versions, but also to keep track in a common place of the different versions and of how the artifact has evolved. This is done with Artifact Definition. Intuitively, Artifact Definition records all the information of the lifecycle of an evidence artifact of an assurance project, i.e., it represents the lifecycle of the evidence artifact.

1.1.3 Artifact

This class corresponds to a distinguishable, concrete, and individual unit of data managed in an assurance project (Fig. 

Fig. 14
figure 14

Assurance evidence metamodel fragment: Artifact

Full size image

14).


Superclass

  • Artifact Element

Attributes

  • versionID: String

    The version number or ID of the Artifact.

  • date: Date

    The date of creation of the current Artifact version.

  • changes: String

    The list of changes describing any update regarding the previous Artifact version.

  • isLatestVersion: Boolean

    A flag to indicate whether the Artifact version is the latest one.

  • isTemplate: Boolean

    A flag indicating whether the Artifact is a Template to create an actual Artifact to be used in an assurance project.

  • isConfigurable: Boolean

    A flag indicating whether the Artifact is subject to configuration management, i.e., whether the Artifact can be configured for specific usage contexts or situations through different versions.

Relationships

  • definition: Artifact Definition [41]

    The Artifact Definition to which the Artifact belongs.

  • artifactPart: Artifact [0..*]

    The parts of the Artifact, which can represent document sections or any element composing the whole Artifact.

  • composite: Artifact [0..1]

    Compound Artifact of which the Artifact is part.

  • precedentVersion: Artifact [0..1]

    A pointer to the previous version of the Artifact.

  • succeedingVersion: Artifact [0..*]

    A pointer to the next versions of the Artifact.

  • event: Artifact Event [0..*]

    The events of which the lifecycle of an Artifact consists.

  • Property: Artifact Property [0..*]

    A set of properties that characterise an Artifact.

  • originatedRel: Artifact Relationship [0..*]

    The Artifact Relationships whose source is the Artifact.

  • endedRel: Artifact Relationship [0..*]

    The Artifact Relationships whose target is the Artifact.

  • recordedRel: Artifact Relationship [0..*]

    The Artifact Relationships whose data are in the Artifact.

  • resource: Resource [0..*]

    The Resource elements which represent tangible objects of an artifact, e.g., the set of architectural model files of an Architecture Design document.

  • userActivity: Activity [0..*]

    The activities for which an Artifact is input.

  • producerActivity: Activity [0..*]

    The activities for which an Artifact is output.

  • technique: Technique [0..*]

    The techniques used to produce the Artifact.

  • responsible: Participant [0..*]

    The set of Participants responsible for the management of the Artifact.


Semantics


In contrast to Artifact Definition, which is used to represent the whole lifecycle of an evidence artifact of an assurance project, Artifact is used to represent the individual versions of an evidence artifact during an assurance project. This is necessary to be able to ascertain the version of an artifact used as evidence of a claim in an assurance case, the version used as input in a given activity, or the version related to another artifact, among other usages. An Artifact specifies the instance of artifacts characterized for a version and a set of resources. Artifacts are subject to traceability, e.g., for change management, and to characterization by means of properties. An Artifact can be composed of other artifacts.

1.1.4 Artifact event

This class corresponds to relevant happenings in the lifecycle of an Artifact (Fig. 

Fig. 15
figure 15

Assurance evidence metamodel fragment: Artifact Event

Full size image

15). This serves to maintain a history log for Artifacts.


Superclass

  • Artifact Element

Attributes

  • type: Event Kind

    The type of happening of an Artifact Event.

  • date: Date

    The date (and time) when an Artifact Event occurred.

Relationships

  • owner: Artifact [41]

    The Artifact for which the Artifact Event has happened.

  • evaluation: Artifact Evaluation [0..*]

    The Artifact Evaluations that result from an Artifact Event.

  • participant: Participant [0..*]

    The Participant or set of Participants that trigger the Artifact Event.

Semantics


Artifacts change during an assurance project: someone creates them, someone makes some modification, someone fixes some problems, etc. Artifact Event is used to represent the happenings that correspond to these changes. Artifact Events can be consulted to know how an Artifact has evolved and to develop confidence in its adequate management.

1.1.5 Artifact property

This class corresponds to a characteristic of an Artifact (Fig. 

Fig. 16
figure 16

Assurance evidence metamodel fragment: Artifact Property

Full size image

16), generally depicted with an attribute (name) and a value.


Superclass

  • Artifact Element

Attributes

  • dataType: Data Type Kind

    The type of the data used to represent the values of an Artifact Property.

  • value: String

    The value of the Artifact Property.

  • unit: String

    The measurement unit corresponding to Artifact Property value.

Relationships

  • owner: Artifact [41]

    The Artifact that the Artifact Property characterises.

Semantics


Artifacts can have characteristics that must be recorded for assurance purposes. For example, it is necessary to know whether a given test case is passed or not. Such characteristics are represented by means of Artifact Property. Artifact Properties correspond to objective characteristics. For judgement-based (or subjective) characteristics, Artifact Evaluation must be used.

1.1.6 Artifact evaluation

This class corresponds to the specification of the result of making a judgement regarding an Artifact (Fig. 

Fig. 17
figure 17

Assurance evidence metamodel fragment: Artifact Evaluation

Full size image

17).

Superclass

  • Artifact Property

Attributes

  • rationale: String

    The justification of the value of an Artifact Evaluation.

Relationships

  • participant: Participant [0..*]

    The Participant or set of Participants that has performed an Evaluation.

Semantics


In addition to Artifact Properties, it can also be necessary to record judgement-based characteristics of Artifacts for an assurance project, typically about their quality (completeness, accuracy, consistency, etc.). It is important to record the rationale behind an Artifact Evaluation to understand the judgement made and how it has led to the evaluation result. The name of an Artifact Evaluation corresponds to the criterion used for evaluation, e.g., completeness.

1.1.7 Artifact relationship

This class corresponds to the existence of a relationship between two Artifacts (Fig. 

Fig. 18
figure 18

Assurance evidence metamodel fragment: Artifact Relationship

Full size image

18).

Superclass

  • Artifact Element

Attributes

  • sourceModificationEffect: Change Effect Kind

    The effect of the modification of the source on the target of an Artifact Relationship.

  • sourceRevocationEffect: Change Effect Kind

    The effect of the revocation of the source on the target of an Artifact Relationship.

  • targetModificationEffect: Change Effect Kind

    The effect of the modification of the target on the source of an Artifact Relationship.

  • targetRevocationEffect: Change Effect Kind

    The effect of the revocation of the target on the source of an Artifact Relationship.

Relationships

  • source: Artifact [41]

    The Artifact corresponding to the source of an Artifact Relationship.

  • target: Artifact [41]

    The artifact corresponding to the target of an artifact relationship.

  • artifact: Artifact [0..1]

The Artifact actually containing the data of the Artifact Relationship, e.g., a document that contains a traceability matrix.

Semantics

Artifact Relationships are the main mechanism for establishing bilateral traceability between artifacts, e.g., the relationship by which a test validates a requirement, and the requirement is validated by the test. Information about modification and revocation effects is important for change impact analysis.

1.1.8 Resource

This class corresponds to the tangible objects representing an Artifact (Fig. 

Fig. 19
figure 19

Assurance evidence metamodel fragment: Resource

Full size image

19).

Superclass

  • Artifact Element

Attributes

  • location: String

    The path or URL specifying the location of the Resource.

  • format: String

    The format of the resource, e.g., MS Word.

Relationships

  • owner: Artifact [41]

    The Artifact for which the Resource is a tangible object.

Semantics

Artifacts are located and accessible somewhere, usually in the form of some electronic file: a Word file, an Excel file, a file created with some modeling tool, etc. Such information is specified by means of Resource.

1.1.9 Activity

This class corresponds to a unit of work performed in a product lifecycle (Fig. 

Fig. 20
figure 20

Assurance evidence metamodel fragment: Activity

Full size image

20).

Superclass

  • Artifact Element

Attributes

  • startTime: Date

    When an Activity starts.

  • endTime: Date

    When an Activity ends.

Relationships

  • subActivity: Activity [0..*]

    Activities of which an Activity consists.

  • composite: Activity [0..1]

    Compound Activity of which an Activity is part.

  • successor: Activity [0..*]

    The Activities that are performed after an Activity.

  • predecessor: Activity [0..*]

    The Activities that are performed before an Activity.

  • input: Artifact [0..*]

    The Artifacts used in an Activity.

  • output: Artifact [0..*]

    The Artifacts produced or changed in an Activity.

  • technique: Technique [0..*]

    The set of Techniques used in the Activity.

  • participant: Participant [0..*]

    The set of Participants involved in the Activity.

Semantics

The Artifacts used in an assurance project are the result of and are managed via the execution of processes, which consist of activities: specification of requirements, design of the system, integration of system components, etc. Activities are used to represent this kind of information. An Activity is a specification of an activity already executed or under execution.

1.1.10 Technique

This class corresponds to specific ways to create an Artifact (Fig. 

Fig. 21
figure 21

Assurance evidence metamodel fragment: Technique

Full size image

21).

Superclass

  • Artifact Element

Attributes

  • Aim: String

    The purpose of using a Technique.

Relationships

  • artifact: Artifact [0..*]

    The set of Artifacts generated using the Technique.

  • activity: Activity [0..*]

    The set of Activities in which a Technique has been used.

Semantics

Artifacts are created, or managed from a more general perspective, via some technique (methods, approaches, languages…) whose use results in specific characteristics for the Artifacts. For example, the use of UML for system design results in a design specification with a set of UML diagrams that could represent static and dynamic internal aspects of the system. Techniques support the specification of this kind of information.

1.1.11 Participant (abstract)

This class corresponds to the concrete parties involved in a product lifecycle (Fig. 

Fig. 22
figure 22

Assurance evidence metamodel fragment: Participant

Full size image

22).

Superclass

  • Artifact Element

Relationships

  • artifact: Artifact [0..*]

    The Artifacts of which a Participant is responsible.

  • activity: Activity [0..*]

    The Activities in which a Participant is involved.

  • event: Artifact Event [0..*]

    The events that a Participant triggers.

  • evaluation: Artifact Evaluation [0..*]

    The evaluations that a Participant conducts.

Semantics

Different parties can participate in an assurance case effort, such as specific people, organizations, and tools.

1.1.12 Person

This class corresponds to individuals that are involved in a product lifecycle (Fig. 

Fig. 23
figure 23

Assurance evidence metamodel fragment: Person

Full size image

23).

Superclass

  • Participant

Attributes

  • email: String

    The email address of a person.

Relationships

  • organization: Organization [0..*]

    The organization for which a person works.\

Semantics

Concrete people participate in the lifecycle of a safety–critical system, e.g., the managers and the engineers of a safety–critical system manufacturer.

1.1.13 Tool

This class corresponds to the software tools used in a product lifecycle (Fig. 

Fig. 24
figure 24

Assurance evidence metamodel fragment: Tool

Full size image

24).

Superclass

  • Participant

Attributes

  • version: String

The version in use of a tool.

Semantics

Safety–critical systems engineering is a complex process that requires the use of tool support for cost-effectiveness. Such tool support usually covers the complete lifecycle stages. Dedicated tools are commonly used for system development (e.g., for requirements management and for coding) and for V&V (e.g., for testing).

1.1.14 Organization

This class corresponds to groups of people (companies, societies, associations, etc.) that are involved in a product lifecycle (Fig. 

Fig. 25
figure 25

Assurance evidence metamodel fragment: Organization

Full size image

25).

Superclass

  • Participant

Attributes

  • address: String

The place where an organization is located.

Relationships

  • suborganization: Organization [0..*]

The Organizations that are part of the Organization, e.g., a unit.

  • composite: Organization [0..1]

The Organization to which an Organization belongs.

  • person: Person [0..*]

Persons that work at an Organization.

Semantics

In addition to individuals and tools, organizations can also participate in the lifecycle of safety–critical systems. The organizations can correspond to entities such as component suppliers and assessors. They can provide artifacts that can be used as assurance evidence, such as component specifications.

1.1.15 Event Kind (enumeration)

This enumeration corresponds to types of events that can occur in the lifecycle of an Artifact.

Literals

  • Creation

When an Artifact is brought into existence.

  • Modification

When a change is made in a characteristic of an Artifact.

  • Evaluation

When an Artifact is evaluated.

  • Revocation

When an Artifact is revoked from an assurance project.

1.1.16 Data type kind (enumeration)

This enumeration corresponds to types of values for Artifact Properties.

Literals

  • String

The value space characterized by a string.

  • Integer

A value space characterized by integer numbers.

  • Float

A value space characterized by real numbers.

1.1.17 Change effect kind (enumeration)

This enumeration corresponds to the possible effects that a change in an Artifact can have in a related Artifact.

Literals

  • None

A change has no effect.

  • ToValidate

A change requires a validation.

  • ToModify

A change requires a modification.

  • Modification

A change causes a modification.

  • Revocation

A change causes a revocation.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

de la Vara, J.L., García, A.S., Valero, J. et al. Model-based assurance evidence management for safety–critical systems. Softw Syst Model 21, 2329–2365 (2022). https://doi.org/10.1007/s10270-021-00957-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10270-021-00957-z

Keywords

Search

Navigation