Skip to main content

Advertisement

SpringerLink
Log in

How assurance case development and requirements engineering interplay: a study with practitioners

  • Original Article
  • Published:
Requirements Engineering Aims and scope Submit manuscript

Abstract

Assurance case development (ACD) is a novel approach for demonstrating that a system is safe for use. Assurance cases comprise various project information, including requirements and their traceability to other artifacts. A practitioners’ survey was performed to understand how requirements engineering and ACD activities currently interplay. This study aimed to identify the state of practice of ACD, the existing integration between requirements engineering and ACD, and the practitioners’ opinion on this integration. The results revealed that the interplay occurs across all requirement engineering activities and that practitioners perceive benefits such as raising safety assurance awareness, early traceability development, and early identification of assurance evidence needs. Practitioners see requirements specification and change request analysis as the most suitable activities for integrating ACD and requirements engineering.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. https://www.adelard.com/asce/choosing-asce/index/.

  2. http://www.goalstructuringnotation.info/.

References

  1. Almendra C, Silva C, Vilela J (2020) Incremental development of safety cases: a mapping study. In: Proceedings of the 34th Brazilian symposium on software engineering, pp 538–547. https://doi.org/10.1145/3422392.3422398

  2. Bloomfield R, Bishop P (2010) Safety and assurance cases: past, present and possible future—an adelard perspective. In: Making systems safer, pp 51–67. https://doi.org/10.1007/978-1-84996-086-1_4

  3. Cheng J, Goodrum M, Metoyer R, Cleland-Huang J (2018) How do practitioners perceive assurance cases in safety-critical software systems?. In: Proceedings of the 11th international workshop on cooperative and human aspects of software engineering, pp 57–60. https://doi.org/10.1145/3195836.3195838

  4. de la Vara JL, Borg M, Wnuk K, Moonen L (2016) An industrial survey of safety evidence change impact analysis practice. IEEE Trans Software Eng 42(12):1095–1117. https://doi.org/10.1109/TSE.2016.2553032

    Article  Google Scholar 

  5. de la Vara JL, Ruiz A, Espinoza H (2018) Recent advances towards the industrial application of model-driven engineering for assurance of safety-critical systems. In: Proceedings of the 6th international conference on model-driven engineering and software development, (Modelsward), pp 632–641. https://doi.org/10.5220/0006733906320641

  6. Doss O, Kelly TP (2016) Challenges and opportunities in agile development in safety critical systems: a survey. SIGSOFT Softw Eng Notes 41(2):30–31. https://doi.org/10.1145/2894784.2894798

    Article  Google Scholar 

  7. Fernández DM, Wagner S, Kalinowski M et al (2017) Naming the pain in requirements engineering. Empir Softw Eng 22(5):2298–2338. https://doi.org/10.1007/s10664-016-9451-7

    Article  Google Scholar 

  8. Hatcliff J, Wassyng A, Kelly T, Comar C, Jones P (2014) Certifiably safe software-dependent systems: challenges and directions. In: Future of software engineering, FOSE 2014—Proceedings, pp 182–200. https://doi.org/10.1145/2593882.2593895

  9. Hawkins R, Habli I, Kelly T, McDermid J (2013) Assurance cases and prescriptive software safety certification: a comparative study. Saf Sci 59:55–71. https://doi.org/10.1016/j.ssci.2013.04.007

    Article  Google Scholar 

  10. Heeager LT, Nielsen PA (2018) A conceptual model of agile software development in a safety-critical context: a systematic literature review. Inf Softw Technol 103:22–39. https://doi.org/10.1016/j.infsof.2018.06.004

    Article  Google Scholar 

  11. Holloway CM (2008) Safety case notations: alternatives for the non-graphically inclined. IET Conf Publ. https://doi.org/10.1049/cp:20080710

    Article  Google Scholar 

  12. IEEE Std 15026-2-2011 2011 (2011) IEEE standard—adoption of ISO/IEC 15026-2: 2011 systems and software engineering—systems and software assurance–part 2: assurance case. Standard. IEEE Computer Society, USA

  13. Kasunic M (2005. Designing an effective survey. Software Engineering Institute

  14. Kelly T (2018) Safety cases. In: Handbook of safety principles, pp. 361–385. https://doi.org/10.1002/9781119443070.ch16

  15. Kitchenham BA, Pfleeger SL (2008) Personal opinion surveys. In: Guide to advanced empirical software engineering, pp. 63–92. https://doi.org/10.1007/978-1-84800-044-5_3

  16. Langari Z, Maibaum T (2013) Safety cases: a review of challenges. In: 2013 1st international workshop on assurance cases for software-intensive systems, ASSURE 2013—Proceedings, pp 1–6. https://doi.org/10.1109/ASSURE.2013.6614263

  17. Linåker J, Sulaman SM, de Mello R, Höst M (2015) Guidelines for conducting surveys in software engineering. Retrieved from https://lup.lub.lu.se/record/5366801

  18. Mäder P, Jones P, Zhang Y, Cleland-Huang J (2013) Strategic traceability for safety-critical projects. IEEE Softw 30(3):58–66. https://doi.org/10.1109/MS.2013.60

    Article  Google Scholar 

  19. Martins LEG, Gorschek T (2016) Requirements engineering for safety-critical systems: a systematic literature review. Inf Softw Technol 75:71–89. https://doi.org/10.1016/j.infsof.2016.04.002

    Article  Google Scholar 

  20. Martins LEG, Gorschek T (2020) Requirements engineering for safety-critical systems: an interview study with industry practitioners. IEEE Trans Softw Eng 46(4):346–361. https://doi.org/10.1109/TSE.2018.2854716

    Article  Google Scholar 

  21. Molléri JS, Petersen K, Mendes E (2020) An empirically evaluated checklist for surveys in software engineering. Inf Softw Technol 119:106240. https://doi.org/10.1016/j.infsof.2019.106240

    Article  Google Scholar 

  22. Myklebust T, Hanssen G, Lyngby N (2017) A survey of the software and safety case development practice in the railway signalling sector. Saf Reliabil Theory Appl. https://doi.org/10.1201/9781315210469-426

    Article  Google Scholar 

  23. Nair S, de la Vara JL, Sabetzadeh M, Briand L (2014) An extended systematic literature review on provision of evidence for safety certification. Inf Softw Technol 56(7):689–717. https://doi.org/10.1016/j.infsof.2014.03.001

    Article  Google Scholar 

  24. Nair S, de la Vara JL, Sabetzadeh M, Falessi D (2015) Evidence management for compliance of critical systems with safety standards: a survey on the state of practice. Inf Softw Technol 60:1–15. https://doi.org/10.1016/j.infsof.2014.12.002

    Article  Google Scholar 

  25. Popping R (2015) Analyzing open-ended questions by means of text analysis procedures. Bull Sociol Methodol Bull Méthodol Sociol 128(1):23–39. https://doi.org/10.1177/0759106315597389

    Article  Google Scholar 

  26. Rinehart DJ, Knight JC, Rowanhill J (2017) Understanding what it means for assurance cases to “Work”. Retrieved from https://ntrs.nasa.gov/citations/20170003806

  27. Sikora E, Tenbergen B, Pohl K (2012) Industry needs and research directions in requirements engineering for embedded systems. Requir Eng 17:57–78. https://doi.org/10.1007/s00766-011-0144-x

    Article  Google Scholar 

  28. Wagner S, Fernández DM, Felderer M et al (2019) Status quo in requirements engineering: a theory and a global family of surveys. ACM Trans Softw Eng Methodol 28(2):1–48. https://doi.org/10.1145/3306607

    Article  Google Scholar 

  29. Wohlin C, Runeson P, Höst M, Ohlsson MC, Regnell B, Wesslén A (2012) Experimentation in software engineering. https://doi.org/10.1007/978-3-642-29044-2

Download references

Acknowledgements

We would like to thank all the participants who contributed to our survey. We also thank the experts that participated in the pilot study.

Funding

We thank CNPQ/Brazil (Conselho Nacional de Desenvolvimento Científico e Tecnológico) for the financial support to the execution of this work, Universidade Federal do Ceará, DARE Research Group from Universidade Federal de Pernambuco, Universidade Federal de São Paulo, and Instituto Tecnológico de Aeronáutica.

Author information

Authors and Affiliations

  1. Campus Quixadá, Universidade Federal do Ceará, CE, Quixadá, Brazil

    Camilo Almendra

  2. Centro de Informática, Universidade Federal de Pernambuco, Recife, PE, Brazil

    Carla Silva

  3. Universidade Federal de São Paulo, São Paulo, SP, Brazil

    Luiz Eduardo G. Martins

  4. Instituto Tecnológico de Aeronáutica, São José dos Campos, SP, Brazil

    Johnny Marques

Authors
  1. Camilo Almendra
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carla Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Luiz Eduardo G. Martins
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Johnny Marques
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

CA contributed to conceptualization, methodology, investigation, data curation, writing—original draft, and writing—review and editing. CS, JM, and LEGM were involved in conceptualization, methodology, investigation, and writing—review and editing.

Corresponding author

Correspondence to Camilo Almendra.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix

It contains the survey questionnaire presented to participants. It comprises a short introduction and 21 questions structured in four groups.

Survey introduction

The target audience of this survey is researchers and professionals working on safety–critical systems development projects in which assurance (safety) cases are produced as part of the development process.

2.1 Group 1—professional profile

1. How did you find this survey?

( ) Post on LinkedIn

( ) Invitation via LinkedIn

( ) Post on a mailing list

( ) Invitation via Email

( ) Post on Research Gate

( ) Invitation via Research Gate

( ) Less than 1 year

( ) Other—please specify: [open text]

2. How long have you been working on development of safety–critical systems?

( ) 1–2 years

( ) 3–5 years

( ) 6–10 years

( ) More than 10 years

3. What is your current main role in the organization?

[open text]

3. How many projects that involved Assurance Cases for safety–critical systems have you participated in?

( ) Less than 5 projects

( ) 5–10 projects

( ) More than 10 projects

4. In which country(ies) have you principally worked upon assurance cases or requirements engineering for safety–critical systems?

[open text]

5. What is the main role of the organization for which you have worked regarding the development of safety–critical systems?

( ) Certification authority

( ) Component supplier

( ) Consultant

( ) Developer/manufacturer of final systems

( ) Development tool vendor

( ) Independent safety assessor

( ) Regulation authority

( ) Research institution

( ) System user

( ) Other—please specify: [open text]

2.2 Group 2—assurance case development practices

Please answer the questions below taking into account your experience in the past 5 years.

6. Is the development of Assurance Cases a mandatory, recommended or optional activity?

Please choose all that apply.

( ) Mandatory by regulations

( ) Mandatory by my organization policy/process

( ) Recommended by regulations

( ) Recommended by my organization policy/process

( ) Optionally performed by the project team

( ) Don’t know

7. How often have you been involved in these following assurance case development activities?

  • Construction

  • Review

  • Assessment

  • Rebuttal

  • Approval

For each activity, the options are:

( ) Every project

( ) Most of the projects

( ) Some projects

( ) Few projects

( ) Never have been involved

8. Which of the following statements apply to how Assurance Cases (ACs) are developed in your projects?

Please choose all that apply.

( ) ACs are constructed pre-development

( ) ACs are reviewed pre-development

( ) ACs are constructed during development in a parallel process/lifecycle

( ) ACs are reviewed during development in a parallel process/lifecycle

( ) ACs are constructed incrementally according to project iterations/sprints/milestones

( ) ACs are reviewed incrementally according to project iterations/sprints/milestones

( ) ACs are only constructed post-development

( ) Other practice - please specify: [open text]

9. Which is the main notation used to construct the Assurance Arguments?

( ) ARM

( ) ASCAD

( ) Bowtie

( ) CAE

( ) Claims table

( ) EUROCONTROL

( ) GSN

( ) KAOS

( ) MDD (MultiMarkdown doc.)

( ) Narrative

( ) SACM

( ) Tabular

( ) TRUST-IT Argument Representation

( ) WeFA

( ) Structured textual

( ) Textual (plain text)

( ) Other—please specify: [open text]

10. Which of the following scenarios best describe how Assurance Case Reports are organized considering only one system to be certified:

( ) One assurance case report containing only one comprehensive assurance argument.

( ) One assurance case report containing multiple assurance arguments.

( ) Multiple assurance case reports, each containing only one assurance argument each.

( ) Multiple assurance case reports, each containing more than one assurance arguments.

( ) Other—please specify: [open text]

11. Is there any kind of automation in the creation or maintenance of Assurance Arguments and Reports?

[open text]

2.3 Group 3—integrated development practices

Please answer the questions below taking into account your experience in the past 5 years.

13. Considering the management of traceability among requirements and safety analysis information, which of the following scenarios best describes the most used practice in your projects?

( ) A matrix or tool is used to manage only the trace links.

( ) A tool is used to manage both artifacts and trace links.

( ) Trace links are stated inside the requirements and safety analysis artifacts.

( ) Trace links are maintained directly in the certification documents (including assurance cases).

( ) Trace links are only recovered to produce final documentation.

( ) Other—please specify: [open text]

14. How often are assurance cases consulted during requirements engineering activities?

  • Elicitation

  • Analysis and negotiation

  • Specification

  • Customer/stakeholder validation

  • Change request analysis

For each activity, the options are:

( ) Always

( ) Very Often

( ) Sometimes

( ) Rarely

( ) Never

( ) Don’t know

15. How often does your team consider the safety assurance when performing requirements engineering activities?

( ) Always

( ) Very Often

( ) Sometimes

( ) Rarely

( ) Never

( ) Don’t know

15. Which of the following collaborations between requirements and safety specialists occurs and in which frequency?

  • Requirements engineers participate in assurance case development activities.

  • Requirements engineers participate in other safety analysis activities.

  • Requirements engineers review safety engineering results/artifacts.

  • Safety engineers participate in requirements engineering activities.

  • Safety engineers review requirements engineering results/artifacts.

For each collaboration scenario, the options are:

( ) Every project

( ) Most of the projects

( ) Some projects

( ) Few projects

( ) Never

( ) Don’t know

2.4 Group 4—personal opinion on integration

(Q17 appeared only for those who selected Few projects or above in any of the first three collaboration scenarios of Q16).

16.What are the benefits of requirements engineers participation in assurance case development or safety analysis development activities?

[open text]

(Q18 appeared only for those who selected Few projects or above in any of the last two collaboration scenarios of Q16)

17. What are the benefits of Safety Engineers participation in requirements engineering activities?

[open text]

18. How much do you agree with the development of assurance cases during requirements engineering activities?

( ) Strongly agree

( ) Agree

( ) Undecided

( ) Disagree

( ) Strongly disagree

18. To what extent do you believe the development of assurance cases during requirements engineering activities could mitigate the following problems in SCS development?

  • Late rework due to impossibility to build a compelling assurance case.

  • Poor quality (incompleteness or inconsistency) of assurance cases.

  • Loss of requirements and design rationale.

  • Late discovery of safety requirements or constraints.

  • Lack of integration between safety engineering and software development.

For each problem, the options are:

( ) To A Great Extent

( ) Somewhat

( ) Very Little

( ) Not At All

( ) Undecided

19. Which requirements engineering activities do you think the development or sketching of assurance cases could be combined with?

Mark all that apply. You may provide an explanation in the open text field.

( ) Elicitation: [open text]

( ) Analysis and negotiation: [open text]

( ) Specification: [open text]

( ) Customer/stakeholder validation: [open text]

( ) Change request analysis: [open text]

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Almendra, C., Silva, C., Martins, L.E.G. et al. How assurance case development and requirements engineering interplay: a study with practitioners. Requirements Eng 27, 273–292 (2022). https://doi.org/10.1007/s00766-022-00375-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00766-022-00375-7

Keywords

Search

Navigation