skip to main content
10.1145/2593882.2593895acmconferencesArticle/Chapter ViewAbstractPublication PagesicseConference Proceedingsconference-collections
Article

Certifiably safe software-dependent systems: challenges and directions

Published: 31 May 2014 Publication History

Abstract

The amount and impact of software-dependence in critical systems impinging on daily life is increasing rapidly. In many of these systems, inadequate software and systems engineering can lead to economic disaster, injuries or death. Society generally does not recognize the potential of losses from deficiencies of systems due to software until after some mishap occurs. Then there is an outcry, reflecting societal expectations; however, few know what it takes to achieve the expected safety and, in general, loss-prevention.
On the one hand there are unprecedented, exponential increases in size, inter-dependencies, intricacies, numbers and variety in the systems and distribution of development processes across organizations and cultures. On the other hand, industry's capability to verify and validate these systems has not kept up. Mere compliance with existing standards, techniques, and regulations cannot guarantee the safety properties of these systems. The gap between practice and capability is increasing rapidly.
This paper considers the future of software engineering as needed to support development and certification of safety-critical software-dependent systems. We identify a collection of challenges and document their current state, the desired state, gaps and barriers to reaching the desired state, and potential directions in software engineering research and education that could address the gaps and barriers.

References

[1]
Ministry of defence defence standard 23-09 – generic vehicle architecture, aug 2010.
[2]
ACM and IEEE Computer Society. Software engineering 2004, curriculum guidelines for undergraduate degree programs in software engineering. http://sites. computer.org/ccse/SE2004Volume.pdf, 2004.
[3]
Associated Press. Okla. jury: Toyota liable in sudden acceleration crash. http: //www.cbsnews.com/news/okla-jury-toyotaliable-in-sudden-acceleration-crash/, 2013.
[4]
AUTomotive Open System ARchitecture. AUTOSAR. https://www.autosar.org, 2014.
[5]
J. Barnes. High Integrity Software—the SPARK Approach to Safety and Security. Addison-Wesley, 2003.
[6]
C. Boettcher, R. DeLong, J. Rushby, and W. Sifre. The MILS component integration approach to secure information sharing. In Proceedings of the 27th IEEE/AIAA Digital Avionics Systems Conference (DASC), Oct. 2008.
[7]
Boston Scientific. PACEMAKER system requirements specification. http://sqrl.mcmaster.ca/pacemaker.htm, 2007.
[8]
Carnegie Mellon. Entertainment Technology Center. http://www.etc.cmu.edu/site/, 2014.
[9]
R. N. Charette. This Car Runs on Code. IEEE Spectrum, Feb. 2009.
[10]
R. DeLong and J. Rushby. A common criteria authoring environment supporting composition. In Proceedings of the 8th International Common Criteria Conference, 2007.
[11]
B. Dion. FACE, ARINC, DO-178C avionics standards help U.S. DoD’s vision of reusable technology to take off. Military Embedded Systems, 9(2):36–39, mar 2013.
[12]
M. B. Dwyer and S. G. Elbaum. Unifying verification and validation techniques: relating behavior and properties through partial evidence. In Proceedings of the Workshop on Future of Software Engineering Research (FoSER 2010), pages 93–98, 2010.
[13]
C. A. Ericson. Hazard Analysis Techniques for System Safety. Wiley-Interscience, 2005.
[14]
European Committee for Electrical Standardization (CENELEC). Railway applications - Communication, signalling and processing systems - Software for railway control and protection systems. CENELEC Standard 50128, 2011.
[15]
P. Feiler. Architecture Analysis and Design Language (AADL) Annex Volume 3: Annex E: Error Model V2 Annex. Number SAE AS5506/3 (Draft) in SAE Aerospace Standard. SAE International, 2013.
[16]
P. Feiler and D. Gluch. Model-Based Engineering with AADL: An Introduction to the SAE Architecture Analysis and Design Language. Addison-Wesley, 2012.
[17]
E. Feliz. Engineering pipeline under pressure. Avionics Today, December 2007.
[18]
N. Gershenfeld, R. Krikorian, and D. Cohen. The internet of things. Scientific American, Oct. 2004.
[19]
S. Grigorova and T. Maibaum. Taking a page from the law books: Considering evidence weight in evaluating assurance case confidence. In ISSRE (Supplemental Proceedings), pages 387–390, 2013.
[20]
C. Hagen, S. Hurt, J. Sorenson, and D. Wall. Software: The brains behind us defense systems. A.T. Kearney Whitepaper, Nov 2012.
[21]
D. Harel and R. Marelly. Come, Let’s Play: Scenario-Based Programming Using LSCs and the Play-Engine. Springer, 2003.
[22]
J. Hatcliff, M. Heimdahl, M. Lawford, T. Maibaum, A. Wassyng, and F. Wurden. A Software Certification Consortium and its Top 9 Hurdles. Electronic Notes in Theoretical Computer Science, 238(4):11–17, Sept. 2009.
[23]
J. Hatcliff, A. King, I. Lee, A. Fernandez, J. Goldman, A. McDonald, M. Robkin, E. Vasserman, and S. Weininger. Rationale and architecture principles for medical application platforms. In Proceedings of the 2012 International Conference on Cyberphysical Systems, 2012.
[24]
J. Hatcliff, A. Wassyng, T. Kelly, and C. Comar. Certifiably safe software-dependent systems: Challenges and directions (paper web site / extended version). http://santoslab.org/pub/papers/fose14certification/.
[25]
R. Hawkins, K. Clegg, R. Alexander, and T. Kelly. Using a software safety argument pattern catalogue: two case studies. In Computer Safety, Reliability, and Security, pages 185–198. Springer, 2011.
[26]
R. Hawkins, I. Habli, T. Kelly, et al. Principled construction of software safety cases. In Proceedings of Workshop SASSUR (Next Generation of System Assurance Approaches for Safety-Critical Systems) of the 32nd International Conference on Computer Safety, Reliability and Security, 2013.
[27]
M. P. Heimdahl. Safety and software intensive systems: Challenges old and new. In 2007 Future of Software Engineering, pages 137–152. IEEE Computer Society, 2007.
[28]
J. Hillebrand, P. Reichenpfader, I. Mandic, H. Siegl, and C. Peer. Establishing confidence in the usage of software tools in context of iso 26262. In Computer Safety, Reliability, and Security - 30th International Conference (SAFECOMP 2011), pages 257–269, 2011.
[29]
C. M. Holloway. Making the implicit explicit: Towards an assurance case for do-178c. In Proceedings of the 31st International System Safety Conference (ISSC), 2013.
[30]
IEEE Computer Society. Software engineering body of knowledge. http://www.computer.org/portal/ web/swebok/html/contents, 2004.
[31]
Institute of Electrical and Electronics Engineers (IEEE). IEEE Standard Criteria for Digital Computers in Safety Systems of Nuclear Power Generating Stations. IEEE Standard 7-4.3.2, 2010.
[32]
International Electrotechnical Commission. Medical device software – Software life cycle processes. IEC Standard 62304 edition 1.0, 2006.
[33]
International Electrotechnical Commission. Functional Safety of Electrical/Electronic/Programmable Electronic Safety-related Systems. IEC Standard 61508 edition 2.0, 2010.
[34]
International Organization for Standardization. Software Engineering – Software product Quality Requirements and Evaluation (SQuaRE) – Guide to SQuaRE. ISO Standard 25000, 2005.
[35]
International Organization for Standardization. Application of Risk Management to Medical Devices. ISO Standard 14971, 2007.
[36]
International Organization for Standardization. Road Vehicles – Functional Safety. ISO Standard 26262, 2011.
[37]
International Organization for Standardization. Systems and software engineering – Systems and software assurance – Part 2: Assurance case. ISO Standard 15026 part 2, 2011.
[38]
ISO, IEC, IEEE. IEEE Standard 24765: Systems and software engineering - Vocabulary, 2010.
[39]
ISO/IEC. ISO/IEC 17000: Conformity assessment – vocabulary and general principles, 2004.
[40]
ISO/IEEE. 11073-x Medical Health Device Communication Standards family.
[41]
D. Jackson. Software Abstractions: Logic, Language, and Analysis. MIT Press, 2012.
[42]
T. Kelly. Using software architecture techniques to support the modular certification of safety-critical systems. In Proceedings of the Eleventh Australian Workshop on Safety Critical Systems and Software - Volume 69, SCS ’06, pages 53–65, 2006.
[43]
J. C. Knight. Focusing software education on engineering. ACM SIGSOFT Software Engineering Notes, 30(2):3–5, 2005.
[44]
J. C. Knight and N. G. Leveson. Software and higher education. Communications of the ACM, 49(1):160–160, 2006.
[45]
B. Larson and J. Hatcliff. Open PCA Pump project website. http://openpcapump.santoslab.org.
[46]
X. Leroy. A formally verified compiler back-end. Journal of Automated Reasoning, 43(4):363–446, Dec. 2009.
[47]
N. Leveson. Safeware: System Safety and Computers. Addison-Wesley, 1995.
[48]
N. Leveson. Engineering a Safer World: Systems Thinking Applied to Safety. MIT Press, 2012.
[49]
R. R. Lutz. Software engineering for safety: a roadmap. In Proceedings of the Conference on The Future of Software Engineering, pages 213–226. ACM, 2000.
[50]
T. Maibaum and A. Wassyng. A product-focused approach to software certification. Computer, 41(2):91–93, 2008.
[51]
Mathew J. Schwartz. Hacked Medical Device Sparks Congressional Inquiry. http://www.informationweek.com/security/ vulnerabilities-and-threats/hackedmedical-device-sparks-congressionalinquiry/d/d-id/1099726, 2011.
[52]
E. McKenna. Embedded overall. Avionics Today, Nov 2008.
[53]
Naval Postgraduate School. Department of Systems Engineering: Programs of Study. http://www.nps. edu/Academics/Schools/GSEAS/Departments/ SE/Academics/ProgramsofStudy.html, 2014.
[54]
Open platform for evolutionary certification of safety-critical systems. www.opencoss-project.eu.
[55]
D. L. Parnas. Software engineering programs are not computer science programs. Software, IEEE, 16(6):19–30, 1999.
[56]
D. L. Parnas, A. J. van Schouwen, and S. P. Kwan. Evaluation of safety-critical software. Communications of the ACM, 33(6):636–648, 1990.
[57]
Patrick J. Graydon, C. Michael Holloway. AESSCS 2014 Workshop. Planning the Unplanned Experiment: Assessing the Efficacy of Standards for Safety Critical Software. http://www.idt.mdh.se/AESSCS\_2014/, 2013.
[58]
K. Pohl, G. Böckle, and F. van der Linden. Software Product Line Engineering: Foundations, Principles and Techniques. Springer, 2005.
[59]
A. Rae, M. Nicholson, and R. Alexander. The state of practice in system safety research evaluation. In 5th IET International Conference on System Safety 2010, pages 1–8, Oct 2010.
[60]
RTCA. Software Considerations in Airborne Systems and Equipment Certification. RTCA Standard DO-178C, 2012.
[61]
J. Rushby. Modular certification. Technical report, Computer Science Laboratory, SRI International, Menlo Park, CA 94025, US, September 2001.
[62]
J. Rushby. Composing safe systems. In Proceedings of Formal Aspects of Component Software (FACS 2011), volume 7253 of Lecture Notes in Computer Science, pages 3–11, 2012.
[63]
Russell Sydnor, Sushil Birla, Michael Waterman. Gap Assessment of IEC and IEEE Standards for Safety Assurance of Digital Systems. Software Certification Consortium Meeting 9: http://cps-vo.org/node/3472, 2012.
[64]
K. Sandler, L. Ohrstrom, L. Moy, and R. McVay. Killed by Code: Software transparency in implantable medical devices. Software Freedom Law Center Whitepaper, 2010.
[65]
G. Schkade. SAE’s Perspective on Connected Vehicle. http://www.itu.int/dms\_pub/itut/oth/06/5B/T065B0000020008PDFE.pdf, 2011.
[66]
O. Slotosch. Model-based tool qualification. http://wiki.eclipse.org/Auto\_IWG\_WP5.
[67]
Society of Automotive Engineers (SAE). Arp4761: Guidelines and methods for conducting the safety assessment process on civil airborne systems and equipment, 1996.
[68]
J. Stern. Google Self-Driving Car License Approved in Nevada. http://abcnews.go.com/blogs/ technology/2012/05/google-self-drivingcar-license-approved-in-nevada/, 2012.
[69]
United Kingdom Ministry of Defence. Safety Management Requirements for Defence Systems. Defence Standard 00-56. Issue 4., 2007.
[70]
University of California, Los Angeles. UCLA Extension: Safety-Critical Software. https://www.uclaextension.edu/pages/ Course.aspx?reg=V9149, 2014.
[71]
U.S. Department of Defense. Mil-std-882e: Department of defense standard practice: System safety, 2012.
[72]
U.S. Food and Drug Administration. Safety, Recalls, Market Withdrawals, & Safety Alerts, Background and Definitions. http://www.fda.gov/Safety/Recalls/ ucm165546.htm, 2009.
[73]
U.S. Nuclear Regulatory Commission. Research information letter (ril) 1101: Technical basis to review hazard analysis of digital safety systems.
[74]
U.S. Nuclear Regulatory Commission. Research Information Letter 1001: Software-related uncertainties in the assurance of digital safety systems – expert clinic findings Part 1. http://adamswebsearch2.nrc.gov/IDMWS/ ViewDocByAccession.asp? \\AccessionNumber=ML111240017, 2011.
[75]
A. van Lamsweerde. Requirements engineering: From craft to discipline. In FSE’2008: 16th ACM Sigsoft Intl. Symposium on the Foundations of Software Engineering (Invited Paper for the ACM Sigsoft Outstanding Research Award), pages 238–249, 2008.
[76]
W. G. Vincenti. What engineers know and how they know it: Analytical studies from aeronautical history. The Johns Hopkins University Press, 1990.
[77]
A. Wassyng, M. Lawford, and T. Maibaum. Separating safety & control systems to reduce complexity. In M. Hinchey and L. Doyle, editors, Conquering Complexity, pages 89––108. Springer, 2012.
[78]
A. Wassyng and M. Lawford. Lessons learned from a successful implementation of formal methods in an industrial project. In K. Arakai, S. Gnesi, and D. Mandrioli, editor, FME 2003: International Symposium of Formal Methods Europe Proceedings, volume 2805 of Lecture Notes in Computer Science, pages 133–153. Springer-Verlag, 2003.
[79]
A. Wassyng, T. Maibaum, M. Lawford, and H. Bherer. Software certification: Is there a case against safety cases? In R. Calinescu and E. Jackson, editors, Foundations of Computer Software. Modeling, Development, and Verification of Adaptive Systems, volume 6662 of Lecture Notes in Computer Science, pages 206–227. Springer Berlin Heidelberg, 2011.
[80]
C. B. Weinstock, J. B. Goodenough, and A. Z. Klein. Measuring assurance case confidence using baconian probabilities. In Proceedings of the 1st International Workshop on Assurance Cases for Software-Intensive Systems (ASSURE), 2013.
[81]
M. Wildmoser, J. Philipps, and O. Slotosch. Determining potential errors in tool chains - strategies to reach tool confidence according to iso 26262. In Computer Safety, Reliability, and Security - 31st International Conference (SAFECOMP 2012), pages 317–327, 2012.
[82]
The Muen Separation Kernel. http://muen.codelabs.ch/.

Cited By

View all
  • (2024)From Textual to Formal Requirements: A Case Study Using Spectra in Safety-Critical Systems DomainJournal of Software Engineering Research and Development10.5753/jserd.2024.374512:1Online publication date: 4-Nov-2024
  • (2024)The Isolette System: Illustrating End-to-End Artifacts for Rigorous Model-Based EngineeringThe Combined Power of Research, Education, and Dissemination10.1007/978-3-031-73887-6_9(93-117)Online publication date: 23-Oct-2024
  • (2023)Redesigning Medical Device Assurance: Separating Technological and Clinical Assurance CasesComputer Safety, Reliability, and Security10.1007/978-3-031-40923-3_3(25-38)Online publication date: 11-Sep-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
FOSE 2014: Future of Software Engineering Proceedings
May 2014
224 pages
ISBN:9781450328654
DOI:10.1145/2593882
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

In-Cooperation

  • TCSE: IEEE Computer Society's Tech. Council on Software Engin.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 31 May 2014

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Certification
  2. assurance
  3. hazard analysis
  4. requirements
  5. safety
  6. standards
  7. validation
  8. verification

Qualifiers

  • Article

Conference

ICSE '14
Sponsor:

Upcoming Conference

ICSE 2025

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)40
  • Downloads (Last 6 weeks)4
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)From Textual to Formal Requirements: A Case Study Using Spectra in Safety-Critical Systems DomainJournal of Software Engineering Research and Development10.5753/jserd.2024.374512:1Online publication date: 4-Nov-2024
  • (2024)The Isolette System: Illustrating End-to-End Artifacts for Rigorous Model-Based EngineeringThe Combined Power of Research, Education, and Dissemination10.1007/978-3-031-73887-6_9(93-117)Online publication date: 23-Oct-2024
  • (2023)Redesigning Medical Device Assurance: Separating Technological and Clinical Assurance CasesComputer Safety, Reliability, and Security10.1007/978-3-031-40923-3_3(25-38)Online publication date: 11-Sep-2023
  • (2022)Awas: AADL information flow and error propagation analysis frameworkInnovations in Systems and Software Engineering10.1007/s11334-021-00410-w18:4(485-504)Online publication date: 1-Dec-2022
  • (2022)How assurance case development and requirements engineering interplay: a study with practitionersRequirements Engineering10.1007/s00766-022-00375-727:2(273-292)Online publication date: 1-Jun-2022
  • (2021)Automatic Code Generation of Safety Mechanisms in Model-Driven DevelopmentElectronics10.3390/electronics1024315010:24(3150)Online publication date: 17-Dec-2021
  • (2021)Engineering Education for Development of Safety-Critical SystemsIEEE Transactions on Education10.1109/TE.2021.306244864:4(398-405)Online publication date: 1-Nov-2021
  • (2021)Reliability and Safety Engineering for Safety-Critical Systems in Computer Science: A Study Into the Mismatch Between Higher Education and Employment in Brazil and IndiaIEEE Transactions on Education10.1109/TE.2021.305761164:4(353-360)Online publication date: 1-Nov-2021
  • (2021)Functional Hazard Analysis for Engineering Safe Software Requirements2021 4th International Conference on Information and Computer Technologies (ICICT)10.1109/ICICT52872.2021.00031(142-148)Online publication date: Mar-2021
  • (2021)Generation of hazard relation diagrams: formalization and tool supportSoftware and Systems Modeling (SoSyM)10.1007/s10270-020-00799-120:1(175-210)Online publication date: 1-Feb-2021
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media