John Hatcliff
ABSTRACT
The amount and impact of software-dependence in critical systems impinging on daily life is increasing rapidly. In many of these systems, inadequate software and systems engineering can lead to economic disaster, injuries or death. Society generally does not recognize the potential of losses from deficiencies of systems due to software until after some mishap occurs. Then there is an outcry, reflecting societal expectations; however, few know what it takes to achieve the expected safety and, in general, loss-prevention.
On the one hand there are unprecedented, exponential increases in size, inter-dependencies, intricacies, numbers and variety in the systems and distribution of development processes across organizations and cultures. On the other hand, industry's capability to verify and validate these systems has not kept up. Mere compliance with existing standards, techniques, and regulations cannot guarantee the safety properties of these systems. The gap between practice and capability is increasing rapidly.
This paper considers the future of software engineering as needed to support development and certification of safety-critical software-dependent systems. We identify a collection of challenges and document their current state, the desired state, gaps and barriers to reaching the desired state, and potential directions in software engineering research and education that could address the gaps and barriers.
- Ministry of defence defence standard 23-09 – generic vehicle architecture, aug 2010.Google Scholar
- ACM and IEEE Computer Society. Software engineering 2004, curriculum guidelines for undergraduate degree programs in software engineering. http://sites. computer.org/ccse/SE2004Volume.pdf, 2004.Google Scholar
- Associated Press. Okla. jury: Toyota liable in sudden acceleration crash. http: //www.cbsnews.com/news/okla-jury-toyotaliable-in-sudden-acceleration-crash/, 2013.Google Scholar
- AUTomotive Open System ARchitecture. AUTOSAR. https://www.autosar.org, 2014.Google Scholar
- J. Barnes. High Integrity Software—the SPARK Approach to Safety and Security. Addison-Wesley, 2003. Google ScholarDigital Library
- C. Boettcher, R. DeLong, J. Rushby, and W. Sifre. The MILS component integration approach to secure information sharing. In Proceedings of the 27th IEEE/AIAA Digital Avionics Systems Conference (DASC), Oct. 2008.Google ScholarCross Ref
- Boston Scientific. PACEMAKER system requirements specification. http://sqrl.mcmaster.ca/pacemaker.htm, 2007.Google Scholar
- Carnegie Mellon. Entertainment Technology Center. http://www.etc.cmu.edu/site/, 2014.Google Scholar
- R. N. Charette. This Car Runs on Code. IEEE Spectrum, Feb. 2009.Google Scholar
- R. DeLong and J. Rushby. A common criteria authoring environment supporting composition. In Proceedings of the 8th International Common Criteria Conference, 2007.Google Scholar
- B. Dion. FACE, ARINC, DO-178C avionics standards help U.S. DoD’s vision of reusable technology to take off. Military Embedded Systems, 9(2):36–39, mar 2013.Google Scholar
- M. B. Dwyer and S. G. Elbaum. Unifying verification and validation techniques: relating behavior and properties through partial evidence. In Proceedings of the Workshop on Future of Software Engineering Research (FoSER 2010), pages 93–98, 2010. Google ScholarDigital Library
- C. A. Ericson. Hazard Analysis Techniques for System Safety. Wiley-Interscience, 2005.Google ScholarCross Ref
- European Committee for Electrical Standardization (CENELEC). Railway applications - Communication, signalling and processing systems - Software for railway control and protection systems. CENELEC Standard 50128, 2011.Google Scholar
- P. Feiler. Architecture Analysis and Design Language (AADL) Annex Volume 3: Annex E: Error Model V2 Annex. Number SAE AS5506/3 (Draft) in SAE Aerospace Standard. SAE International, 2013.Google Scholar
- P. Feiler and D. Gluch. Model-Based Engineering with AADL: An Introduction to the SAE Architecture Analysis and Design Language. Addison-Wesley, 2012. Google ScholarDigital Library
- E. Feliz. Engineering pipeline under pressure. Avionics Today, December 2007.Google Scholar
- N. Gershenfeld, R. Krikorian, and D. Cohen. The internet of things. Scientific American, Oct. 2004.Google ScholarCross Ref
- S. Grigorova and T. Maibaum. Taking a page from the law books: Considering evidence weight in evaluating assurance case confidence. In ISSRE (Supplemental Proceedings), pages 387–390, 2013.Google ScholarCross Ref
- C. Hagen, S. Hurt, J. Sorenson, and D. Wall. Software: The brains behind us defense systems. A.T. Kearney Whitepaper, Nov 2012.Google Scholar
- D. Harel and R. Marelly. Come, Let’s Play: Scenario-Based Programming Using LSCs and the Play-Engine. Springer, 2003. Google ScholarDigital Library
- J. Hatcliff, M. Heimdahl, M. Lawford, T. Maibaum, A. Wassyng, and F. Wurden. A Software Certification Consortium and its Top 9 Hurdles. Electronic Notes in Theoretical Computer Science, 238(4):11–17, Sept. 2009. Google ScholarDigital Library
- J. Hatcliff, A. King, I. Lee, A. Fernandez, J. Goldman, A. McDonald, M. Robkin, E. Vasserman, and S. Weininger. Rationale and architecture principles for medical application platforms. In Proceedings of the 2012 International Conference on Cyberphysical Systems, 2012. Google ScholarDigital Library
- J. Hatcliff, A. Wassyng, T. Kelly, and C. Comar. Certifiably safe software-dependent systems: Challenges and directions (paper web site / extended version). http://santoslab.org/pub/papers/fose14certification/.Google Scholar
- R. Hawkins, K. Clegg, R. Alexander, and T. Kelly. Using a software safety argument pattern catalogue: two case studies. In Computer Safety, Reliability, and Security, pages 185–198. Springer, 2011. Google ScholarDigital Library
- R. Hawkins, I. Habli, T. Kelly, et al. Principled construction of software safety cases. In Proceedings of Workshop SASSUR (Next Generation of System Assurance Approaches for Safety-Critical Systems) of the 32nd International Conference on Computer Safety, Reliability and Security, 2013.Google Scholar
- M. P. Heimdahl. Safety and software intensive systems: Challenges old and new. In 2007 Future of Software Engineering, pages 137–152. IEEE Computer Society, 2007. Google ScholarDigital Library
- J. Hillebrand, P. Reichenpfader, I. Mandic, H. Siegl, and C. Peer. Establishing confidence in the usage of software tools in context of iso 26262. In Computer Safety, Reliability, and Security - 30th International Conference (SAFECOMP 2011), pages 257–269, 2011. Google ScholarDigital Library
- C. M. Holloway. Making the implicit explicit: Towards an assurance case for do-178c. In Proceedings of the 31st International System Safety Conference (ISSC), 2013.Google Scholar
- IEEE Computer Society. Software engineering body of knowledge. http://www.computer.org/portal/ web/swebok/html/contents, 2004.Google Scholar
- Institute of Electrical and Electronics Engineers (IEEE). IEEE Standard Criteria for Digital Computers in Safety Systems of Nuclear Power Generating Stations. IEEE Standard 7-4.3.2, 2010.Google Scholar
- International Electrotechnical Commission. Medical device software – Software life cycle processes. IEC Standard 62304 edition 1.0, 2006.Google Scholar
- International Electrotechnical Commission. Functional Safety of Electrical/Electronic/Programmable Electronic Safety-related Systems. IEC Standard 61508 edition 2.0, 2010.Google Scholar
- International Organization for Standardization. Software Engineering – Software product Quality Requirements and Evaluation (SQuaRE) – Guide to SQuaRE. ISO Standard 25000, 2005.Google Scholar
- International Organization for Standardization. Application of Risk Management to Medical Devices. ISO Standard 14971, 2007.Google Scholar
- International Organization for Standardization. Road Vehicles – Functional Safety. ISO Standard 26262, 2011.Google Scholar
- International Organization for Standardization. Systems and software engineering – Systems and software assurance – Part 2: Assurance case. ISO Standard 15026 part 2, 2011.Google Scholar
- ISO, IEC, IEEE. IEEE Standard 24765: Systems and software engineering - Vocabulary, 2010.Google Scholar
- ISO/IEC. ISO/IEC 17000: Conformity assessment – vocabulary and general principles, 2004.Google Scholar
- ISO/IEEE. 11073-x Medical Health Device Communication Standards family.Google Scholar
- D. Jackson. Software Abstractions: Logic, Language, and Analysis. MIT Press, 2012. Google ScholarDigital Library
- T. Kelly. Using software architecture techniques to support the modular certification of safety-critical systems. In Proceedings of the Eleventh Australian Workshop on Safety Critical Systems and Software - Volume 69, SCS ’06, pages 53–65, 2006. Google ScholarDigital Library
- J. C. Knight. Focusing software education on engineering. ACM SIGSOFT Software Engineering Notes, 30(2):3–5, 2005. Google ScholarDigital Library
- J. C. Knight and N. G. Leveson. Software and higher education. Communications of the ACM, 49(1):160–160, 2006. Google ScholarDigital Library
- B. Larson and J. Hatcliff. Open PCA Pump project website. http://openpcapump.santoslab.org.Google Scholar
- X. Leroy. A formally verified compiler back-end. Journal of Automated Reasoning, 43(4):363–446, Dec. 2009. Google ScholarDigital Library
- N. Leveson. Safeware: System Safety and Computers. Addison-Wesley, 1995. Google ScholarCross Ref
- N. Leveson. Engineering a Safer World: Systems Thinking Applied to Safety. MIT Press, 2012.Google ScholarCross Ref
- R. R. Lutz. Software engineering for safety: a roadmap. In Proceedings of the Conference on The Future of Software Engineering, pages 213–226. ACM, 2000. Google ScholarDigital Library
- T. Maibaum and A. Wassyng. A product-focused approach to software certification. Computer, 41(2):91–93, 2008. Google ScholarDigital Library
- Mathew J. Schwartz. Hacked Medical Device Sparks Congressional Inquiry. http://www.informationweek.com/security/ vulnerabilities-and-threats/hackedmedical-device-sparks-congressionalinquiry/d/d-id/1099726, 2011.Google Scholar
- E. McKenna. Embedded overall. Avionics Today, Nov 2008.Google Scholar
- Naval Postgraduate School. Department of Systems Engineering: Programs of Study. http://www.nps. edu/Academics/Schools/GSEAS/Departments/ SE/Academics/ProgramsofStudy.html, 2014.Google Scholar
- Open platform for evolutionary certification of safety-critical systems. www.opencoss-project.eu.Google Scholar
- D. L. Parnas. Software engineering programs are not computer science programs. Software, IEEE, 16(6):19–30, 1999. Google ScholarDigital Library
- D. L. Parnas, A. J. van Schouwen, and S. P. Kwan. Evaluation of safety-critical software. Communications of the ACM, 33(6):636–648, 1990. Google ScholarDigital Library
- Patrick J. Graydon, C. Michael Holloway. AESSCS 2014 Workshop. Planning the Unplanned Experiment: Assessing the Efficacy of Standards for Safety Critical Software. http://www.idt.mdh.se/AESSCS\_2014/, 2013.Google Scholar
- K. Pohl, G. Böckle, and F. van der Linden. Software Product Line Engineering: Foundations, Principles and Techniques. Springer, 2005. Google ScholarCross Ref
- A. Rae, M. Nicholson, and R. Alexander. The state of practice in system safety research evaluation. In 5th IET International Conference on System Safety 2010, pages 1–8, Oct 2010.Google ScholarCross Ref
- RTCA. Software Considerations in Airborne Systems and Equipment Certification. RTCA Standard DO-178C, 2012.Google Scholar
- J. Rushby. Modular certification. Technical report, Computer Science Laboratory, SRI International, Menlo Park, CA 94025, US, September 2001.Google Scholar
- J. Rushby. Composing safe systems. In Proceedings of Formal Aspects of Component Software (FACS 2011), volume 7253 of Lecture Notes in Computer Science, pages 3–11, 2012.Google ScholarCross Ref
- Russell Sydnor, Sushil Birla, Michael Waterman. Gap Assessment of IEC and IEEE Standards for Safety Assurance of Digital Systems. Software Certification Consortium Meeting 9: http://cps-vo.org/node/3472, 2012.Google Scholar
- K. Sandler, L. Ohrstrom, L. Moy, and R. McVay. Killed by Code: Software transparency in implantable medical devices. Software Freedom Law Center Whitepaper, 2010.Google Scholar
- G. Schkade. SAE’s Perspective on Connected Vehicle. http://www.itu.int/dms\_pub/itut/oth/06/5B/T065B0000020008PDFE.pdf, 2011.Google Scholar
- O. Slotosch. Model-based tool qualification. http://wiki.eclipse.org/Auto\_IWG\_WP5.Google Scholar
- Society of Automotive Engineers (SAE). Arp4761: Guidelines and methods for conducting the safety assessment process on civil airborne systems and equipment, 1996.Google Scholar
- J. Stern. Google Self-Driving Car License Approved in Nevada. http://abcnews.go.com/blogs/ technology/2012/05/google-self-drivingcar-license-approved-in-nevada/, 2012.Google Scholar
- United Kingdom Ministry of Defence. Safety Management Requirements for Defence Systems. Defence Standard 00-56. Issue 4., 2007.Google Scholar
- University of California, Los Angeles. UCLA Extension: Safety-Critical Software. https://www.uclaextension.edu/pages/ Course.aspx?reg=V9149, 2014.Google Scholar
- U.S. Department of Defense. Mil-std-882e: Department of defense standard practice: System safety, 2012.Google Scholar
- U.S. Food and Drug Administration. Safety, Recalls, Market Withdrawals, & Safety Alerts, Background and Definitions. http://www.fda.gov/Safety/Recalls/ ucm165546.htm, 2009.Google Scholar
- U.S. Nuclear Regulatory Commission. Research information letter (ril) 1101: Technical basis to review hazard analysis of digital safety systems.Google Scholar
- U.S. Nuclear Regulatory Commission. Research Information Letter 1001: Software-related uncertainties in the assurance of digital safety systems – expert clinic findings Part 1. http://adamswebsearch2.nrc.gov/IDMWS/ ViewDocByAccession.asp? \\AccessionNumber=ML111240017, 2011.Google Scholar
- A. van Lamsweerde. Requirements engineering: From craft to discipline. In FSE’2008: 16th ACM Sigsoft Intl. Symposium on the Foundations of Software Engineering (Invited Paper for the ACM Sigsoft Outstanding Research Award), pages 238–249, 2008. Google ScholarDigital Library
- W. G. Vincenti. What engineers know and how they know it: Analytical studies from aeronautical history. The Johns Hopkins University Press, 1990.Google Scholar
- A. Wassyng, M. Lawford, and T. Maibaum. Separating safety & control systems to reduce complexity. In M. Hinchey and L. Doyle, editors, Conquering Complexity, pages 89––108. Springer, 2012.Google Scholar
- A. Wassyng and M. Lawford. Lessons learned from a successful implementation of formal methods in an industrial project. In K. Arakai, S. Gnesi, and D. Mandrioli, editor, FME 2003: International Symposium of Formal Methods Europe Proceedings, volume 2805 of Lecture Notes in Computer Science, pages 133–153. Springer-Verlag, 2003.Google Scholar
- A. Wassyng, T. Maibaum, M. Lawford, and H. Bherer. Software certification: Is there a case against safety cases? In R. Calinescu and E. Jackson, editors, Foundations of Computer Software. Modeling, Development, and Verification of Adaptive Systems, volume 6662 of Lecture Notes in Computer Science, pages 206–227. Springer Berlin Heidelberg, 2011. Google ScholarDigital Library
- C. B. Weinstock, J. B. Goodenough, and A. Z. Klein. Measuring assurance case confidence using baconian probabilities. In Proceedings of the 1st International Workshop on Assurance Cases for Software-Intensive Systems (ASSURE), 2013.Google ScholarDigital Library
- M. Wildmoser, J. Philipps, and O. Slotosch. Determining potential errors in tool chains - strategies to reach tool confidence according to iso 26262. In Computer Safety, Reliability, and Security - 31st International Conference (SAFECOMP 2012), pages 317–327, 2012. Google ScholarDigital Library
- The Muen Separation Kernel. http://muen.codelabs.ch/.Google Scholar
Index Terms
- Certifiably safe software-dependent systems: challenges and directions
Index terms have been assigned to the content through auto-classification.
Recommendations
A UML profile for developing airworthiness-compliant (RTCA DO-178B), safety-critical software
MODELS'07: Proceedings of the 10th international conference on Model Driven Engineering Languages and SystemsMany safety-related, certification standards exist for developing safety-critical systems. System safety assessments are common practice and system certification according to a standard requires submitting relevant software safety information to ...
Addressing the 4+1 Software Safety Assurance Principles within Scrum
XP '16 Workshops: Proceedings of the Scientific Workshop Proceedings of XP2016The 4+1 principles document common principles of software safety assurance that can be observed from software safety standards and best practice. These principles are constant across domains and across projects, and can be regarded as the immutable core ...
Safety of embedded software
HILT '12: Proceedings of the 2012 ACM conference on High integrity language technologyTraditional safety techniques were created 40-50 years ago for electro-mechanical systems. The underlying assumptions of these techniques about the cause of accidents (e.g., component failure) do not match software nor do they match the types of ...
Comments