Abstract
In this paper we identify special quality assurance and test requirements of software for safety systems and show that even the best currently available practices meet these requirements only at very high cost and by application of empirical rather than technically rigorous criteria. Redundancy can help but is expensive and the reduction in failure probability due to software redundancy is uncertain. We identify a qualitative approach to test data interpretation, particularly the examination of rare events in the conditions that precipitated an observed failure, as a possible avenue for a more economical reliability assessment. This can be used as an alternative or as a supplement to redundancy for meeting the reliability requirements for software in safety systems. Further research in this area is recommended.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Code of Federal Regulations (1986), General Design Criteria for Nuclear Power Plants, Volume 10, Part 50, Appendix A, Criterion 21 — Protection System Reliability and Testability, National Archives and Records Administration, Office of the Federal Register, Washington, DC.
Department of Defense (1987), Reliability Test Methods, Plans, and Environments for Engineering Development, Qualification and Production, MIL-HDBK-781, Department of Defense, Washington, DC.
Eckhardt, D.E., A.K. Caglayan, J.C. Knight, L.D. Lee, D.F. McAllister, M.A. Vouk, and J.P.J. Kelly (1991), “An Experimental Evaluation of Software Redundancy as a Strategy for Improving Reliability,” IEEE Transactions on Software Engineering 17,7, 692–702.
Eckhardt, D.E. and L.D. Lee (1985), “An Analysis of the Effects of Coincident Errors on Multi-Version Software,” In Proceedings of the 5th Conference Computers in Aerospace, AiAA, New York, NY, pp. 370–373.
FAA (1988), Advisory Circular 25.1309-1A, US Department of Transportation, Washington, DC.
Hamlet, R. and J. Voas (1993), “Faults on Its Sleeve: Amplifying Software Reliability Testing,” In Proceedings of the ACM SIGSOFT International Symposium on Software Testing and Analysis, ACM Press, New York, NY, pp. 59–98.
Hecht, H. (1993), “Rare Conditions — An Important Cause of Failures,” In Proceedings of COMPASS'93, IEEE, Piscataway, NJ, pp. 81–85.
Hecht, H. and P. Crane (1994), “Rare Conditions and their Effect on Software Failures,” In Proceedings of the 1994 Reliability and Maintainability Symposium, IEEE, Piscataway, NJ, pp. 334–337.
Hecht, H., M. Hecht, G. Dinsmore, S. Hecht, and D. Tang (1995), “Verification and Validation Guidelines for High Integrity Systems,” Technical Report NUREG/CR-6293, US Nuclear Regulatory Commission, Washington, D.C.
Howden, W.E. (1978), “An Evaluation of the Effectiveness of Symbolic Testing,” Software Practice and Experience 8, 381–397.
Leveson, N.G. (1995), Safeware, Addison-Wesley, Reading, MA.
Musa, J., A. Iannino, and K. Okumoto (1987), Software Reliability Measurement Prediction, Application, McGraw-Hill, New York, NY.
Seth, S., W. Bail, D. Cleaves, H. Cohen, D. Hybertson, C. Shaefer, G. Stark, A. Ta, and B. Ulery (1995), High Integrity Software for Nuclear Power Plants, NUREG/CR-6263, US Nuclear Regulatory Commission, Washington, DC.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Hecht, H., Hecht, M. Quality assurance and testing for safety systems. Annals of Software Engineering 4, 191–200 (1997). https://doi.org/10.1023/A:1018927113776
Issue Date:
DOI: https://doi.org/10.1023/A:1018927113776