Probability-informed testing for reliability assurance through Bayesian hypothesis methods

doi:10.1016/j.ress.2009.11.006

Reliability Engineering & System Safety

Volume 95, Issue 4, April 2010, Pages 361-368

https://doi.org/10.1016/j.ress.2009.11.006 Get rights and content

Abstract

Bayesian inference techniques play a central role in modern risk and reliability evaluations of complex engineering systems. These techniques allow the system performance data and any relevant associated information to be used collectively to calculate the probabilities of various types of hypotheses that are formulated as part of reliability assurance activities. This paper proposes a methodology based on Bayesian hypothesis testing to determine the number of tests that would be required to demonstrate that a system-level reliability target is met with a specified probability level. Recognizing that full-scale testing of a complex system is often not practical, testing schemes are developed at the subsystem level to achieve the overall system reliability target. The approach uses network modeling techniques to transform the topology of the system into logic structures consisting of series and parallel subsystems. The paper addresses the consideration of cost in devising subsystem level test schemes. The developed techniques are demonstrated using several examples. All analyses are carried out using the Bayesian analysis tool WinBUGS, which uses Markov chain Monte Carlo simulation methods to carry out inference over the network.

Introduction

As systems are designed and constructed for application, frequently the question of testing to demonstrate a desired level of reliability arises. For example, in the nuclear power generation industry, regulatory standards mandate that “risk important” components be demonstrably able to achieve their claimed reliability values [1], [2]. For space applications, the use of human-crewed space vehicles heightens the desire to achieve an acceptable level of reliability to provide assurance that fatality risks are low. While this desire to have acceptable reliability levels is pervasive, formal methods to demonstrate compliance with target reliability levels are not commonly known or employed. It is the focus of this paper to provide such methods by providing a theoretical discussion of the Bayesian approach and applying that method to representative problems.

We will begin the discussion by simply posing the following question:

How many trials are needed to show that the unreliability of a device (i.e., the probability it fails on demand) is 0.001 with 50% probability?

The short answer to this question is that we use Bayesian methods [3] via Bayes’ Theorem, to determine how many trials (or tests to use test-engineering terminology) are required to ensure – with probability of at least 50% – that the device unreliability is no larger than 0.001. To see how this approach is useful, we first need to review the fundamentals of the Bayesian approach in Section 2. In Section 3, we address the problem of determining the number of tests of a single device required for reliability assurance in the case of no prior information. Then, in Section 4, we cover the same problem but for the case where prior information is available. In Section 5 we address the analysis methods required when determining Bayesian reliability assurance tests for complex systems. Lastly, we provide conclusions in Section 6.

Section snippets

Review of Bayesian concepts

We will begin the presentation of the Bayesian¹ method of induction, or simply the “Bayes” method, by noting that logic theory uses a variety of deductive and inductive methods. Logic theory relies on two plausibility statements, the so-called “strong” and “weak” statements:

Strong:	If A, then B	B is false	thus, A is false
Weak:	If A, then B	B is true	thus, A is

Bayesian hypothesis testing with little prior information

First, let us return to our original question, but we will take a naïve approach by interpreting the problem, “the unreliability of a device (on demand) is 0.001,” as implying that there is no uncertainty in the probability of failure on demand, p. We are looking for the necessary number of trials (denoted by n) to achieve a “comfort” level on the device reliability. The number of failures is assumed to be x=0 (if the device is as reliable as we claim, we would not expect to see any failures in

Bayesian hypothesis testing with prior information

While the Bayesian approach to hypothesis testing is viable when there is no prior information, in many real-world situations, much prior information is available for the device in question. Under these conditions, the general approach to determine testing requirements does not change. However, a difficult part of the analysis is frequently the process of translating prior information into the probabilistic format necessary to apply Bayes’ Theorem.

Recall that information is, in the Bayes

Estimating test requirements for complex systems

In this section, we will examine arrangements into a system of the three components from Section 4, where Component 1 is described by Database I, Component 2 is described by Database II, and Component 3 is described by Database III. A goal for failure probability is now set at the system level, and the number of tests required to demonstrate this goal with a specified probability is determined. In other words, we will allocate a number of trials that will provide desired overall system

Conclusions

Frequentist estimates have been commonly used to suggest testing methods—however, the frequentist approach to inference does not allow useful past information to be incorporated. In contrast, the Bayesian approach that we have described in this paper allows the analyst to incorporate additional sources of information and provides probabilities of observable events (tests), and these estimates take into account the epistemic uncertainty inherent in the problem.

For simple cases such as

Acknowledgements

This paper has been authorized by Battelle Energy Alliance, LLC (BEA) under Contract no. DE-AC07-05ID14517 with the US Department of Energy. The Government and BEA make no express or implied warranty as to the conditions of the research or any intellectual property, generated information, or product made or developed under this technical assistance project, or the ownership, merchantability, or fitness for a particular purpose of the research or resulting product; that the goods, services,

References (16)

N. Siu et al.
Bayesian parameter estimation in probabilistic risk assessment
Reliability Engineering and System Safety
(1998)
D.L. Kelly et al.
Bayesian inference in probabilistic risk assessment—the current state of the art
Reliability Engineering and System Safety
(2009)
Dube DA, Atwood CL, Eide SA, Mrowca BB, Youngblood RW, Zeek DP. Independent verification of the Mitigating Systems...
NRC Inspection Manual. Inspection Procedure 62706, Maintenance Rule,...
T. Bayes
An essay towards solving a problem in the doctrine of chances
Philosophical Transactions of the Royal Society
(1763)
E. Jaynes
Probability theory—the logic of science
(2003)
Barlow RE. Engineering reliability. American Statistical Association and the Society for Industrial and Applied...
NASA/SP-2009-569. Bayesian inference for NASA probabilistic risk and reliability analysis. NASA Scientific and...

There are more references available in the full text version of this article.

Cited by (6)

Bayesian post-processing of Monte Carlo simulation in reliability analysis
2022, Reliability Engineering and System Safety
Citation Excerpt :
For the conceptually similar problem of reliability estimation based on data collected from sampling tests, [18] discusses how to consider available prior information. Another relevant discussion of how to account for prior information in the more general context of Bayesian hypothesis testing can be found in [17]. Bayesian post-processing of the outcome of a MCS is straightforward and inexpensive.
In reliability analysis with Monte Carlo simulation, the uncertainty about the probability of failure can be formally quantified through Bayesian statistics. Credible intervals for the probability of failure can be derived analytically. This paper gives a detailed overview of Bayesian post-processing for Monte Carlo simulation. We investigate the influence of different weakly-informative prior assumptions on the resulting credible intervals. On this basis, we recommend to use a prior distribution on the probability of failure that follows from the principle of maximum information entropy. We also show that even if no failure sample occurs in a Monte Carlo simulation, Bayesian post-processing still allows to deduce useful information about the probability of failure. The presented Bayesian post-processing strategy can also be applied if Monte Carlo simulation is used for reliability updating; i.e., to evaluate the probability of failure conditional on data or observations. We derive expectations for credible intervals for this case.
Experimental estimation of time variant system reliability of vibrating structures based on subset simulation with Markov chain splitting
2018, Reliability Engineering and System Safety
Citation Excerpt :
On the other hand, the problem of time variant reliability estimation, through dynamic testing, has received relatively much less attention. In the broader context of reliability testing of engineering systems, the idea of accelerated testing and related issues has been widely investigated (see, for example, [41–46]). These studies focus on long-range degradation of systems due to reasons, such as, corrosion and fatigue, and, also consider reliability models for instrumented systems via Bayesian model updating procedures.
The study investigates the application of ideas from variance reduction schemes, developed in the area of computational structural reliability modelling, to the problems of experimental estimation of time variant reliability of randomly excited vibrating structures. The study considers series/parallel system reliability of vibrating systems under multi-component random excitations. An experimental protocol, based on subset simulation with Markov chain splitting, is proposed to estimate probabilities of failure as low as $10^{- 5}$ to $10^{- 4}$ with a relatively smaller number of samples and hence with reduced test times. Illustrative examples consist of earthquake shake table studies on a three-storied bending-torsion coupled building frame under bi-axial nonstationary, random earthquake support motions.
Optimal number of tests to achieve and validate product reliability
2014, Reliability Engineering and System Safety
Citation Excerpt :
The problem of reliability demonstration has gained large interest in recent years, due to the increasing industrial demands, under physical, functional and time considerations [2]. Most of the works in the literature are based on the Bayesian techniques, either by using the failure free time period testing [1], or by using failure data within a Bayesian procedure in order to carry out inference analysis on product reliability [3,4]. In these approaches, the reliability and the confidence intervals have been previously stipulated by the customer.
The reliability validation of engineering products and systems is mandatory for choosing the best cost-effective design among a series of alternatives. Decisions at early design stages have a large effect on the overall life cycle performance and cost of products. In this paper, an optimization-based formulation is proposed by coupling the costs of product design and validation testing, in order to ensure the product reliability with the minimum number of tests. This formulation addresses the question about the number of tests to be specified through reliability demonstration necessary to validate the product under appropriate confidence level. The proposed formulation takes into account the product cost, the failure cost and the testing cost. The optimization problem can be considered as a decision making system according to the hierarchy of structural reliability measures. The numerical examples show the interest of coupling design and testing parameters.
Augmenting system tests with component tests for reliability assurance
2024, Journal of Quality Technology
An emotional learning-based fuzzy inference system for improvement of system reliability evaluation in redundancy allocation problem
2013, International Journal of Advanced Manufacturing Technology
Probability model of the compressive strength of bricks based on the bayesian inference
2013, Advanced Materials Research

View full text