An artificial neural network supported Wiener process based reliability estimation method considering individual difference and measurement error
Introduction
Nowadays, with the development of manufacturing technique, modern products always have long lifetime and high reliability [1,2]. For some safety-critical components, the high reliability during the whole lifetime is demanded to guarantee the safety of the system [3]. Hence, before the components are put into the system the reliability tests are usually performed [4]. Using the reliability tests, two kinds of data can be obtained, including degradation data and lifetime data [5]. In engineering practice, due to the long lifetime of modern products, degradation data is more easily to be collected than lifetime data by reliability testing. Hence, degradation modeling based methods attract lots of attentions in reliability estimation [6].
In degradation modeling based reliability estimation, stochastic process is one of the most practical approaches [7]. There are three kinds of widely used stochastic processes. Wiener process can be used to model both monotonic and non-monotonic degradation processes, while Gamma process and inverse Gaussian process are only suitable for modeling monotonic degradation processes [8], [9], [10], [11]. Three factors need to be determined when applying stochastic process, including degradation process, mean function and model parameters [12].
In most cases, there may be several suitable processes for a specific degradation dataset [13]. Several methods have been presented to select and combine the candidate processes. Several criterions have been proposed to assess the candidate processes and select the best process, such as Akaikes information criterion (AIC), Bayesian information criterion (BIC), etc. [14], [15], [16]. Furthermore, several model combining methods have also been presented to combine the candidate processes, such as adjustment factor method, Bayesian model averaging method (BMA) and evidence theory based method [17], [18], [19].
Bayesian method has been applied to estimate the model parameters, due to its powerful ability in data fusion and parameters estimation [20], [21], [22], [23]. For example, Ref. [23] has presented several inverse Gaussian process models and the related Bayesian analysis methods to model the degradation process and infer the degradation of the product. Besides the Bayesian approaches, some other methods have also been proposed to adapt stochastic process to small sample conditions, such as random fuzzy theory based methods, evidence theory based methods, etc. [24,25].
For a specific degradation dataset, there may be a number of available mean functions, such as multi-phase based mean functions, mean functions with time varying degradation rates, etc. [26], [27], [28], [29]. When using stochastic process, the mean function is normally determined by model selecting method or combination method among the existing candidates. However, the existing mean functions should not be suitable for all applications. For a specific application, it may be not suitable to determine the mean function by selecting or combining within the existing candidates. Hence, the mean functions are determined by degradation mechanism analysis for some specific applications. Ma et al. have presented an engineering‐driven performance degradation analysis method [30]. The degradation of a kind of piston pumps is modeled by inverse Gaussian process. The mean function is given by analyzing the degradation mechanism of the pump. Similar methods can also be found in Ref. [31]. The researchers argue that the theoretical mean functions predicted by analyzing the degradation mechanism are not the existing candidates and the theoretical mean functions show superiority in engineering practice. However, it is obvious that the mechanism analysis based methods need large amount of knowledge about the application objects.
Due to the flexibility on relationship describing and fitting, artificial neural network (ANN) has been widely used and shows superiority in some practical applications. For instance, Hong et al. have employed ANN in oil debris detection and proposed a hybrid detection strategy to reduce the detecting error [32]. An experimental study is carried out to verify the proposed method. It is concluded that the detection accuracy is improved due to utilizing the advantage of ANN. Furthermore, ANN has also been introduced to describe the relationship between the wear rate and its affecting factors in wear failure analysis [33,34]. It is concluded that the ANN supported models can predict the wear failure process precisely. Ma et al. have constructed a degradation modeling approach for proton exchange membrane fuel cell based on deep learning method [35]. The recurrent neural network with grid long short-term memory cell is utilized in the degradation prognostic model. The superiorities of the proposed method are verified based on an actual degradation dataset from eight different fuel cells with eight different load profiles. The above mentioned researches have validated the strong ability of ANN on relationship description and data fitting. Hence, in order to describe the mean function, we have combined ANN with Wiener process for estimating reliability of the products [36]. ANN is applied to fit the mean function in Wiener process. Simulation studies have been performed to verify the advantages of the presented method. In addition, an actual case is also used to demonstrate the presented reliability estimation method. It is concluded that ANN is capable in mean function fitting and the presented method is capable in reliability estimation. However, the measurement error and individual difference are neglected in the above mentioned research.
In fact, the measurement error is an inevitable phenomenon and need to be considered. Ma et al. have proposed an inverse Gaussian process model with measurement error and the corresponding parameters estimation method based on expectation maximization algorithm [37]. The presented model and the associated method are verified by comparing to the method neglecting measurement error based on a simulation study and an actual case study. The authors argue that considering the measurement error is necessary in stochastic process based reliability estimation. Ye et al. have proposed several Wiener process models with measurement errors and the corresponding statistical inference methods to capture the effects of the measurement errors on the degradation modeling and reliability estimation [38]. The necessity of considering the measurement errors is verified based on a simulation study and two illustrative examples. In Ref. [39], the imperfect inspection issue in dynamic reliability assessment is focused and handled, especially the measurement error. The dynamic reliability assessment for nonrepairable multistate systems is discussed. A set of two-stage recursive Bayesian formulations are constructed to utilize the imperfect inspection data. The effectiveness of the proposed method is verify based on a case study. Li et al. have presented a Wiener process model considering the measurement errors to analyze accelerated degradation data [40]. The measurement error term is introduced in Wiener process model and described by a normal distribution with zero mean. The maximum likelihood estimations of the model parameters are deduced. The superiority of the proposed method is illustrated by comparing to the method neglecting the measurement errors based on an actual degradation dataset regarding LED. Besides, Pan et al. have adopted expectation maximization algorithm to the Wiener process degradation model with measurement errors [41]. It is argued that, the effectiveness of the Wiener process based method has been improved due to considering the measurement error. In Ref. [42], inverse Gaussian process model has been extended to consider the measurement errors by introducing measurement error item, which is assumed to follow a normal distribution with zero mean. The model parameters are estimated based on Monte Carlo algorithm. A simulation study and two actual case studies are performed to verify the effectiveness of the proposed method. In brief, the measurement error is an inevitable issue and need to be considered in degradation modeling and reliability estimation.
Besides the measurement error, the individual difference also need to be considered. In engineering practice, hyper parameters are always introduced to handle the random effects caused by individual difference. In Ref. [43], several Gaussian processes with random effects have been introduced. The unit-to-unit variability is incorporated into the model by applying the natural conjugate distribution. The presented models are demonstrated based on a Monte Carlo simulation study and several case studies. Based on the simulation results, the author argues that the presented models provide the applicability and effectiveness in degradation modeling, due to considering the individual difference. Wang has presented a class of Wiener processes with random effects to analyze degradation data [44]. The model parameters can vary from unit to unit by introducing the random drift and diffusion parameters. Expectation maximization algorithm is applied to estimate the unknown parameters. Based on the simulation study, it is concluded that introducing random effects in Wiener processes is necessary. In addition, a series of gamma process models with the random effects have also been presented in Ref. [45]. The drift and diffusion parameters are assumed to be randomly distributed to describe the random effects caused by the individual difference. The model parameters are estimated by Bayesian inference. The necessity of considering the individual differences is proved by several case studies. Based on the above discussions, it can be seen that considering the individual difference is necessary.
As discussed above, in degradation modeling and reliability estimation, both the measurement error and individual difference should to be considered. Hence, the above two issues are focused in the presented research. Furthermore, because of the modeling ability on both non-monotonic and monotonic degradation processes, Wiener process is presented as an example in this paper. Hence, the ANN supported Wiener process based reliability estimation method is improved. The distinguished features of the proposed reliability estimation method are as follows. Both the measurement error and individual difference are considered in ANN supported Wiener process based degradation model. The measurement error is described by a normal distribution with zero mean. The individual difference is described by introducing hyper parameters. The ANN supported mean function is trained by maximizing the likelihood of the corresponding Wiener process model to the entire evaluation degradation dataset. The hyper parameters for the testing sample population and the model parameters for the monitoring individual are evaluated by Bayesian inference combining with maximum likelihood estimation (MLE).
This paper is organized as follows. The ANN supported Wiener process considering the measurement error and individual difference is presented in Section 2. A three stages parameter estimation method and the associated procedure are presented in Section 3. The proposed model and the corresponding reliability estimation method are demonstrated based on a simulation study and an illustrate example in Sections 4 and 5. Finally, the conclusions are presented in Section 6.
For a Wiener degradation process Y(t), Y(t) has independent increments. Furthermore, the increments ΔY follow Gaussian distribution, as where N(•,•) indicates Gaussian distribution. ΔΛ(t)=Λ(t+Δt)-Λ(t), where Δt is time increment. Λ(t) is the mean function and needs to be monotonically increasing.
Furthermore, the degradation at time t is where Λ(t) and λ2Λ(t) means the degradation mean and variance at time t. λ is called diffusion parameter. The parameter vector for Wiener process θW = (λ).
An ANN supported Wiener process with measurement error and individual difference is proposed. Considering the measurement error, the degradation observation z is given by Eq. (3). where y means the true degradation. Measurement error term ε is assumed to follow a Gaussian distribution with σ2 variance and zero mean. The measurement error parameter vector θE=(σ). Furthermore, similarly as the assumptions in Refs. [37,38,[40], [41], [42]], the measurement error only depend on the utilized sensor and irrelevant to degradation process, so the measurement error is assumed to be s-independent with the other process model's parameters.
Based on an ANN and two parameters, the mean function is given as where ANN(•) means a single input single output ANN, which is used to indicate the failure mechanism and does not vary with different individuals. μ and α are introduced to control the changing rate and the shape of the mean function. Mean function parameter vector θΛ=(α,μ), which are varied with different individuals. The schematic of the proposed ANN supported mean function is shown in Fig. 1.
In the above ANN, sigmoid function is used as activation function for the hidden neurons. Namely, the ANN can be described as where xo is the output of the ANN, xi is the input of the ANN, hk is the value of kth hidden neuron, vk is kth weight of the input layer, wk is kth weight of the output layer, bk is the bias of kth hidden neuron and n is the neuron size of the hidden layer. The ANN parameter vector θANN=(V,W,B) and does not vary with different individuals, where V=(v1,v2,…,vn), W=(w1,w2,…,wn) and B=(b1,b2,…,bn). Here, the model parameter vector θ=(θΛ,θW,θE)=(α,μ,λ,σ). In order to consider the individual differences among the units, the model parameters are described by truncated normal (TN) distribution with hyper parameters, including the diffusion parameter λ, measurement error parameter σ, mean function parameters α and μ. Namely,where θH=(γα,δα,γμ,δμ,γλ,δλ,γσ,δσ) means the corresponding hyper parameter vector. TN(•,•,0,+∞) means the TN distribution and the corresponding probability density function (PDF) is given bywhere ϕ(•) and Φ(•) mean the PDF and the cumulative probability function (CDF) of standard normal distribution.
In this section, a three-stages parameter estimation method is constructed for applying the proposed ANN supported Wiener process. First, an ANN supported mean function training approach is constructed based on the likelihood function, see Stage I. Second, the priors of the hyper parameters are set based on the training results and MLE, see Stage II. Finally, Bayesian inference method is introduced to estimate the hyper parameters when evaluating the testing sample population and update the model parameters when predicting the individual, see Stage II.
For ith sample, the degradation dataset includes the time and observation vectors, Eqs. (8) and (9). where ni is the measurement size of ith sample, tij means j-th inspection time of ith sample and zij is the corresponding measured degradation.
Measurement error is considered in the presented research, so the measured degradation is sum of the true degradation and the measurement error. Namely, zij=yij+εij. Hence, for ith sample, the corresponding PDF is given bywhere fN(•|•,•) means the PDF of Gaussian distribution. ΔΛij=Λ(tij)-Λ(ti(j-1)), Δzij=zij-zi(j-1), Δεij=εij-εi(j-1), zi0=0 and ti0=0. θi=(αi,μi,λi,σi) means the model parameters for ith sample. Furthermore, Δεi follows a multivariate normal distribution, as where the tridiagonal matrix Σ is given by Eq. (12) and |Σ| means the corresponding determinant covariance.
For ith sample, the corresponding likelihood function is given by
The likelihood function for the entire degradation testing dataset is given bywhere means the inspection times of all the nd samples and means the corresponding degradation observations. is the model parameters for all the nd samples.
Hence, the corresponding minus log-likelihood function is given by
It is widely known that, the goodness-of-fit of the model to the dataset is better when the corresponding likelihood is bigger. In order words, the corresponding minus log-likelihood is smaller the goodness-of-fit is better. Hence, the loss function is constructed based on the minus log-likelihood function and a regularization term is introduced to avoid the overfitting problem. The loss function is given by where K means the regularization coefficient.
Furthermore, the mean function should be monotonically increasing. The parameters μ, α, λ and σ should be positive considering their meanings. Hence, the ANN and can be obtained by minimizing the loss function with the above inequality constraints based on the following method, aswhere where the derivative of the activation function is given by
In the presented research, the above optimization problem is handled based on genetic algorithm by using ga function in MATLAB. For the simulation and case studies presented in this paper, the main configuration parameters are set as follows. The crossover fraction is 0.8. The constraint tolerance is 0.001. The population size is 200. The function tolerance is 0.000001. The trained results include the trained ANN parameter vector and trained model parameter vector , where . In the following sections, is used to indicate the trained ANN. Furthermore, the ANN only depends on the failure-generation mechanism, so it does not vary with different individuals and determined at Stage I.
Hyper parameters are introduced to describe the model parameters to consider the individual differences among the units. In the presented research, Bayesian inference is applied to estimate the hyper parameters based on the entire degradation testing dataset and the training results of the model parameters, which are used to set the prior distributions of the hyper parameters. In this section, the priors of hyper parameters are set by MLE based on the above training results of the model parameters. The MLEs of the hyper parameters are obtained by solving Eqs. (20)–(23). The corresponding deductions can be found in the supplementary material.where , , and mean the training results of the corresponding model parameters.
Bayesian inference method is applied to estimate the model parameters. Generally, the parameters’ priors can be set based on the engineering experience and related information when applying Bayesian inference method. Furthermore, the non-informative prior also can be adopted for the conditions lacking related information. For the simulation and case studies presented in this paper, the priors of the hyper parameters are set based on TN distribution and the above MLEs. Furthermore, similarly as Ref. [18], the variances are set to be one-tenth of the MLEs of the hyper parameters for the simulation and case studies, whereas in other cases, it can be set based on engineering experience. The prior distributions of the hyper parameters for the simulation study in Section 4 and case study in Section 5, π(θH), are set as Eq. (24).where the MLEs of the hyper parameters .
Based on the above hyper parameters’ priors, the Bayesian inference for the hyper parameters of the testing sample population, Eq. (25), is performed to estimate the hyper parameters for the testing sample population by solving Eq. (26) with Markov Chain Monte Carlo (MCMC) method.where is the hyper parameters’ posterior distributions for the testing sample population. is the likelihood function for the testing sample population, and , and given bywhere fTN(θi|θH) means the PDF of the parameter vector and given by
Based on Bayesian inference and MCMC method, the hyper parameters for the testing sample population are estimated and indicated as . Given the failure threshold D, the reliability function and the PDF of the lifetime distribution for the testing sample population are given by Eqs. (29) and (30) [46].
Besides the reliability and lifetime of the testing sample population, the individual reliability and lifetime are also noteworthy in engineering practice. When evaluating the model parameters for the monitoring individual, the model parameters’ priors, π(θ), can be set based on the above estimated hyper parameters, as
Based on the above model parameters’ priors, the posterior distributions of the model parameters for the monitoring individual, Z and T, are estimated by Bayesian inference method, Eq. (32). The posterior distributions can be obtained based on Eq. (33) by MCMC method.where means the posterior distributions of the model parameters for the monitoring individual. is the likelihood function for the monitoring individual and given by
Based on the estimation results of the model parameters, , the reliability function and the PDF of the lifetime distribution for the monitoring individual are given by Eq. (35) and Eq. (36), respectively.where means the estimated mean function. Namely, .
The presented reliability estimation method can be applied to estimate the performances of the testing sample population and individual, including lifetime and reliability, considering the measurement error and individual difference among the units. Fig. 2 shows the procedure of the proposed method. Three steps need to be performed. First, the ANN supported mean function and the model parameters are obtained based on the training method shown in Section 3.1. Second, the training results of the model parameters are used to set the hyper parameters’ priors by MLE based method, Section 3.2. Finally, based on the above priors, the hyper parameters for the testing sample population are evaluated by Bayesian inference and used to evaluate the reliability and lifetime of the testing sample population, Section 3.3. The model parameters for the monitoring individual are estimated by Bayesian inference and used to infer the individual reliability and lifetime, in which the model parameters’ priors are set based on the above estimated hyper parameters. The corresponding algorithm is shown in Algorithm 1.
Algorithm 1. Algorithm of the proposed method.
Section snippets
Method verification
The effectiveness of the proposed method is verified by comparing to other two methods, including the method neglecting measurement error and the method neglecting individual difference, as shown in Table 1. In the rests of this paper, M0 indicates the proposed Wiener process and the associated parameter estimation method, M1 indicates the method neglecting measurement error and M2 indicates the method neglecting individual difference. The algorithms for M1 and M2 can be found in the
CRediT authorship contribution statement
Di Liu: Conceptualization, Methodology, Software, Investigation, Visualization, Writing – original draft. Shaoping Wang: Project administration, Writing – review & editing. Xiaoyu Cui: Writing – review & editing.
Declaration of Competing Interest
No author associated with this paper has disclosed any potential or pertinent conflicts which may be perceived to have impending conflict with this work.
Acknowledgments
This study was co-supported by the National Natural Science Foundation of China (52105045, 51620105010, 51575019), the Youth Scholars of Beijing Technology and Business University (QNJJ2021–35), the Natural Science Foundation of Beijing Municipality (L171003) and the China Postdoctoral Science Foundation (2020M680289).
References (46)
- et al.
A prognostic methodology for power MOSFETs under thermal stress using echo state network and particle filter
Microelectron Reliab
(2018) - et al.
A comparative study of statistical and soft computing techniques for reliability prediction of automotive manufacturing
Appl Soft Comput
(2021) - et al.
Physics of failure-based reliability prediction of turbine blades using multi-source information fusion
Appl Soft Comput
(2018) - et al.
Objective Bayesian analysis accelerated degradation test based on Wiener process models
Appl Math Model
(2016) - et al.
An improved Wiener process model with adaptive drift and diffusion for online remaining useful life prediction
Mech Syst Signal Process
(2019) - et al.
Model selection for degradation modeling and prognosis with health monitoring data
Reliab Eng Syst Saf
(2018) - et al.
Two methods for the structured assessment of model uncertainty by experts in performance assessments of radioactive waste repositories
Reliab Eng Syst Saf
(1996) - et al.
Bayesian model averaging based reliability analysis method for monotonic degradation dataset based on inverse Gaussian process and Gamma process
Reliab Eng Syst Saf
(2018) - et al.
An evidence theory based model fusion method for degradation modeling and statistical analysis
Inf Sci
(2020) - et al.
A Bayesian reliability evaluation method with integrated accelerated degradation testing and field information
Reliab Eng Syst Saf
(2013)
Inverse Gaussian process models for degradation analysis: a Bayesian perspective
Reliab Eng Syst Saf
A degradation modeling and reliability estimation method based on Wiener process and evidential variable
Reliab Eng Syst Saf
Wear volume prediction with artificial neural networks
Tribol Int
An artificial neural network supported regression model for wear rate
Tribol Int
Data-driven proton exchange membrane fuel cell degradation predication through deep learning method
Appl Energy
An artificial neural network supported stochastic process for degradation modeling and prediction
Reliab Eng Syst Saf
Reliability estimation from two types of accelerated testing data considering measurement error
Reliab Eng Syst Saf
A Wiener process model for accelerated degradation analysis considering measurement errors
Microelectron Reliab
A reliability estimation approach via Wiener degradation model with measurement errors
Appl Math Comput
Degradation analysis based on an extended inverse Gaussian process model with skew-normal random effects and measurement errors
Reliab Eng Syst Saf
Wiener processes with random effects for degradation data
J Multivar Anal
A load sequence design method for hydraulic piston pump based on time-related Markov matrix
IEEE Trans Reliab
A general accelerated degradation model based on the Wiener process
Materials
Cited by (25)
Accelerated degradation testing for lifetime analysis considering random effects and the influence of stress and measurement errors
2024, Reliability Engineering and System SafetyA Subdomain uncertainty-guided Kriging method with optimized feasibility metric for time-dependent reliability analysis
2024, Reliability Engineering and System SafetyUncertain random accelerated degradation modelling and statistical analysis with aleatory and epistemic uncertainties from multiple dimensions
2024, Reliability Engineering and System SafetyPhysics-informed neural network for first-passage reliability assessment of structural dynamic systems
2023, Computers and Structures