LSTM-augmented deep networks for time-variant reliability assessment of dynamic systems
Introduction
Reliability analysis is an important procedure during engineering products development, which aims at computing the probability that a system or component successfully fulfills its intended functionalities with the consideration of various uncertainties. Research efforts have been devoted to enhancing the accuracy and efficiency when estimating the system reliability. First- and Second-order reliability methods (FORM and SORM) [1], [2], [3], [4] have been utilized to analytically estimate the time-independent reliability. By transforming a limit state function into a standard normal space, a most probable point (MPP) is identified as the closest point from the failure surface to the origin for reliability assessment based on the first- or second-order Taylor expansion. Surrogate modeling techniques [5], [6], [7], [8], [9] have been adopted to alleviate the burden of expensive computational costs and approximate reliability by employing sampling methods such as Monte Carlo simulation (MCS) [10], [11], [12] and importance sampling [13], [14], [15], [16]. As the performance of engineered systems gradually degrades over time, effective time-dependent reliability analysis [17,18] is of critical importance to practical engineering applications.
In the literature, time-dependent reliability analysis approaches have been investigated [19], [20], [21], [22], [23], such as extreme value based methods [24,25], out-crossing rate method [26], [27], [28], and surrogate model based methods [29,30]. The primary target of extreme value-based methods is to identify the extreme system performance, which allows the use of traditional time-independent reliability analysis methods for assessing the time-dependent reliability. However, the analytical solution for the probabilistic characterization of extreme response is intractable for practical problems. Though surrogate modeling techniques [31,32] can be used to estimate the distribution of extreme response, the required number of function evaluations is often significant. The out-crossing event occurs when dynamic response passes an allowable threshold, and the rate of change in the probability with respect to time is defined as the out-crossing rate. Rice's formula [33] assumes that all the out-crossing events are statistically independent, and the time-dependent reliability is estimated based on the integration of the out-crossing rate throughout the entire time period. In PHI2 method, Andrieu-Renaud et al. [34] proposed a parallel system with limit state functions at each time instant to estimate the out-crossing rate based on FORM and SORM. To enhance the accuracy of time-dependent reliability analysis, Hu and Du [35] employed FORM to compute the out-crossing rate as well as the joint out-crossing rate, which accounts for the correlation between out-crossing events. The Polynomial Chaos Expansion (PCE) based methods [36], [37], [38] build a global surrogate model based on a series of orthogonal polynomials with respect to the distributions of the input random variables, and Hawchar et al. [39] extended the PCE method to time-dependent reliability analysis. Based on the instantaneous performance functions, the principal component analysis method is adopted to identify a reduced number of components, and time-dependent reliability is estimated by a sparse PCE with an adaptive algorithm. To alleviate the complexity of the training process, Hu and Mahadevan [40] developed a single-loop updating algorithm to identify global extreme response values. Wu et al. [41] integrated parallel computing with adaptive Kriging methods for efficient time-dependent reliability analysis, where a parallel expected improvement criteria is introduced to determine multiple additional samples for updating the surrogate model. Zafar and Wang [42] developed a transfer learning-based method for efficient time-dependent reliability analysis. Jiang et al. [43] developed an adaptive sampling method for improving the accuracy of time-dependent reliability analysis using a Kriging model, where the statistical information of the stochastic process is used to identify the high probability density sampling region. Though significant advances have been made in time-dependent reliability analysis, most of the existing methods handle time-dependent uncertainties by building instantaneous reliability models, which lacks the capability of capturing the long-term dependency of system dynamics.
Recently, deep learning has become popular in the machine learning field due to its potential advantages in handling large scale data and feature extraction. Based on the information collected from data, deep learning utilizes a layered structure of algorithms to make predictions, and it has been applied for classification and regression problems in a variety of fields [44,45] such as image processing, medical information processing, robotics and control, computer vision and etc. To deal with data sequences, inputs and historical information are processed in the internal memory of recurrent neural network. Vanishing and exploding gradient problems may occur during the training process of recurrent neural network, which significantly affect the training speed and prediction accuracy. To tackle the challenge, long-short term memory network (LSTM) has been introduced with a structure of multiple gates to control the information flow, and it has been successfully applied in speech recognition, natural language processing, and condition-based maintenance [46,47].
In this paper, a LSTM-augmented deep learning framework is developed to handle time-dependent uncertainties in reliability analysis of dynamic systems. A set of local-limit state functions is first introduced by sampling time-independent random variables. Then a long short-term memory network is trained to learn the time-dependent behavior of the dynamic system for each of local-limit state functions. With multiple local surrogate LSTM models, augmented dataset can be collected accordingly. To build a global surrogate model of dynamic systems, two types of feedforward neural networks (FNN) can be constructed based on the augmented data to predict the minimum response or the overall time-dependent responses, respectively. By employing the Gaussian process (GP) regression technique, the numbers of neurons of the FNN are optimized through a response surface of validation loss. As a result, the trained FNNs can be utilized as global surrogate models of dynamic system to make predictions for system responses with the consideration of both time-independent and time-dependent uncertainties. By employing the Monte Carlo simulation, the global surrogate model can be directly utilized for estimating the time-dependent reliability without incurring extra computational costs. The remainder of this paper is organized as follows. Section 2 presents the problem statement of time-dependent reliability analysis. Section 3 introduces the details of the LSTM-augmented deep learning framework. The effectiveness of the proposed approach is demonstrated using three examples in Section 4, while discussions and concluding remarks will be presented in Section 5.
Section snippets
Problem statement
A dynamic system can be modeled by a series of differential equation representations, expressed aswhere xs(t) denotes the state variables, z(t) = [z1(t), z2(t), …, zk(t)] is a stochastic process. In general, the output of a dynamic system can be modeled in a discrete space by the following formwhere F(.) is an unknown function representing the system dynamics, z(t) and y(t) represent the system input and output, ny and nz are
LSTM-augmented deep learning framework
The proposed framework aims at effectively handling both time-independent and time-dependent uncertainties for reliability analysis of dynamic systems. Multiple local surrogate LSTM models are first constructed to learn local-limit state functions that are identified by sampling the time-independent random space. Then time-dependent response predictions from the local surrogate LSTM models are collected and further utilized for training a FNN. A response surface of validation loss with respect
Case studies
In this section, three examples are used to demonstrate the effectiveness of the proposed approach by solving time-dependent reliability analysis problems.
Conclusion
In this paper, a LSTM-augmented deep learning framework is established for time-dependent reliability analysis of dynamic systems. To systematically handle time-independent random variables and stochastic processes, multiple LSTM models are first constructed to learn local-limit state functions. With the local surrogate LSTMs, a set of augmented data is collected by predicting time-dependent responses for random realizations of the stochastic process. Based on the augmented data, a feedforward
CRediT authorship contribution statement
Mingyang Li: Data curation, Formal analysis, Investigation, Software, Visualization, Writing – original draft. Zequn Wang: Conceptualization, Investigation, Methodology, Software, Supervision, Validation, Visualization, Writing – review & editing.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
References (50)
- et al.
A hybrid relaxed first-order reliability method for efficient structural reliability analysis
Struct Saf
(2017) - et al.
A confidence-based reliability optimization with single loop strategy and second-order reliability method
Comput Methods Appl Mech Eng
(2020) - et al.
A general failure-pursuing sampling framework for surrogate-based reliability analysis
Reliab Eng Syst Saf
(2019) - et al.
A new adaptive sequential sampling method to construct surrogate models for efficient reliability analysis
Reliab Eng Syst Saf
(2018) - et al.
Active learning and active subspace enhancement for PDEM-based high-dimensional reliability analysis
Struct Saf
(2021) - et al.
Efficient reliability analysis based on adaptive sequential sampling design and cross-validation
Appl Math Model
(2018) - et al.
An adaptive directional importance sampling method for structural reliability analysis
Struct Saf
(2018) - et al.
Importance sampling-based system reliability analysis of corroding pipelines considering multiple failure modes
Reliab Eng Syst Saf
(2018) - et al.
Reliability analysis with stratified importance sampling based on adaptive Kriging
Reliab Eng Syst Saf
(2020) - et al.
Time-dependent reliability analysis of service-proven quay walls subject to corrosion-induced degradation
Reliab Eng Syst Saf
(2020)
A time-variant extreme-value event evolution method for time-variant reliability analysis
Mech Syst Signal Process
Dynamic reliability analysis using the extended support vector regression (X-SVR)
Mech Syst Signal Process
Sequential time-dependent reliability analysis for the lower extremity exoskeleton under uncertainty
Reliab Eng Syst Saf
Application of adaptive surrogate models in time-variant fatigue reliability assessment of welded joints with surface cracks
Reliab Eng Syst Saf
An efficient Kriging based method for time-dependent reliability based robust design optimization via evolutionary algorithm
Comput Methods Appl Mech Eng
A new approach for reliability analysis with time-variant performance characteristics
Reliab Eng Syst Saf
Confidence-based adaptive extreme response surface for time-variant reliability analysis under random excitation
Struct Saf
The PHI2 method: a way to compute time-variant reliability
Reliab Eng Syst Saf
Stochastic model reduction for polynomial chaos expansion of acoustic waves using proper orthogonal decomposition
Reliab Eng Syst Saf
Identification of high-dimension polynomial chaos expansions with random coefficients for non-Gaussian tensor-valued random fields using partial and limited experimental data
Comput Methods Appl Mech Eng
Principal component analysis and polynomial chaos expansion for time-variant reliability problems
Reliab Eng Syst Saf
Parallel efficient global optimization method: a novel approach for time-dependent reliability analysis and applications
Expert Syst Appl
An active failure-pursuing Kriging modeling method for time-dependent reliability analysis
Mech Syst Signal Process
A dual-LSTM framework combining change point detection and remaining useful life prediction
Reliab Eng Syst Saf
Prediction of remaining useful life of multi-stage aero-engine based on clustering and LSTM fusion
Reliab Eng Syst Saf
Cited by (19)
Line sampling for time-variant failure probability estimation using an adaptive combination approach
2024, Reliability Engineering and System SafetyA loading contribution degree analysis-based strategy for time-variant reliability analysis of structures under multiple loading stochastic processes
2024, Reliability Engineering and System SafetyApplication of LSTM based on the BAT-MCS for binary-state network approximated time-dependent reliability problems
2023, Reliability Engineering and System SafetyCitation Excerpt :In general, MCS-based algorithms uniformly generate random values for nodes/arcs to simulate the state of the entire network repeatedly, and then average the results to approximate reliability. By employing the Monte Carlo simulation, the proposed approach can be directly utilized for estimating the time-dependent reliability without incurring extra computational costs [33]. However, MCS-based algorithms cannot provide fixed and acceptable approximations of reliability even after generating a large number of random values.
Real-time pipeline leak detection and localization using an attention-based LSTM approach
2023, Process Safety and Environmental ProtectionA global–local attention network for uncertainty analysis of ground penetrating radar modeling
2023, Reliability Engineering and System Safety