Assessment of the transition-rates importance of Markovian systems at steady state using the unscented transformation
Introduction
Markov models are often used to assess the performance of repairable systems when considering reliability and/or availability metrics [1], [2], [3], [4], [5]. The assessment generally involves the calculations of the steady-state probabilities for each possible system performance state by solving a series of simultaneous equations based on transition rates among component states. A system analyst could then understand how the transition rates affect the performance of a system the most.
One difficult problem is the quantification of these transition rates; if they are available, e.g. from measurement, they may only be considered as estimates with their corresponding uncertainty. Several approaches based on different techniques have been considered to assess the effects of uncertainties in Markov models, for example: Markov set-chains theory [6]; Linear programming [7]; Imprecise theory [8]; Interval probabilities [9], [10]; partial derivatives [11], [12], the Monte Carlo (MC) method [5], Interval Arithmetic (IA) [13], [14], and Affine Arithmetic (AA) [15].
In general, the interval widths derived using these approaches can be used as an importance measure to rank transition rates individually, by quantifying their effects on steady-state probabilities for each possible system performance state. However, they do not allow assessing the possible effects of interactions among components׳ transition rates (e.g., the simultaneously effect of two or more transition rates on steady-state probabilities).
Rocco and Zio [16] proposed the use of global sensitivity analysis (SA) as an alternative approach for assessing transition-rate importance in Markov models. These global methods evaluate the effect of a factor while all others factors are varying as well, thus allowing the exploration of the multi-dimensional input space [17]. The uncertainty in the transition rates is considered as a random variable modeled via a known probability density function and then, specific SA techniques such as FAST [18] are applied to evaluate their corresponding importance. However, one of the main disadvantages of such methods is that they require many evaluations (i.e., solving several linear systems of simultaneous equations).
To avoid the high computational cost, Rocco and Zio [19] proposed the use of a special meta-model based on polynomial chaos expansion (PCE) techniques. A PCE is a multi-dimensional polynomial approximation of the model with coefficients determined by evaluating the model in a significantly reduced set (when compared against traditional SA techniques) of predetermined points. Importance index values are then derived directly from the PCE.
Rocco and Ramirez-Marquez [20] proposed obtaining the importance of the components in the reliability assessment of a system, using an extension of the Unscented Transformation (UT) technique. The approach requires evaluating a very small set of models, linearly proportional to the number of components. In addition, the UT considers that model variables could be statistically depended (e.g., due to the presence of an external variable that simultaneously affects a set of transition rates).
This paper extends the approach presented in Rocco and Ramirez Marquez [20] for assessing the importance of the transition rate uncertainties in the evaluation of the steady-state probability of Markovian behaved systems. To our knowledge, this assessment has not been previously analyzed.
The remainder of this paper is organized as follows: Section 2 defines the Markov model to be considered. Section 3 reviews the UT approach while Section 4 describes the approach to obtain importance measures. Section 5 provides results of experimentation and Section 6 presents the main conclusions.
- CTMC
: continuous time Markov chain
- IM
: importance measure
: probability density function
- SA
: sensitivity analysis
- SR
: state reduction algorithm
- λij
transition rate from state i to state j.
- m
number of state
- Q
transition rate matrix
- n
number of different transition rates
- π
steady-state probability vector (π1, π2, … πm)
- πi
steady-state probability of state i
- Si
main order sensitivity IM
- STi
total order sensitivity IM
- 1.
The system is in steady-state.
- 2.
Time between two successive failures and service time each follows an exponential distribution, so failure rate and repair rate are constant; the process is stationary.
- 3.
The number of states in the Markov models is finite.
- 4.
The Markov chain is aperiodic and irreducible.
Section snippets
Markov model
The transition of a system through different states that represent different operational components states can be described by a discrete-state, continuous time Markov chain (CTMC) Z={z(t), t≥0}, with finite state space E={1,2,…, m}. For each i,j∈E, let λij be the transition rate from state i to state j and λii=−Σi≠jλij, with λii representing the principal diagonal of matrix Q, and defined as the negative sum of transitional rates from j into i. Where Q is defined as the m×m transition rate
The unscented transformation
The UT uses the fact that it is “easier to approximate a probability distribution than to approximate an arbitrary nonlinear function or transformation” [23].The approach is based on selecting a set of points (called sigma points) so that their mean and covariance match the mean and covariance of a selected distribution (not necessarily Gaussian type). Each sigma point is then transformed in a cloud of points that allows the estimation of the mean and covariance of the transformation.
Let X be a
Sensitivity analysis
Sensitivity analysis (SA) is “the study of how uncertainty in the output of a model (numerical or otherwise) can be apportioned to different sources of uncertainty in the model input factors”, with X=(x1, x2, …., xn) being the vector of factors in the model (i.e., the transition rates in our context) [18].
Global Sensitivity Analysis (GSA) considers the exploration of the multi-dimensional input space [17]. Through GSA one is able to determine the effect of a factor while all other factors are
Cases study
Two examples are presented in this section to show the approach based on UT is implemented
Conclusions
In this paper the Unscented Transformation has been used for analyzing the uncertainty propagation problem and for assessing the importance of the transition rates in a Markovian system. The uncertainty in the steady-state probability is approximated solving several linear system equations on a set of deterministically defined “sigma points”, proportional to the number of transition rates. The UT considers that transition rates could be statistically depended. As a result, tight estimations of
References (38)
- et al.
Solving Markovian systems of O.D.E. for availability and reliability calculations
Reliab Eng Syst Saf
(1995) - et al.
Carlo estimation of the marginal distributions in a problem of probabilistic dynamics
Reliab Eng Syst Saf
(1996) Discrete time Markov chains with interval probabilities
Int J Approx Reason
(2009)- et al.
Imprecise Markov chains with absorption
Int J Approx Reason
(2010) - et al.
Reliability importance analysis of Markovian systems at steady state using perturbation analysis
Reliab Eng Syst Saf
(2008) - et al.
From differential to difference importance measures for Markov reliability models
Eur J Oper Res
(2010) - et al.
An effective screening design for sensitivity analysis of large models
Environ Model Softw
(2007) An unscented Kalman smoother for volatility extraction: evidence from stock prices and options
Comput Stat Data Anal
(2013)- et al.
Reliability model of the power transformer with ONAF cooling
Electr Power Energy Syst
(2012) - et al.
Composite indicators for security of energy supply using ordered weighted averaging
Reliab Eng Syst Saf
(2011)
Reliability evaluation of engineering systems
Sensitivity and uncertainty analysis of markov-reward models
IEEE Trans Reliab
Markov set-chains
Interval-valued finite Markov chains
Reliab Comput
Imprecise Markov chains and their limit behaviour
Probab Eng Inf Sci
Cited by (23)
QB-II for evaluating the reliability of binary-state networks
2023, Reliability Engineering and System SafetyCitation Excerpt :Reliability is a popular index for evaluating network performance. Reliability is the success probability that the network is still functioning under pregiven conditions, such as flow amount [22,23], signal quality [24], budget [25], transition time [26], computer grids [27] and redundance numbers [28]. It is both NP-hard and # P-hard to calculate the exact reliability of binary-state networks [7–9].
Novel self-adaptive Monte Carlo simulation based on binary-addition-tree algorithm for binary-state network reliability approximation
2022, Reliability Engineering and System SafetyNew binary-addition tree algorithm for the all-multiterminal binary-state network reliability problem
2022, Reliability Engineering and System SafetyNovel binary addition tree algorithm (BAT) for calculating the direct lower-bound of the highly reliable binary-state network reliability
2022, Reliability Engineering and System SafetyCitation Excerpt :Reliability is a very useful performance and functionality metric for networks. Reliability has been widely used in many studies in recent decades [6,16–21,23–26]. For example, Aven proposed a standard for measuring the reliability of binary-state transportation networks [16].
Novel Algorithm for Computing All-Pairs Homogeneity-Arc Binary-State Undirected Network Reliability
2021, Reliability Engineering and System SafetyCitation Excerpt :Among various real-life emerging applications, the binary-state network is the fundamental structure and each component is either working or failed. For example, the network transmission problems involving (signals) communication [1], (liquids or gases) distribution [2,3], (traffic) transportation [4], (topology) transformation [5,6,7], (power) transmission [8]; grid/cloud computing [9]; data mining [10]; Internet of Things [11,12]; and network resilience problems [13,14] all can be modeled as binary-state networks. All components (arcs and/or nodes) in binary-state networks composed of binary states, i.e., working or failed [15,16].
A quick BAT for evaluating the reliability of binary-state networks
2021, Reliability Engineering and System Safety