Assessment of the transition-rates importance of Markovian systems at steady state using the unscented transformation

https://doi.org/10.1016/j.ress.2015.05.019Get rights and content

Highlights

  • Evaluation of the effects of uncertainty in transition rates.

  • Uncertainty propagation of steady-state probabilities through a reduced number of evaluations.

  • Immediate computation of transition rate importances.

  • Application to real Markovian systems.

Abstract

The Unscented Transformation (UT) is a technique to understand and compute how the uncertainty of a set of random variables, with known mean and variance is propagated on the outputs of a model, through a reduced set of model evaluations as compared with other approaches (e.g., Monte Carlo). This computational effort reduction along with the definition of a proper UT model allows proposing an alternative approach to quantify the transition rates (TR) having the highest contribution to the variance of the steady-state probability, for each possible state of a system represented by a Markov model. The so called “main effects” of each transition rate, as well as high order component interactions are efficiently derived from the solution of only (2n+1) linear system of simultaneous equations, being n the number of transition rates in the model.

Introduction

Markov models are often used to assess the performance of repairable systems when considering reliability and/or availability metrics [1], [2], [3], [4], [5]. The assessment generally involves the calculations of the steady-state probabilities for each possible system performance state by solving a series of simultaneous equations based on transition rates among component states. A system analyst could then understand how the transition rates affect the performance of a system the most.

One difficult problem is the quantification of these transition rates; if they are available, e.g. from measurement, they may only be considered as estimates with their corresponding uncertainty. Several approaches based on different techniques have been considered to assess the effects of uncertainties in Markov models, for example: Markov set-chains theory [6]; Linear programming [7]; Imprecise theory [8]; Interval probabilities [9], [10]; partial derivatives [11], [12], the Monte Carlo (MC) method [5], Interval Arithmetic (IA) [13], [14], and Affine Arithmetic (AA) [15].

In general, the interval widths derived using these approaches can be used as an importance measure to rank transition rates individually, by quantifying their effects on steady-state probabilities for each possible system performance state. However, they do not allow assessing the possible effects of interactions among components׳ transition rates (e.g., the simultaneously effect of two or more transition rates on steady-state probabilities).

Rocco and Zio [16] proposed the use of global sensitivity analysis (SA) as an alternative approach for assessing transition-rate importance in Markov models. These global methods evaluate the effect of a factor while all others factors are varying as well, thus allowing the exploration of the multi-dimensional input space [17]. The uncertainty in the transition rates is considered as a random variable modeled via a known probability density function and then, specific SA techniques such as FAST [18] are applied to evaluate their corresponding importance. However, one of the main disadvantages of such methods is that they require many evaluations (i.e., solving several linear systems of simultaneous equations).

To avoid the high computational cost, Rocco and Zio [19] proposed the use of a special meta-model based on polynomial chaos expansion (PCE) techniques. A PCE is a multi-dimensional polynomial approximation of the model with coefficients determined by evaluating the model in a significantly reduced set (when compared against traditional SA techniques) of predetermined points. Importance index values are then derived directly from the PCE.

Rocco and Ramirez-Marquez [20] proposed obtaining the importance of the components in the reliability assessment of a system, using an extension of the Unscented Transformation (UT) technique. The approach requires evaluating a very small set of models, linearly proportional to the number of components. In addition, the UT considers that model variables could be statistically depended (e.g., due to the presence of an external variable that simultaneously affects a set of transition rates).

This paper extends the approach presented in Rocco and Ramirez Marquez [20] for assessing the importance of the transition rate uncertainties in the evaluation of the steady-state probability of Markovian behaved systems. To our knowledge, this assessment has not been previously analyzed.

The remainder of this paper is organized as follows: Section 2 defines the Markov model to be considered. Section 3 reviews the UT approach while Section 4 describes the approach to obtain importance measures. Section 5 provides results of experimentation and Section 6 presents the main conclusions.

    CTMC

    : continuous time Markov chain

    IM

    : importance measure

    pdf

    : probability density function

    SA

    : sensitivity analysis

    SR

    : state reduction algorithm

    λij

    transition rate from state i to state j.

    m

    number of state

    Q

    transition rate matrix

    n

    number of different transition rates

    π

    steady-state probability vector (π1, π2, … πm)

    πi

    steady-state probability of state i

    Si

    main order sensitivity IM

    STi

    total order sensitivity IM

  • 1.

    The system is in steady-state.

  • 2.

    Time between two successive failures and service time each follows an exponential distribution, so failure rate and repair rate are constant; the process is stationary.

  • 3.

    The number of states in the Markov models is finite.

  • 4.

    The Markov chain is aperiodic and irreducible.

Section snippets

Markov model

The transition of a system through different states that represent different operational components states can be described by a discrete-state, continuous time Markov chain (CTMC) Z={z(t), t≥0}, with finite state space E={1,2,…, m}. For each i,jE, let λij be the transition rate from state i to state j and λii=−Σi≠jλij, with λii representing the principal diagonal of matrix Q, and defined as the negative sum of transitional rates from j into i. Where Q is defined as the m×m transition rate

The unscented transformation

The UT uses the fact that it is “easier to approximate a probability distribution than to approximate an arbitrary nonlinear function or transformation” [23].The approach is based on selecting a set of points (called sigma points) so that their mean and covariance match the mean and covariance of a selected distribution (not necessarily Gaussian type). Each sigma point is then transformed in a cloud of points that allows the estimation of the mean and covariance of the transformation.

Let X be a

Sensitivity analysis

Sensitivity analysis (SA) is “the study of how uncertainty in the output of a model (numerical or otherwise) can be apportioned to different sources of uncertainty in the model input factors”, with X=(x1, x2, …., xn) being the vector of factors in the model (i.e., the transition rates in our context) [18].

Global Sensitivity Analysis (GSA) considers the exploration of the multi-dimensional input space [17]. Through GSA one is able to determine the effect of a factor while all other factors are

Cases study

Two examples are presented in this section to show the approach based on UT is implemented

Conclusions

In this paper the Unscented Transformation has been used for analyzing the uncertainty propagation problem and for assessing the importance of the transition rates in a Markovian system. The uncertainty in the steady-state probability is approximated solving several linear system equations on a set of deterministically defined “sigma points”, proportional to the number of transition rates. The UT considers that transition rates could be statistically depended. As a result, tight estimations of

References (38)

  • R. Billinton et al.

    Reliability evaluation of engineering systems

    (1992)
  • Blake J, Reibman AL, Trivedi KS. Sensitivity analysis of reliability and performability systems. In: Proceedings of the...
  • B. Havekort et al.

    Sensitivity and uncertainty analysis of markov-reward models

    IEEE Trans Reliab

    (1995)
  • D.J. Hartfiel

    Markov set-chains

    (1998)
  • I.O. Kozine et al.

    Interval-valued finite Markov chains

    Reliab Comput

    (2002)
  • G. De Cooman et al.

    Imprecise Markov chains and their limit behaviour

    Probab Eng Inf Sci

    (2009)
  • Rocco CM. Uncertainty analysis of Markov systems using interval arithmetic. In: Gutwin TB, editor. Proceedings of the...
  • Rocco CM. Effects of transition-rate uncertainty on steady-state probabilities of Markov models using interval...
  • Rocco CM. Affine arithmetic for assessing the uncertainty propagation on steady-state probabilities of Markov models...
  • Cited by (23)

    • QB-II for evaluating the reliability of binary-state networks

      2023, Reliability Engineering and System Safety
      Citation Excerpt :

      Reliability is a popular index for evaluating network performance. Reliability is the success probability that the network is still functioning under pregiven conditions, such as flow amount [22,23], signal quality [24], budget [25], transition time [26], computer grids [27] and redundance numbers [28]. It is both NP-hard and # P-hard to calculate the exact reliability of binary-state networks [7–9].

    • Novel binary addition tree algorithm (BAT) for calculating the direct lower-bound of the highly reliable binary-state network reliability

      2022, Reliability Engineering and System Safety
      Citation Excerpt :

      Reliability is a very useful performance and functionality metric for networks. Reliability has been widely used in many studies in recent decades [6,16–21,23–26]. For example, Aven proposed a standard for measuring the reliability of binary-state transportation networks [16].

    • Novel Algorithm for Computing All-Pairs Homogeneity-Arc Binary-State Undirected Network Reliability

      2021, Reliability Engineering and System Safety
      Citation Excerpt :

      Among various real-life emerging applications, the binary-state network is the fundamental structure and each component is either working or failed. For example, the network transmission problems involving (signals) communication [1], (liquids or gases) distribution [2,3], (traffic) transportation [4], (topology) transformation [5,6,7], (power) transmission [8]; grid/cloud computing [9]; data mining [10]; Internet of Things [11,12]; and network resilience problems [13,14] all can be modeled as binary-state networks. All components (arcs and/or nodes) in binary-state networks composed of binary states, i.e., working or failed [15,16].

    • A quick BAT for evaluating the reliability of binary-state networks

      2021, Reliability Engineering and System Safety
    View all citing articles on Scopus
    View full text