On almost sure stability of continuous-time Markov jump linear systems

doi:10.1016/j.automatica.2006.02.007

Automatica

Volume 42, Issue 6, June 2006, Pages 983-988

https://doi.org/10.1016/j.automatica.2006.02.007 Get rights and content

Abstract

In this paper, we study the almost sure stability of continuous-time jump linear systems with a finite-state Markov form process. A sufficient condition for almost sure stability is derived that refers to the statistics of the transition matrix over m switches. It is shown that, if the system is exponentially almost sure stable, there exists a finite m such that the criterion is satisfied. In order to evaluate the expected value appearing in the condition, an efficient Monte Carlo algorithm is worked out.

Introduction

Markov jump linear systems (MJLS) are a class of stochastic hybrid systems in which the switches between different linear systems are governed by a finite-state Markov chain. They are well suited to represent dynamic systems subject to random switches between alternative configurations. As such, they are used to model technological and economic systems, including failure-prone plants and communication lines subject to random delays (Chizeck et al., 1986, Costa et al., 2005, Fang et al., 1991, Gomez-Puig and Montalvo, 1997, Krtolica et al., 1991, Mariton, 1990).

The stability theory of MJLS is rather complex in that there exist several stability notions that differ in conservativeness as well as ease of testability. The most important stability notions are mean-square (MS) stability, moment stability, and almost sure (AS) stability. MS-stability has to do with the asymptotic convergence to zero of the second moment of the state norm. There exist necessary and sufficient conditions for MS-stability involving either the solution of coupled Lyapunov equations or the location in the complex plane of the eigenvalues of suitable augmented matrices (Fang and Loparo, 2002b, Fang et al., 1994, Feng et al., 1992).

Moment stability, also called $δ$ -moment stability, requires the convergence to zero of the moment of order $δ$ (MS-stability is just a particular case for $δ = 2$ ). Although there exist some practical sufficient conditions, a simple necessary and sufficient condition for testing $δ$ -moment stability is not available (except for $δ = 2$ ).

Finally, AS-stability holds if the sample path of the state converges to zero with probability one. Checking AS-stability involves the determination of the sign of the top Lyapunov exponent, which is usually a rather difficult task (Arnold et al., 1986, Fang and Loparo, 2002a). In practice, one may exploit the fact that $δ$ -moment stability implies AS-stability and that, for $δ$ tending to zero, AS and $δ$ -moment stability become equivalent (Fang et al., 1994, Feng et al., 1992). However, in view of the lack of simple necessary and sufficient conditions for $δ$ -moment stability, the problem of assessing AS-stability in the least conservative way is still open. In practical applications the most useful notion would be AS-stability because it guarantees the convergence of almost all realizations of the sample path. Conversely, $δ$ -moment stability may be too conservative. Sufficient conditions for AS-stability that do not rely on $δ$ -moment stability are reported in Fang (1997).

With reference to discrete-time MJLS, the authors of the present paper have recently derived a family of criteria for testing AS-stability whose conservativeness can be made arbitrarily small at the expense of computational complexity (Bolzern, Colaneri, & De Nicolao, 2004). More precisely, a sufficient condition for AS-stability is applied to a lifted representation over m steps of the original MJLS. It has been proven that if the system is AS-stable, a finite m exists such that the criterion is fulfilled. As the lifting horizon m grows, it becomes necessary to resort to a Monte Carlo strategy in order to calculate the expected value appearing in the stability criterion.

The extension of these recent results to the continuous-time case is all but straightforward. In fact, a lifted representation over a given time-interval would yield an infinite-state Markov chain. For this reason, a different approach is pursued in the present paper. A condition for testing the contractivity of the MJLS after m switches is applied. Such a contractivity depends on the expected value of the logarithm of the norm of the transition matrix, an expectation that can be calculated by means of a Monte Carlo strategy. The Monte Carlo algorithm requires only the stochastic simulation of m switches of the continuous-time Markov chain and takes a decision according to a prescribed confidence level. It is shown that, if the MJLS is AS-stable, there exists a finite value of m such that the contractivity condition is satisfied. In other words, the sufficient condition asymptotically approaches necessity.

The paper is organized as follows. In Section 2, some basic definitions are recalled. The new AS-stability condition is worked out in Section 3 together with a couple of related necessary conditions, while Section 4 deals with the randomized algorithm. An illustrative example is discussed in Section 5. The paper ends with some concluding remarks (Section 6).

Section snippets

Continuous-time MJLS—preliminaries

Consider the continuous-time MJLS $\dot{x} (t) = A (σ (t)) x (t),$ where $x (t) \in R^{n}$ , and the form process $σ (t)$ is a finite-state, time homogeneous, Markov stochastic process taking values in a finite set $S = {1, 2, \dots, M}$ , with stationary transition probabilities $\Pr {σ (t + h) = j | σ (t) = i} = q_{ij} h + o (h), i \neq j$ , where $h > 0$ , and $q_{ij}$ is the transition rate from mode i at time t to mode j at time $t + h$ . Letting $q_{ii} = - \sum_{j = 1, j \neq i}^{M} q_{ij}$ and defining $Q = [q_{ij}]$ , the matrix Q is called the infinitesimal generator of the Markov process. Let $τ_{k}, k = 0, 1, \dots$ be

Main result

In this section, we will work out a new sufficient condition for EAS-stability that is less restrictive than Theorem 1. The key idea is to consider the evolution of the state $x (t)$ over an interval of time corresponding to m transitions and to impose that the system is averagely norm-contractive over such an interval. To this aim, define $T_{m} = \sum_{k = 0}^{m - 1} τ_{k}$ , with $T_{0} = 0$ , as the (random) time to the mth transition, and recall that $Φ (T_{m}, 0)$ is the (random) state transition matrix over the interval $[0, T_{m}]$ .

Randomized algorithm for assessment of almost sure stability

When applying condition (4), one is faced with the problem of calculating the expected value. An exact formula is not available, so that a Monte Carlo strategy is proposed.

First of all, the user has to select a confidence level $δ$ (e.g. $δ = 0.01$ ). The scalar $δ$ is the probability that the algorithm erroneously decides that the expected value is negative when it is actually equal to zero. The same confidence level is chosen for the probability of deciding that the expected value is positive when it

Example

As an example, consider the MJLS (1) with $M = 2$ , and $A (1) = [\begin{matrix} 2 & 0 \\ 0 & - 1 \end{matrix}], A (2) = [\begin{matrix} - 15 & 1 \\ 0 & - 2 \end{matrix}], Q = [\begin{matrix} - 1 & 1 \\ 5 & - 5 \end{matrix}] .$ The unique invariant distribution of the continuous-time form process is $\bar{π} = {[\begin{matrix} 5 / 6 & 1 / 6 \end{matrix}]}^{'}$ and the invariant distribution of the embedded Markov chain turns out to be $\bar{ϑ} = [1 / 2 1 / 2]^{'} .$ First of all, observe that this system is not MS-stable. In fact, the matrix $A (1) + 0.5 q_{11} I$ is not stable, so that the necessary condition for MS-stability given in Fang (1994) is violated. Nevertheless, it could be EAS-stable.

Since $μ (A (1)) = 2$ ,

Concluding remarks

A new sufficient condition for almost sure stability of continuous-time MJLS has been worked out. Roughly speaking, the condition relies on the average contractivity of the system over m transitions. It is shown that, if the system is EAS-stable, there exists a finite m such that the condition is fulfilled. Therefore, in some sense, the sufficient condition approaches necessity as m grows. The computation is carried out by means of a Monte Carlo algorithm guaranteeing a prescribed confidence

Acknowledgments

This paper has been partially supported by the Italian National Research Council (CNR) and by MIUR project “New methods and algorithms for identification and adaptive control of technological systems”.

References (16)

Arnold, L., Kleimann, W., & Oeljeklaus, E. (1986). Lyapunov exponents of linear stochastic systems. In L. Arnold, & V....
Bolzern, P., Colaneri, P., & De Nicolao, G. (2004). On almost sure stability of discrete-time Markov jump linear...
P. Bremaud
Markov chains: Gibbs fields, Monte Carlo simulation, and queues
(1998)
H.J. Chizeck et al.
Discrete-time Markovian-jump quadratic optimal control
International Journal of Control
(1986)
O.L.V. Costa et al.
Discrete-time Markov jump linear systems
(2005)
C.A. Desoer et al.
Feedback systems: Input–output properties
(1975)
Fang, Y. (1994). Stability analysis of linear control systems with uncertain parameters. Ph.D. Dissertation, Department...
Y. Fang
A new general sufficient condition for almost sure stability of jump linear systems
IEEE Transactions on Automatic Control
(1997)

There are more references available in the full text version of this article.

Cited by (108)

Almost sure finite-time synchronization of Markov jump complex dynamical networks with asynchronous switching
2024, Neurocomputing
The almost sure finite-time synchronization for Markovian jump complex dynamical networks (CDNs) with time-varying delay is studied. The designed controller is supposed to be asynchronous with the CDNs states. The existing almost sure synchronization result of networks is explored within a finite time interval. The sampled-data control approach is employed to improve the control precision. Some sufficient criteria are obtained under the definition of almost sure finite-time synchronization for the CDNs with Markovian switching topology. To determine the asynchronous controller gain, the linear matrix inequality-based conditions are proposed. Then, two examples are given to illustrate the effectiveness.
Control of discrete-time semi-Markovian switched linear singular systems
2024, Systems and Control Letters
This paper studies the issues of stability analysis and stabilizing controller design for a class of discrete-time semi-Markovian switched linear singular systems. The novel sufficient criteria for such systems to be $σ$ -error mean square admissible are proposed for the first time. By adopting the improved technology of eliminating matrix power terms for singular systems, the existence conditions are proposed for state feedback controller in terms of a group of linear matrix inequalities. An example is presented to illustrate the validity of our results.
Stability of stochastic systems with semi-Markovian switching and impulses
2023, Chaos, Solitons and Fractals
The aim of this paper is to investigate the stochastic asymptotic stability of semi-Markov switched systems with mixed impulses. A novel definition of average impulse gain is given to estimate the intensity of the mixed impulses. Based on the multiple Lyapunov function approach and average impulse interval method, we provide sufficient conditions on the stochastic asymptotic stability for the semi-Markov switched system with impulses. Moreover, the influence of impulses on the system is estimated by applying the average impulse gain and average impulse interval method, which are not only suitable for synchronous impulses but also for asynchronous impulses. The theoretical results are demonstrated by two examples.
Stability analysis and stabilization of discrete-time non-homogeneous semi-Markov jump linear systems: A polytopic approach
2020, Automatica
Citation Excerpt :
To model this type of dynamic systems effectively, Markov jump systems (MJSs) are employed extensively, see, e.g., Benjelloun and Boukas (1998), Bolzern, Colaneri, and Nicolao (2006), Chen, Sun, and Chen (2018), Chen, Sun, and Xia (2018), Geromel, Deaecto, and Colaneri (2016), Samidurai, Manivannan, Ahn, and Karimi (2018) and Zhu and Zhang (2017). During the past two decades, quite a few significant works have been reported on homogeneous MJSs, see, e.g., Bolzern et al. (2006), Chen, Sun, and Chen (2018), Chen, Sun, and Xia (2018), Geromel et al. (2016), Wang and Zhu (2017), Zhu (2014) and Zhu (2017), in which the switching dynamics are governed by homogeneous Markov chains. In homogeneous MJSs, the transition probabilities (TPs) are certain and time-invariant, and the random sojourn time obeys exponential distributions in continuous-time domain or geometric distributions in the discrete-time case.
This paper addresses the issues of stability analysis and stabilizing controller design for two classes of discrete-time non-homogeneous semi-Markov jump linear systems (S-MJLSs). The first class is concerned with the sojourn-time probability mass functions that are independent of jump instants, while the other considers the existence of probability mass functions of sojourn time depending on jump instants. New techniques are developed respectively for both classes of systems by transforming their corresponding time-varying semi-Markov kernels into more tractable forms. Based on the stochastic stability allowing for bounded sojourn time, the derived stability and stabilization criteria containing the probability distribution information of random sojourn time can be numerically tested. In order to reduce the possible conservatism of the obtained results, a novel class of polytopic quadratic Lyapunov function is constructed, which depends on both the current mode and the elapsed time in the current mode. Finally, the proposed control strategies are applied to an illustrative application of automotive electronic throttle valve to show the effectiveness of the theoretical results. The results also reveal the significance of considering the inhomogeneity of S-MJLSs and the importance of our constructed Lyapunov function.
Almost sure stability and stabilization of Markovian jump systems with alternative and continuous controller failures
2020, Journal of the Franklin Institute
This paper considers the exponential almost sure (EAS) stability and stabilization problems of continuous-time Markovian jump systems (MJSs) with alternative and continuous controller failures. By introducing a piecewise constant function, the dwell times of a controller useful or not are modeled to be time-variant, whose bounds are also included. Then, sufficient conditions for EAS stability are established by using a method on its stochastic transfer matrix. Moreover, the statistically transient properties of Markov process are considered, and the control gains is gotten by solving some Lyapunov equations. Two examples are used to demonstrate the effectiveness and superiority of the proposed methods.
Stabilization of Discrete-Time Hidden Semi-Markov Jump Linear Systems With Partly Unknown Emission Probability Matrix
2024, IEEE Transactions on Automatic Control

View all citing articles on Scopus

Paolo Bolzern was born in Milano, Italy in 1955. He received a Doctor's degree (Laurea) in Electronic Engineering from the Politecnico di Milano in 1978. From 1979 to 1981 he worked on modeling and identification of environmental systems at IIASA, Laxenburg, Austria, and at the Politecnico di Milano. From 1983 to 1987, he was a research fellow with the Centro di Teoria dei Sistemi of the National Research Council (CNR) in Milan. Since July 1987, he has joined the Politecnico di Milano, where he is presently Full Professor of Automatica at the Dipartimento di Elettronica e Informazione. In the period 2000–2002 he served as vice-chairman of the Department and he is currently the Coordinator of the Automatica division. He is a member of IEEE and a member of the IFAC Technical Committee on Optimal Control. His current research interests include adaptive identification and control, robust filtering, automatic guidance of articulated vehicles, and control of switched systems. On these and other related topics he has authored or coauthored some 60 journal papers.

Patrizio Colaneri was born in Palmoli, Italy, on October 12, 1956. He received the doctor degree (Laurea) in electrical engineering in 1981 from the Politecnico di Milano, Milan, Italy, and the Ph.D. degree (Dottorato di Ricerca) in Automatic Control in 1987 from the Ministero della Pubblica Istruzione of Italy. From 1982 to 1984 he worked in Industry on simulation and control of electrical power plants. From 1984 to 1992, he was with the Centro di Teoria dei Sistemi of the Italian National Research Council (CNR). He held visiting positions at the Systems Research Center of the University of Maryland and the Johannes Kepler University in Linz. He is currently professor of Automatica at the Faculty of Engineering of the Politecnico di Milano. Prof. Colaneri was a YAP (Young Author Prize) finalist at the 1990 IFAC World Congress, Tallin (USSR). He is a member of SIAM, the chair of the IFAC technical committee on Control Design, a senior member of the IEEE and a member of the EUCA council. He served for six years as an Associate Editor of the IFAC journal Automatica. His main interests are in the area of periodic systems and control, robust filtering and control, digital and multirate control. On these arguments he authored or coauthored about 150 papers, two books in italian and the book Control Systems Design, an ${RH}_{2}$ and ${RH}_{\infty}$ viewpoint (published by Academic Press in 1997).

Giuseppe De Nicolao was born in Padova, Italy, in 1962. He received the degree in Electronic Engineering (“Laurea”) in 1986 from the Politecnico of Milano, Italy. From 1987 to 1988 he was with the Biomathematics and Biostatistics Unit of the Institute of Pharmacological Researches “Mario Negri”, Milano. In 1988 he joined the Italian National Research Council (C.N.R.) as a researcher of the Centro di Teoria dei Sistemi in Milano. From 1992 to 2000 he was Associate Professor and, since 2000, Full Professor of Model Identification in the Dipartimento di Informatica e Sistemistica of the Università di Pavia (Italy). In 1991, he held a visiting fellowship at the Department of Systems Engineering of the Australian National University, Canberra (Australia). In 1998 he was a plenary speaker at the workshop on “Nonlinear model predictive control: Assessment and future directions for research”, Ascona, Switzerland. He is a Senior Member of the IEEE, and, from 1999 to 2001, he has been Associate Editor of the IEEE Transactions on Automatic Control. His research interests include model predictive control, optimal and robust filtering and control, Bayesian learning, neural networks, deconvolution techniques, modeling and identification of biomedical systems, statistical process control and fault diagnosis for semiconductor manufacturing. On these subjects he has authored or coauthored 80 journal papers and is coinventor of two patents.

^☆: A preliminary version of this paper was presented at the 16th IFAC World Congress, Prague, 2005. This paper was recommended for publication in revised form by Associate Editor Mario Sznaier under the direction of Editor Roberto Tempo.

View full text

Brief paperOn almost sure stability of continuous-time Markov jump linear systems☆

Abstract

Introduction

Section snippets

Continuous-time MJLS—preliminaries

Main result

Randomized algorithm for assessment of almost sure stability

Example

Concluding remarks

Acknowledgments

Markov chains: Gibbs fields, Monte Carlo simulation, and queues

Discrete-time Markovian-jump quadratic optimal control

International Journal of Control

Discrete-time Markov jump linear systems

Feedback systems: Input–output properties

A new general sufficient condition for almost sure stability of jump linear systems

IEEE Transactions on Automatic Control

Brief paper
On almost sure stability of continuous-time Markov jump linear systems☆