Elsevier

Automatica

Volume 45, Issue 6, June 2009, Pages 1407-1414
Automatica

Stochastic algorithms for robustness of control performances

https://doi.org/10.1016/j.automatica.2009.02.018Get rights and content

Abstract

In recent years, there has been a growing interest in developing statistical learning methods to provide approximate solutions to “difficult” control problems. In particular, randomized algorithms have become a very popular tool used for stability and performance analysis as well as for design of control systems. However, as randomized algorithms provide an efficient solution procedure to the “intractable” problems, stochastic methods bring closer to understanding the properties of the real systems. The topic of this paper is the use of stochastic methods in order to solve the problem of control robustness: the case of parametric stochastic uncertainty is considered. Necessary concepts regarding stochastic control theory and stochastic differential equations are introduced. Then a convergence analysis is provided by means of the Chernoff bounds, which guarantees robustness in mean and in probability. As an illustration, the robustness of control performances of example control systems is computed.

Introduction

The modern approach to performance analysis of control systems is mainly focused on deterministic methods (Bhattacharyya et al., 1995, Chen et al., 2007, Milanese and Tempo, 1985, Zinober, 1990). The main drawback of this approach, however, lies in computational complexity limitations, especially for controller analysis and synthesis in problems perceived to be “difficult” (in the sense of being NP-hard) (Blondel and Tsitsiklis, 2000, Nemirovskii, 1993). As models become more complex, an exhausting search through the solution space gets rapidly out of hand, and can be unworkable for real world systems (Vidyasagar, 1998). The second reason for intractability is dimensionality: even though some results might be achieved for low-order systems, problems become intractably difficult as the dimension of the system increases (Papadimitriou & Tsitsiklis, 1986). These issues forced the research community to seek ways around the “complexity-theoretic barriers” to overcome these difficulties.

As a result, the study of probabilistic methods has recently received a growing attention (Rust, 1997). Indeed, probability is central in the theory of choices under uncertainty (e.g. game theory). Moreover, probabilistic and randomized algorithms, built on standard Monte Carlo approaches, provide complementary methodology for studying robustness (Calafiore et al., 2003, Ray and Stengel, 1993), control synthesis (Ishii et al., 2005, Koltchinskii et al., 2000, Vidyasagar, 1998), and control design (Vidyasagar, 1998). Even if random techniques are willingly used for performance verification in control systems (Calafiore, Dabbene, & Tempo, 2007), nevertheless very often they may serve to describe just idealized problems. In particular, since random algorithms treat just the static case of stochastic processes (static random variables), they are not applicable to control systems with time-varying uncertainties (e.g. external noise, internal parameters influencing plants and controllers).

This paper offers stochastic methods, which refer to the dynamic case of stochastic processes (random variables dependent upon time). Our aim is to show that stochastic processes (e.g. Brownian motions) perfectly model realistic control systems. This will allow us to treat the problem by means of more advanced and innovative mathematical tools, such as stochastic differential equations (Oksendal, 1985, Yong and Zhou, 1999). However, as there exists a number of papers dealing with applications of stochastic methods in various scientific domains, there are only few authors, who have brought up the proposal of applying stochastic approach to robust control. Ravichandran (1991) presents various stochastic models that are used in the reliability analysis of redundant repairable systems. Marti (2000) shows that, besides the computation of robust optimal decision, stochastic methods may determine the solution of optimal control problems under real-time conditions (e.g. the optimal control of robots). The problem of constructing a robust system, according to an averaged performance criterion, of a stochastic control system was undertaken by Fetiso (2004). The paper (Brockett, 2000) addresses a number of stochastic optimal control problems in the case of quantized feedback, with performance criteria formulated in terms of the steady-state probability density. In Rico-Ramirez and Diwekar (2004) the stochastic maximum principle for optimal control under uncertainty has been presented. An important type of noises/disturbances described by Brownian motions has been addressed for filtering problems by Zhang, Chen, and Tseng (2005). In Digailova, Kurzhanski, and Varaiya (2008) reachability problems are considered for the case of stochastic Brownian noise. Since there are only few papers dealing with stochastic methods in robust control, the subject is relatively new. Our work presents a stochastic approach to robustness analysis of control systems.

To fix the ideas, we use the platform design approach of Agostini, Balluchi, Bicchi, Piccoli, Sangiovanni-Vincentelli, and Zadarnowska (2005). Thus, for a given control system a set of off-shelf feedbacks is used to match some performance criterion. The system includes parameters (both influencing the dynamics or the feedback), which may evolve stochastically. In this framework, the main outcome of this paper is the following: stochastic algorithms give rise to realistic probability distribution functions of the performance indices. In particular the distribution functions are less scattered than those produced by randomized algorithms.

We first develop some necessary theoretical results to deal with stochastic differential equations, both for the state of the system and the parameter evolutions. In particular, we want to keep the main features of Brownian motions, that take values in bounded sets. For definitiveness, we make a precise choice of the evolution for the stochastic process governing the parameters, however the results of the paper hold in the general case. Also, the simulation techniques can be adapted to any Ito diffusion. To implement simulations we need to produce samples of Brownian motion paths, as explained in Section 4 referring to existing literature on convergence analysis. Then, we provide natural definitions of robustness of performance indices in terms of both mathematical expectations and in probability for given bounds. The effectiveness of the method is proved by using Chernoff bounds (which were already used for the randomized case (Calafiore et al., 2003, Chernoff, 1952)). In particular, we show how to use Chernoff bounds in direct and indirect ways in order to obtain results both for mean and probability robustness.

Then we pass to deal with specific examples. In the first one we focus on a trajectory tracking problem for a Dubins car with feedback inaccuracies and external noise. The stochastic algorithm is tested and compared with a randomized one. Explicit values, both of the expected performance index and of the probability of staying below a fixed level, are given using the analysis developed starting from Chernoff bounds. The computation of probability distribution functions (briefly pdfs) permits to illustrate the effective differences between stochastic and randomized algorithms. Finally, we focus on a robotic manipulation example illustrating a technique to address parameter sensitivity. We use stochastic algorithms to determine the influence of each parameter on the system via a two step procedure.

The paper is organised as follows. Next subsection illustrates the main differences between results of random and stochastic simulations of Section 4. Section 2 formulates the problem. It explains the concepts of static and dynamic stochastic processes as well as introduces the context of stochastic differential equations. Section 3 deals with Chernoff bounds and robustness of the system with respect to given performance indices. Section 4 provides results of computer simulations, devoted to solving problems of robustness of some example control systems. The paper is concluded with Section 5.

Uncertainty can be modelled in many different ways, not bounded to random and stochastic approaches, but also including fuzzy analysis, differential inclusions, and others. However, stochastic calculus permits a mathematically satisfactory treatment of the problem from theoretical point of view, while Monte Carlo methods and other probabilistic techniques allow us to address complex problems with high dimensionality.

In Section 4 some simulations are run using both random and stochastic algorithms for a tracking problem using the toy model of a Dubins car. The obtained probability distribution functions are illustrated in Fig. 1, Fig. 2.

It is evident that the stochastic simulation approach generates much less scattered pdfs in comparison with the randomized ones. The case of stochastic simulation with random initial data is comparable to the randomized case, with even more scattered pdf. We can interpret this situation as follows. If the inaccuracies and noises are completely random at every instant of time (with independence assumptions) then the right approach to follow is the stochastic simulation. The effect on the correct estimation of performance indices, with respect to randomized algorithm, is quite important as Fig. 2 shows. In particular, the performance criterion can be met more easily, in other words the randomized algorithm overestimates the noises’ effects.

On the other side, we may have production faults in which the effect of noises is not random, but prescribes corresponding parameters a precise value. In the latter case, we can however expect a stochastic effect to be summed to the (not time-varying) random one. This is to say, that the approach of stochastic simulation with random initial data is to be considered as opposed to the randomized case. In this case, the opposite happens and the stochastic simulation gives more scattered pdfs than the randomized one (see Fig. 1). In particular, the performance criterion can be met less easily, in other words the randomized algorithm underestimates the noises’ effects.

Notation:Rn and Rn×m denote respectively, the n-dimensional Euclidean space and the set of all n×m real matrices. || denotes the Euclidean norm in Rn. L1 is the space of integrable functions. Let (Ω,F,{Ft}t0,P) be a filtered probability space equipped with nonempty event set Ω, a σ-algebra F, and probability measure P. The filtration {Ft}t0 satisfies the usual conditions (i.e., the filtration contains all P-null sets and is right continuous). E{} stands for the mathematical expectation operator with respect to the given probability P. W(t):[0,T]×ΩRl indicates a standard {Ft}t0 Brownian motion. dW(t) denotes white noise.

Section snippets

Stochastic equations

Let us consider a control system given in the form {dx=f(t,x,u,p)dt;x(0)=x0, where xRn, uRm, pPRs indicates a vector of parameters, with performance index requirement J(x)ε, defining acceptable states x. Since our aim is to focus on robustness of a given feedback control u(x) with respect to (2) under assumption that parameters evolve stochastically, we may rewrite the control system (1) in the form of a stochastic differential equation (SDE) {dX(t)=f(t,X,u(X),p)dt+σ(t,X,u(X),p)dW(t)X(0)=ξ.

Stochastic algorithms and Chernoff bounds

Our goal is then to provide estimates of the performance index for solutions to the system (3), (10). Let us first define two concepts of robustness:

Definition 12

Eε Robustness

The system (3), (10) is robust at ε level in expectation, if E[J(X,p)]ε.

Definition 13

Pεp Robustness

The system (3), (10) is robust at ε level with p(0,1) probability, if P(J(X,p)ε)p.

One concept may be preferable over the other, depending on the characteristic of the performance index. For instance, if a safety issue is considered, the second concept should be adopted

Simulations

To implement stochastic simulations, we need to approximate Brownian motion trajectories. We describe a technique to generate a l-dimensional Brownian motion on [0,T], referring the reader to Karatzas and Shreve (1998) and Knight (1981) for general theory.

Denote by N(μ,η2) the normal distribution with mean value μ and variance η2. We can simulate Brownian motions using the following algorithm.

Algorithm 3

  • (1)

    Divide the interval [0,T] into N equidistant parts;

  • (2)

    Generate N independent random numbers Yj which are

Conclusions and future works

We have considered stochastic algorithms for control performances derived from a stochastic evolution of system parameters. This type of evolution, opposed to static random variables, permits to obtain more realistic probability distribution functions (pdfs) for the performance indices.

We first developed a theory to allowing us to study the stochastic evolution in bounded domains, that keeps the main features of the Brownian motion evolution. Then, we used Chernoff bounds to derive conclusions

Benedetto Piccoli is Research Director at IACCNR in Rome. His main research interests include traffic flow on networks, control, biomathematics and financial mathematics. He published more than 100 research papers and 3 books. He is Editor in Chief of the journal Networks and Heterogeneous Media.

References (30)

  • Calafiore, G., Dabbene, F., & Tempo, R. (2003). Randomized algorithms in robust control. In Proc. 42nd IEEE conf....
  • H. Chernoff

    A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations

    Annals of Mathematical Statistics

    (1952)
  • Digailova, I. A., Kurzhanski, A. B., & Varaiya, P. (2008). Stochastic reachability and measurement feedback under...
  • V.N. Fetiso

    Some problems of robust control of a stochastic object

    Automation and Remote Control

    (2004)
  • H. Ishii et al.

    Randomized algorithms for synthesis of switching rules for multimodal systems

    IEEE Transactions on Automatic Control

    (2005)
  • Benedetto Piccoli is Research Director at IACCNR in Rome. His main research interests include traffic flow on networks, control, biomathematics and financial mathematics. He published more than 100 research papers and 3 books. He is Editor in Chief of the journal Networks and Heterogeneous Media.

    Katarzyna Zadarnowska received the Ph.D. degree in Technical Sciences in 2005. She has been participated in several research projects, e.g. Endogenous configuration space approach to mobile manipulators (supported by Polish State Committee of Scientific Research) and Hybrid systems with applications to automotive control (supported by Marii Curie Training Site). Since 2005 she has been an Assistant Professor at the Institute of Computer Engineering, Control and Robotics, Wroclaw University of Technology, Poland. Her current interests include nonholonomic systems, hybrid systems, mobile robots, optimal control and stochastic algorithms.

    Matteo Gaeta is an Associate Professor of Information Processing Systems at the Engineering Faculty of the University of Salerno. His research interests include Complex Information Systems Architecture, Software Engineering, Systems of Knowledge Representation, Semantic Web, Virtual Organization and Grid Computing. He gained a large experience in the Implementation and Design of Complex Information Systems. He is the author of more than 80 papers, published on journals, proceedings and books. He is the Scientific Coordinator and Manager of several International Research Projects. He is the Coordinator of the MIUR Working Group. He is a member of the Panel for the Scientific Assessment of Research and Testing Projects of the MiPAF. He is a Member of the Experts Register of the Department for Education and Skills and of the innovation technology Experts Register of the Department for the Economy Development.

    This work was partially supported by European projects CTS and HyCon. This paper was not presented at any IFAC meeting. This paper was recommended for publication in revised form by Associate Editor Fabrizio Dabbene under the direction of Editor Roberto Tempo.

    View full text