Stochastic algorithms for robustness of control performances☆
Introduction
The modern approach to performance analysis of control systems is mainly focused on deterministic methods (Bhattacharyya et al., 1995, Chen et al., 2007, Milanese and Tempo, 1985, Zinober, 1990). The main drawback of this approach, however, lies in computational complexity limitations, especially for controller analysis and synthesis in problems perceived to be “difficult” (in the sense of being NP-hard) (Blondel and Tsitsiklis, 2000, Nemirovskii, 1993). As models become more complex, an exhausting search through the solution space gets rapidly out of hand, and can be unworkable for real world systems (Vidyasagar, 1998). The second reason for intractability is dimensionality: even though some results might be achieved for low-order systems, problems become intractably difficult as the dimension of the system increases (Papadimitriou & Tsitsiklis, 1986). These issues forced the research community to seek ways around the “complexity-theoretic barriers” to overcome these difficulties.
As a result, the study of probabilistic methods has recently received a growing attention (Rust, 1997). Indeed, probability is central in the theory of choices under uncertainty (e.g. game theory). Moreover, probabilistic and randomized algorithms, built on standard Monte Carlo approaches, provide complementary methodology for studying robustness (Calafiore et al., 2003, Ray and Stengel, 1993), control synthesis (Ishii et al., 2005, Koltchinskii et al., 2000, Vidyasagar, 1998), and control design (Vidyasagar, 1998). Even if random techniques are willingly used for performance verification in control systems (Calafiore, Dabbene, & Tempo, 2007), nevertheless very often they may serve to describe just idealized problems. In particular, since random algorithms treat just the static case of stochastic processes (static random variables), they are not applicable to control systems with time-varying uncertainties (e.g. external noise, internal parameters influencing plants and controllers).
This paper offers stochastic methods, which refer to the dynamic case of stochastic processes (random variables dependent upon time). Our aim is to show that stochastic processes (e.g. Brownian motions) perfectly model realistic control systems. This will allow us to treat the problem by means of more advanced and innovative mathematical tools, such as stochastic differential equations (Oksendal, 1985, Yong and Zhou, 1999). However, as there exists a number of papers dealing with applications of stochastic methods in various scientific domains, there are only few authors, who have brought up the proposal of applying stochastic approach to robust control. Ravichandran (1991) presents various stochastic models that are used in the reliability analysis of redundant repairable systems. Marti (2000) shows that, besides the computation of robust optimal decision, stochastic methods may determine the solution of optimal control problems under real-time conditions (e.g. the optimal control of robots). The problem of constructing a robust system, according to an averaged performance criterion, of a stochastic control system was undertaken by Fetiso (2004). The paper (Brockett, 2000) addresses a number of stochastic optimal control problems in the case of quantized feedback, with performance criteria formulated in terms of the steady-state probability density. In Rico-Ramirez and Diwekar (2004) the stochastic maximum principle for optimal control under uncertainty has been presented. An important type of noises/disturbances described by Brownian motions has been addressed for filtering problems by Zhang, Chen, and Tseng (2005). In Digailova, Kurzhanski, and Varaiya (2008) reachability problems are considered for the case of stochastic Brownian noise. Since there are only few papers dealing with stochastic methods in robust control, the subject is relatively new. Our work presents a stochastic approach to robustness analysis of control systems.
To fix the ideas, we use the platform design approach of Agostini, Balluchi, Bicchi, Piccoli, Sangiovanni-Vincentelli, and Zadarnowska (2005). Thus, for a given control system a set of off-shelf feedbacks is used to match some performance criterion. The system includes parameters (both influencing the dynamics or the feedback), which may evolve stochastically. In this framework, the main outcome of this paper is the following: stochastic algorithms give rise to realistic probability distribution functions of the performance indices. In particular the distribution functions are less scattered than those produced by randomized algorithms.
We first develop some necessary theoretical results to deal with stochastic differential equations, both for the state of the system and the parameter evolutions. In particular, we want to keep the main features of Brownian motions, that take values in bounded sets. For definitiveness, we make a precise choice of the evolution for the stochastic process governing the parameters, however the results of the paper hold in the general case. Also, the simulation techniques can be adapted to any Ito diffusion. To implement simulations we need to produce samples of Brownian motion paths, as explained in Section 4 referring to existing literature on convergence analysis. Then, we provide natural definitions of robustness of performance indices in terms of both mathematical expectations and in probability for given bounds. The effectiveness of the method is proved by using Chernoff bounds (which were already used for the randomized case (Calafiore et al., 2003, Chernoff, 1952)). In particular, we show how to use Chernoff bounds in direct and indirect ways in order to obtain results both for mean and probability robustness.
Then we pass to deal with specific examples. In the first one we focus on a trajectory tracking problem for a Dubins car with feedback inaccuracies and external noise. The stochastic algorithm is tested and compared with a randomized one. Explicit values, both of the expected performance index and of the probability of staying below a fixed level, are given using the analysis developed starting from Chernoff bounds. The computation of probability distribution functions (briefly pdfs) permits to illustrate the effective differences between stochastic and randomized algorithms. Finally, we focus on a robotic manipulation example illustrating a technique to address parameter sensitivity. We use stochastic algorithms to determine the influence of each parameter on the system via a two step procedure.
The paper is organised as follows. Next subsection illustrates the main differences between results of random and stochastic simulations of Section 4. Section 2 formulates the problem. It explains the concepts of static and dynamic stochastic processes as well as introduces the context of stochastic differential equations. Section 3 deals with Chernoff bounds and robustness of the system with respect to given performance indices. Section 4 provides results of computer simulations, devoted to solving problems of robustness of some example control systems. The paper is concluded with Section 5.
Uncertainty can be modelled in many different ways, not bounded to random and stochastic approaches, but also including fuzzy analysis, differential inclusions, and others. However, stochastic calculus permits a mathematically satisfactory treatment of the problem from theoretical point of view, while Monte Carlo methods and other probabilistic techniques allow us to address complex problems with high dimensionality.
In Section 4 some simulations are run using both random and stochastic algorithms for a tracking problem using the toy model of a Dubins car. The obtained probability distribution functions are illustrated in Fig. 1, Fig. 2.
It is evident that the stochastic simulation approach generates much less scattered pdfs in comparison with the randomized ones. The case of stochastic simulation with random initial data is comparable to the randomized case, with even more scattered pdf. We can interpret this situation as follows. If the inaccuracies and noises are completely random at every instant of time (with independence assumptions) then the right approach to follow is the stochastic simulation. The effect on the correct estimation of performance indices, with respect to randomized algorithm, is quite important as Fig. 2 shows. In particular, the performance criterion can be met more easily, in other words the randomized algorithm overestimates the noises’ effects.
On the other side, we may have production faults in which the effect of noises is not random, but prescribes corresponding parameters a precise value. In the latter case, we can however expect a stochastic effect to be summed to the (not time-varying) random one. This is to say, that the approach of stochastic simulation with random initial data is to be considered as opposed to the randomized case. In this case, the opposite happens and the stochastic simulation gives more scattered pdfs than the randomized one (see Fig. 1). In particular, the performance criterion can be met less easily, in other words the randomized algorithm underestimates the noises’ effects.
Notation: and denote respectively, the -dimensional Euclidean space and the set of all real matrices. denotes the Euclidean norm in . is the space of integrable functions. Let be a filtered probability space equipped with nonempty event set , a -algebra , and probability measure . The filtration satisfies the usual conditions (i.e., the filtration contains all -null sets and is right continuous). stands for the mathematical expectation operator with respect to the given probability . indicates a standard Brownian motion. denotes white noise.
Section snippets
Stochastic equations
Let us consider a control system given in the form where , , indicates a vector of parameters, with performance index requirement defining acceptable states . Since our aim is to focus on robustness of a given feedback control with respect to (2) under assumption that parameters evolve stochastically, we may rewrite the control system (1) in the form of a stochastic differential equation (SDE)
Stochastic algorithms and Chernoff bounds
Our goal is then to provide estimates of the performance index for solutions to the system (3), (10). Let us first define two concepts of robustness: Definition 12 The system (3), (10) is robust at level in expectation, if . Definition 13 The system (3), (10) is robust at level with probability, if . Robustness
Robustness
Simulations
To implement stochastic simulations, we need to approximate Brownian motion trajectories. We describe a technique to generate a l-dimensional Brownian motion on , referring the reader to Karatzas and Shreve (1998) and Knight (1981) for general theory.
Denote by the normal distribution with mean value and variance . We can simulate Brownian motions using the following algorithm. Algorithm 3 Divide the interval into equidistant parts; Generate independent random numbers which are
Conclusions and future works
We have considered stochastic algorithms for control performances derived from a stochastic evolution of system parameters. This type of evolution, opposed to static random variables, permits to obtain more realistic probability distribution functions (pdfs) for the performance indices.
We first developed a theory to allowing us to study the stochastic evolution in bounded domains, that keeps the main features of the Brownian motion evolution. Then, we used Chernoff bounds to derive conclusions
Benedetto Piccoli is Research Director at IACCNR in Rome. His main research interests include traffic flow on networks, control, biomathematics and financial mathematics. He published more than 100 research papers and 3 books. He is Editor in Chief of the journal Networks and Heterogeneous Media.
References (30)
- et al.
A survey of computational complexity results in systems and control
Automatica
(2000) - et al.
A survey of randomized algorithms for control synthesis and performance verification
Journal of Complexity
(2007) - et al.
Robustness analysis of responses of vibration control structures with uncertain parameters using interval algorithm
Structural Safety
(2007) - et al.
Stochastic maximum principle for optimal control under uncertainty
Computers and Chemical Engineering
(2004) - et al.
A Monte Carlo approach to the analysis of control systems robustness
Automatica
(1993) - et al.
Filtering on nonlinear time-delay stochastic systems
Automatica
(2003) - et al.
filtering for uncertain stochastic time-delay systems with sector-bounded nonlinearities
Automatica
(2008) - Agostini, A., Balluchi, A., Bicchi, A., Piccoli, B., Sangiovanni-Vincentelli, A., & Zadarnowska, K. (2005). Randomized...
- et al.
Robust control: The parametric approach
(1995) Nonlinear Feedback Systems Perturbed by Noise: Steady-State Probability Distributions and Optimal Control
IEEE Transactions on Automatic Control
(2000)
A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations
Annals of Mathematical Statistics
Some problems of robust control of a stochastic object
Automation and Remote Control
Randomized algorithms for synthesis of switching rules for multimodal systems
IEEE Transactions on Automatic Control
Cited by (5)
Flight control for air-breathing hypersonic vehicles using linear quadratic regulator design based on stochastic robustness analysis
2017, Frontiers of Information Technology and Electronic EngineeringA probabilistic robust control method for a small UAV with bounded uncertainty
2016, Chinese Control Conference, CCCDesign of structure parameters for gasoline engine common rail system based on random algorithm
2015, Chinese Control Conference, CCCA new probabilistic robust control approach for system with uncertain parameters
2015, Asian Journal of ControlDesign of cascaded vehicle velocity observer based on randomized algorithms
2014, Jilin Daxue Xuebao (Gongxueban)/Journal of Jilin University (Engineering and Technology Edition)
Benedetto Piccoli is Research Director at IACCNR in Rome. His main research interests include traffic flow on networks, control, biomathematics and financial mathematics. He published more than 100 research papers and 3 books. He is Editor in Chief of the journal Networks and Heterogeneous Media.
Katarzyna Zadarnowska received the Ph.D. degree in Technical Sciences in 2005. She has been participated in several research projects, e.g. Endogenous configuration space approach to mobile manipulators (supported by Polish State Committee of Scientific Research) and Hybrid systems with applications to automotive control (supported by Marii Curie Training Site). Since 2005 she has been an Assistant Professor at the Institute of Computer Engineering, Control and Robotics, Wroclaw University of Technology, Poland. Her current interests include nonholonomic systems, hybrid systems, mobile robots, optimal control and stochastic algorithms.
Matteo Gaeta is an Associate Professor of Information Processing Systems at the Engineering Faculty of the University of Salerno. His research interests include Complex Information Systems Architecture, Software Engineering, Systems of Knowledge Representation, Semantic Web, Virtual Organization and Grid Computing. He gained a large experience in the Implementation and Design of Complex Information Systems. He is the author of more than 80 papers, published on journals, proceedings and books. He is the Scientific Coordinator and Manager of several International Research Projects. He is the Coordinator of the MIUR Working Group. He is a member of the Panel for the Scientific Assessment of Research and Testing Projects of the MiPAF. He is a Member of the Experts Register of the Department for Education and Skills and of the innovation technology Experts Register of the Department for the Economy Development.
- ☆
This work was partially supported by European projects CTS and HyCon. This paper was not presented at any IFAC meeting. This paper was recommended for publication in revised form by Associate Editor Fabrizio Dabbene under the direction of Editor Roberto Tempo.