Brief paperReliable approximations of probability-constrained stochastic linear-quadratic control☆
Introduction
In this paper we consider a probability-constrained discrete-time stochastic linear-quadratic (LQ) control problem with additive, zero mean, and finite second-moment disturbances. Unconstrained discrete-time stochastic LQ control has been extensively studied over the last half century, and it is well known that there exists a closed-form optimal solution which can be expressed in terms of the discrete-time algebraic Riccati equations, provided the system is controllable and observable (see, for example, Bertsekas, 2007). However, constrained problems present unique challenges that are generally not addressed by classical methods.
Recently, probability-constrained linear-quadratic control has been a very active research area in the control community (e.g., see Bernardini & Bemporad, 2009, Couchman, Cannon, & Kouvaritakis, 2006, Schwarm & Nikolaou, 1999 and Shin & Primbs, 2010), as it is a natural extension of constrained deterministic linear quadratic control. Probability-constrained optimization problems were first studied by Charnes, Cooper, and Symonds (1958), Miller and Wagner (1965), and Prekopa (1970). Hard, robust constraints (see Ben-Tal & Nemirovski, 1998) can be viewed as probability constraints that must hold with probability 1. Interestingly, sometimes a small relaxation of this probability requirement can lead to a significant improvement in the achievable objective function value. Currently there are two mainstream approaches dealing with probability constraints: probabilistic approximation (see Ben-tal & Nemirovski, 2000, Cogill & Zhou, in press, Nemirovski, 2003, Nemirovski and Shapiro, 2006) and sampling (see Calafiore & Campi, 2006, Campi & Garatti, 2008, Nemirovski & Shapiro, 2005). However, many existing results primarily deal with scalar cases. A special case is addressed in Oldewurtel, Jones, and Morari (2008), where the additive disturbance consists of independent identically distributed normal random variables. To the authors’ knowledge, relatively few practical methods exist for general multi-dimensional probability constraints.
In deterministic linear-quadratic regulator (LQR) problems, open-loop and closed-loop strategies are equivalent. So, in finite horizon problems, an optimal control sequence can be computed by solving a constrained quadratic program, even when inequality constraints on states and controls are present. In a stochastic setting, performance can be improved by utilizing a closed-loop controller. Generally a closed-loop strategy can use either state feedback or disturbance feedback. That is, past knowledge can be incorporated into the control actions taken at each time period. Here, we consider the design of closed-loop controllers that minimize an LQR cost subject to a probability constraint on the system’s states.
The main purpose of this paper is to develop tractable approximations of the probability-constrained stochastic LQR problem, where the class of constraints considered is linear with respect to the system state. To develop a tractable approximation, we will use a Chebyshev bound to approximate the joint probabilistic constraint. We then present a recursive algorithm which can efficiently deal with a special class of state-constrained problems using state feedback. The algorithm is inherently guaranteed to produce a feasible solution respecting the probabilistic constraints. The algorithm is tested in a numerical experiment and the results are compared to existing approaches.
Section snippets
Problem formulation
In this paper we consider a probabilistically constrained version of the classical finite-horizon stochastic LQR problem. We aim to control a linear system with stochastic dynamics given by for . In period , is the system state, is the control input, and is the disturbance input. The initial state is a known, given vector. The disturbance inputs are independent, have zero mean, and have a known covariance matrix . Our goal is to minimize the
Approximation of the joint probabilistic constraint
In this section we present an approximation for the probabilistic constraint We will approximate this by a constraint based on the multi-dimensional Chebyshev inequality. As we will show, the Chebyshev-based constraint can be handled naturally within the framework of stochastic LQR. The Chebyshev constraint is an inner approximation, in the sense that control laws that satisfy this constraint are guaranteed to satisfy the original probabilistic constraint in (P0).
There exist
State-separable approximation
In this section we study a special case in which the state probability constraint is approximated by a state-separable Chebyshev bound, and show how that property can be used in an algorithm for computing a control law. In particular, it turns out that a state-separable approximation can be solved very effectively by a recursive algorithm, as we will show soon. First, we will define the state-separable Chebyshev bound of a state probability constraint. Definition 2 If an inscribed ellipsoid of
Numerical examples
In this section we demonstrate the recursive algorithm on a state-constrained LQR problem with two states and two control inputs. The example is chosen in a way that the maximum volume ellipsoid gives state-separable Chebyshev bounds, so that we can run all algorithms on the same example. We compare our approaches with the certainty equivalent (CE) approach, which replaces random variables with their expected values as well as the scenario-based approach from Campi and Garatti (2008). All the
Conclusion
In this paper, we have proposed a method to approximate probabilistic constraints in stochastic control problems using the multi-dimensional Chebyshev bound and the maximum volume inscribed ellipsoid. We studied the special property given by a state-separable Chebyshev bound and connected state-separable approximations of (P0) to classical unconstrained LQR and summarized an efficient recursive algorithm. The approach we propose can readily be used as a subroutine in a model predictive control
Zhou Zhou received his B.S. and M.S. degrees in 2005 and 2008, respectively, from the Control Science and Engineering Department at Zhejiang University, China. From 2006 to 2008 he was a research assistant in the State Laboratory of Industrial Control Technology of China. He joined the Department of Systems and Information Engineering at the University of Virginia in 2008 and was a research assistant in Prof. Randy Cogill’s research group. His research interests include stochastic control and
References (21)
- et al.
Robust convex optimization
Mathematics of Operations Research
(1998) - et al.
Robust solutions of linear programming problems contaminated with uncertain data
Mathematical Programming
(2000) - Bernardini, D., & Bemporad, A. (2009). Scenario-based model predictive control of stochastic constrained linear...
Dynamic programming and optimal control, vol. II
(2007)- et al.
Convex optimization
(2004) - et al.
The scenario approach to robust control design
IEEE Transactions on Automatic Control
(2006) - et al.
Robust model predictive control via scenario optimization
IEEE Transactions on Automatic Control
(2013) - et al.
The exact feasibility of randomized solutions of uncertain convex programs
SIAM Journal on Optimization
(2008) - et al.
Cost horizons and certainty equivalents: an approach to stochastic programming of heating oil
Management Science
(1958) - Cogill, R., & Zhou, Z. (2011). A Chernoff bound approximation for risk-averse integer programming. In Proceedings of...
Cited by (0)
Zhou Zhou received his B.S. and M.S. degrees in 2005 and 2008, respectively, from the Control Science and Engineering Department at Zhejiang University, China. From 2006 to 2008 he was a research assistant in the State Laboratory of Industrial Control Technology of China. He joined the Department of Systems and Information Engineering at the University of Virginia in 2008 and was a research assistant in Prof. Randy Cogill’s research group. His research interests include stochastic control and optimization algorithms and their applications. He completed his Ph.D. at the University of Virginia with the topic “Stochastic LQ Control with Probabilistic Constraints” in 2012.
Randy Cogill is an Assistant Professor in the department of Systems and Information Engineering at the University of Virginia. Prior to joining the University of Virginia, he was a Ph.D. candidate in the department of Electrical Engineering at Stanford University, earning his Ph.D. in June 2007. His research interests fall into the broad categories of control, optimization, networks, and statistical inference.
- ☆
The material in this paper was partially presented at the 2011 American Control Conference (ACC2011), June 29–July 1, 2011, San Francisco, California, USA. This paper was recommended for publication in revised form by Associate Editor Fabrizio Dabbene under the direction of Editor Roberto Tempo.
- 1
Tel.: +1 650 497 7270; fax: +1 434 249 2643.