Elsevier

Automatica

Volume 49, Issue 8, August 2013, Pages 2435-2439
Automatica

Brief paper
Reliable approximations of probability-constrained stochastic linear-quadratic control

https://doi.org/10.1016/j.automatica.2013.03.010Get rights and content

Abstract

Here we consider a state-constrained stochastic linear-quadratic control problem. This problem has linear dynamics and a quadratic cost, and states are required to satisfy a probabilistic constraint. In this paper, the joint probabilistic constraint in the model is converted to a conservative deterministic constraint using a multi-dimensional Chebyshev bound. A maximum volume inscribed ellipsoid problem is solved to obtain this probability bound. Using the probability bound, we develop a recursive state feedback control algorithm for a special class of state-constrained stochastic linear-quadratic regulator (LQR). The performance of this approach is explored in a numerical example.

Introduction

In this paper we consider a probability-constrained discrete-time stochastic linear-quadratic (LQ) control problem with additive, zero mean, and finite second-moment disturbances. Unconstrained discrete-time stochastic LQ control has been extensively studied over the last half century, and it is well known that there exists a closed-form optimal solution which can be expressed in terms of the discrete-time algebraic Riccati equations, provided the system is controllable and observable (see, for example, Bertsekas, 2007). However, constrained problems present unique challenges that are generally not addressed by classical methods.

Recently, probability-constrained linear-quadratic control has been a very active research area in the control community (e.g., see Bernardini & Bemporad, 2009, Couchman, Cannon, & Kouvaritakis, 2006, Schwarm & Nikolaou, 1999 and Shin & Primbs, 2010), as it is a natural extension of constrained deterministic linear quadratic control. Probability-constrained optimization problems were first studied by Charnes, Cooper, and Symonds (1958), Miller and Wagner (1965), and Prekopa (1970). Hard, robust constraints (see  Ben-Tal & Nemirovski, 1998) can be viewed as probability constraints that must hold with probability 1. Interestingly, sometimes a small relaxation of this probability requirement can lead to a significant improvement in the achievable objective function value. Currently there are two mainstream approaches dealing with probability constraints: probabilistic approximation (see Ben-tal & Nemirovski, 2000, Cogill & Zhou, in press, Nemirovski, 2003, Nemirovski and Shapiro, 2006) and sampling (see Calafiore & Campi, 2006, Campi & Garatti, 2008, Nemirovski & Shapiro, 2005). However, many existing results primarily deal with scalar cases. A special case is addressed in Oldewurtel, Jones, and Morari (2008), where the additive disturbance consists of independent identically distributed normal random variables. To the authors’ knowledge, relatively few practical methods exist for general multi-dimensional probability constraints.

In deterministic linear-quadratic regulator (LQR) problems, open-loop and closed-loop strategies are equivalent. So, in finite horizon problems, an optimal control sequence can be computed by solving a constrained quadratic program, even when inequality constraints on states and controls are present. In a stochastic setting, performance can be improved by utilizing a closed-loop controller. Generally a closed-loop strategy can use either state feedback or disturbance feedback. That is, past knowledge can be incorporated into the control actions taken at each time period. Here, we consider the design of closed-loop controllers that minimize an LQR cost subject to a probability constraint on the system’s states.

The main purpose of this paper is to develop tractable approximations of the probability-constrained stochastic LQR problem, where the class of constraints considered is linear with respect to the system state. To develop a tractable approximation, we will use a Chebyshev bound to approximate the joint probabilistic constraint. We then present a recursive algorithm which can efficiently deal with a special class of state-constrained problems using state feedback. The algorithm is inherently guaranteed to produce a feasible solution respecting the probabilistic constraints. The algorithm is tested in a numerical experiment and the results are compared to existing approaches.

Section snippets

Problem formulation

In this paper we consider a probabilistically constrained version of the classical finite-horizon stochastic LQR problem. We aim to control a linear system with stochastic dynamics given by xk+1=Axk+Buk+wk for k{0,,N1}. In period k, xkRn is the system state, ukRm is the control input, and wkRn is the disturbance input. The initial state x0 is a known, given vector. The disturbance inputs are independent, have zero mean, and have a known covariance matrix WSn. Our goal is to minimize the

Approximation of the joint probabilistic constraint

In this section we present an approximation for the probabilistic constraint P(k=0NTkxkb)α. We will approximate this by a constraint based on the multi-dimensional Chebyshev inequality. As we will show, the Chebyshev-based constraint can be handled naturally within the framework of stochastic LQR. The Chebyshev constraint is an inner approximation, in the sense that control laws that satisfy this constraint are guaranteed to satisfy the original probabilistic constraint in (P0).

There exist

State-separable approximation

In this section we study a special case in which the state probability constraint is approximated by a state-separable Chebyshev bound, and show how that property can be used in an algorithm for computing a control law. In particular, it turns out that a state-separable approximation can be solved very effectively by a recursive algorithm, as we will show soon. First, we will define the state-separable Chebyshev bound of a state probability constraint.

Definition 2

If an inscribed ellipsoid (P,q,r) of k=0NT

Numerical examples

In this section we demonstrate the recursive algorithm on a state-constrained LQR problem with two states and two control inputs. The example is chosen in a way that the maximum volume ellipsoid gives state-separable Chebyshev bounds, so that we can run all algorithms on the same example. We compare our approaches with the certainty equivalent (CE) approach, which replaces random variables with their expected values as well as the scenario-based approach from Campi and Garatti (2008). All the

Conclusion

In this paper, we have proposed a method to approximate probabilistic constraints in stochastic control problems using the multi-dimensional Chebyshev bound and the maximum volume inscribed ellipsoid. We studied the special property given by a state-separable Chebyshev bound and connected state-separable approximations of (P0) to classical unconstrained LQR and summarized an efficient recursive algorithm. The approach we propose can readily be used as a subroutine in a model predictive control

Zhou Zhou received his B.S. and M.S. degrees in 2005 and 2008, respectively, from the Control Science and Engineering Department at Zhejiang University, China. From 2006 to 2008 he was a research assistant in the State Laboratory of Industrial Control Technology of China. He joined the Department of Systems and Information Engineering at the University of Virginia in 2008 and was a research assistant in Prof. Randy Cogill’s research group. His research interests include stochastic control and

References (21)

  • A. Ben-Tal et al.

    Robust convex optimization

    Mathematics of Operations Research

    (1998)
  • A. Ben-tal et al.

    Robust solutions of linear programming problems contaminated with uncertain data

    Mathematical Programming

    (2000)
  • Bernardini, D., & Bemporad, A. (2009). Scenario-based model predictive control of stochastic constrained linear...
  • D. Bertsekas

    Dynamic programming and optimal control, vol. II

    (2007)
  • S. Boyd et al.

    Convex optimization

    (2004)
  • G. Calafiore et al.

    The scenario approach to robust control design

    IEEE Transactions on Automatic Control

    (2006)
  • G.C. Calafiore et al.

    Robust model predictive control via scenario optimization

    IEEE Transactions on Automatic Control

    (2013)
  • M.C. Campi et al.

    The exact feasibility of randomized solutions of uncertain convex programs

    SIAM Journal on Optimization

    (2008)
  • A. Charnes et al.

    Cost horizons and certainty equivalents: an approach to stochastic programming of heating oil

    Management Science

    (1958)
  • Cogill, R., & Zhou, Z. (2011). A Chernoff bound approximation for risk-averse integer programming. In Proceedings of...
There are more references available in the full text version of this article.

Cited by (0)

Zhou Zhou received his B.S. and M.S. degrees in 2005 and 2008, respectively, from the Control Science and Engineering Department at Zhejiang University, China. From 2006 to 2008 he was a research assistant in the State Laboratory of Industrial Control Technology of China. He joined the Department of Systems and Information Engineering at the University of Virginia in 2008 and was a research assistant in Prof. Randy Cogill’s research group. His research interests include stochastic control and optimization algorithms and their applications. He completed his Ph.D. at the University of Virginia with the topic “Stochastic LQ Control with Probabilistic Constraints” in 2012.

Randy Cogill is an Assistant Professor in the department of Systems and Information Engineering at the University of Virginia. Prior to joining the University of Virginia, he was a Ph.D. candidate in the department of Electrical Engineering at Stanford University, earning his Ph.D. in June 2007. His research interests fall into the broad categories of control, optimization, networks, and statistical inference.

The material in this paper was partially presented at the 2011 American Control Conference (ACC2011), June 29–July 1, 2011, San Francisco, California, USA. This paper was recommended for publication in revised form by Associate Editor Fabrizio Dabbene under the direction of Editor Roberto Tempo.

1

Tel.: +1 650 497 7270; fax: +1 434 249 2643.

View full text