An alternative use of the Riccati recursion for efficient optimization

Abstract

In optimization routines used for on-line Model Predictive Control (MPC), linear systems of equations are solved in each iteration. This is true both for Active Set (AS) solvers as well as for Interior Point (IP) solvers, and for linear MPC as well as for nonlinear MPC and hybrid MPC. The main computational effort is spent while solving these linear systems of equations, and hence, it is of great interest to solve them efficiently. In high performance solvers for MPC, this is performed using Riccati recursions or generic sparsity exploiting algorithms. To be able to get this performance gain, the problem has to be formulated in a sparse way which introduces more variables. The alternative is to use a smaller formulation where the objective function Hessian is dense. In this work, it is shown that it is possible to exploit the structure also when using the dense formulation. More specifically, it is shown that it is possible to efficiently compute a standard Cholesky factorization for the dense formulation. This results in a computational complexity that grows quadratically in the prediction horizon length instead of cubically as for the generic Cholesky factorization.

Introduction

Model Predictive Control (MPC) is one of the most commonly used control strategies in industry. Some important reasons for its success include that it can handle multi-variable systems and constraints on control signals and state variables in a structured way. In each sample, some kind of optimization problem is solved. In the methods considered in this paper, the optimization problem is assumed to be solved on-line. The optimization problem can be of different types depending on which type of system and problem formulation that is used. The most common variants are linear MPC, non-linear MPC and hybrid MPC. In most cases, the effort spent in the optimization problems boils down to solving Newton-system-like equations. Hence, a great deal of research has been done in the area of solving this type of system of equations efficiently when it has the special form from MPC. It is well-known that these equations (or at least a large part of them) can be cast in the form of a finite horizon LQ control problem and as such it can be solved using a Riccati recursion. Some examples of how Riccati recursions have been used to speed up optimization routines can be found in, for example, [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11].

The main purpose of this paper is twofold. First, this paper introduces a new factorization of the KKT coefficient matrix of an MPC problem given in dense form (states eliminated). It is shown that this factorization is closely related to the Cholesky factorization of the coefficient matrix, when the order of the optimization variables is reversed. Second, using this new factorization, a standard Cholesky factorization can be efficiently computed. This makes it possible to develop a drop-in replacement for a generic Cholesky factorization in, for example, standard range-space or null-space Active Set (AS) methods. Compared to the generic factorization, computational performance can be significantly improved for long prediction horizons. The benefit with the proposed factorization compared to standard Riccati based methods is that it requires only small changes to the solver which it is plugged into since the result from the algorithm is a standard Cholesky factorization. This factorization can, as the generic Cholesky factorization, be reused to solve several systems of equations with different right hand sides, and it can be updated using standard methods.

In this article, ${\mathbb{S}}^n$ denotes symmetric matrices with $n$ columns. Furthermore, ${\mathbb{S}}_{++}^n$ (${\mathbb{S}}_{+}^n$) denotes symmetric positive (semi) definite matrices with $n$ columns. The matrix $\bar{I}$ denotes the exchange matrix, which is a matrix with ones in the counter-diagonal and zeros in other elements. Note that $\bar{I}M\bar{I}$ for a matrix $M$ is nothing but $M$ with the rows and columns in reversed order, which can be performed using zero flops. A Sans Serif font is used to indicate that a matrix or a vector is, in some way, composed of stacked matrices or vectors from different time instants. The stacked matrices or vectors have a similar symbol as the composed matrix, but in an ordinary font. For example, $Q_u=\mathrm{diag}(Q_u,\dots ,Q_u)$.