Stationary behavior of an anti-windup scheme for recursive parameter estimation under lack of excitation

Abstract

Stationary properties of a recently suggested windup prevention scheme for recursive parameter estimation are investigated in the case of insufficient excitation. When the regressor vector contains data covering the whole parameter space, the algorithm has only one stationary point, the one defined by a weighting matrix. If the excitation is insufficient, the algorithm is shown to possess a manifold of stationary points and a complete parametrization of this manifold is given. However, if the past excitation conditions already caused the algorithm to converge to a certain point, the stationary solution would not be affected by current lack of excitation. This property guarantees good anti-windup properties of the studied parameter estimation algorithm.

Introduction

Consider the following regressor model

$$y(t)={\varphi }^{\mathrm{T}}(t)\theta +e(t)\text{,}$$

where $y(t)$ is the scalar output measured at discrete time instances $t=[0,\infty )$, $\varphi \in R^n$ is the regressor vector, $\theta \in R^n$ is the parameter vector to be estimated and the scalar e is disturbance.

The estimation of $\theta$ is often performed by a linear recursive algorithm of the “prediction–correction” form

$$\widehat{\theta }(t)=\widehat{\theta }(t-1)+K(t)(y(t)-{\varphi }^{\mathrm{T}}(t)\widehat{\theta }(t-1))\text{,}$$

where the first (prediction) term in the right-hand part of the equation highlights the fact that the parameter vector is assumed to be constant.

If $e(t)$ is white and the parameter vector is subject to the random walk model driven by a zero-mean white noise sequence $w(t)$

$$\theta (t)=\theta (t-1)+w(t)$$

the optimal, in the sense of minimum of the a posteriori parameter error covariance matrix, estimate is yielded by (2) with the Kalman gain

$$K(t)=P(t)\varphi (t)\text{,}$$

where $P(t)$, $t=[1,\infty )$ is the solution to the Riccati equation

$$P(t)=P(t-1)-\frac{P(t-1)\varphi (t){\varphi }^{\mathrm{T}}(t)P(t-1)}{r(t)+{\varphi }^{\mathrm{T}}(t)P(t-1)\varphi (t)}+Q(t)$$

for some $P(0)=P^{\mathrm{T}}(0),P(0)⩾0$ describing the covariance of the initial guess $\widehat{\theta }(0)$. Optimality of the estimate is guaranteed only when

$$Q(t)=\mathrm{cov}w(t),r(t)=\mathrm{var}e(t)$$

see Ljung and Gunnarsson (1990). Since these quantities are seldom a priori known, they are usually treated as design parameters of the estimation algorithm and chosen as some $Q(·)\in R^{n\times n},Q(·)⩾0,r(·)>0$ in order to achieve desired properties of the filter.

The regressor vector $\varphi (t)$ is called persistently exciting (Söderström & Stoica, 1989), if there exist a constant $0<c<\infty$ and an integer $m>0$ such that for all t

$$\sum _{k=t}^{t+m-1}\varphi (k){\varphi }^{\mathrm{T}}(k)⩾\mathit{cI}\text{.}$$

Thus, when $\varphi (t)$ is persistently exciting, the space $R^n$ is spanned by $\varphi (t)$ in at most m steps.

Excitation properties of the regressor vector sequence play an important role in the dynamic behavior of (4). When the excitation in the input data is non-persistent, a phenomenon referred to as (covariance) windup (a.k.a. blow-up) can occur. This means that some eigenvalues of P rise linearly with time.

As pointed out in Cao and Schwartz (2004), the windup phenomenon in the Kalman filter has not been much analyzed until recently. Therefore most of the suggested anti-windup schemes for Kalman filter parameter estimation are of ad hoc nature and are lacking strict proof of non-divergence under lack of excitation, see e.g. Hägglund (1983) and Bittanti, Bolzern, and Campi (1990).

In the approach taken in Stenlund and Gustafsson (2002), which is in the sequel referred to as the Stenlund–Gustafsson (SG) algorithm, a special choice of $Q(t)$ is used to control the convergence point of the P-matrix

$$Q(t)=\frac{P_{\mathrm{d}}\varphi (t){\varphi }^{\mathrm{T}}(t)P_{\mathrm{d}}}{r(t)+{\varphi }^{\mathrm{T}}(t)P_{\mathrm{d}}\varphi (t)}\text{,}$$

where $P_{\mathrm{d}}\in R^{n\times n},P_{\mathrm{d}}>0$. Consequently, the optimality of the Kalman filter estimate is lost.

The structure of (7) is designed to update $P(t)$ only in the subspace where excitation is present, that is the image space $\text{Im}Q=\text{Im}\varphi {\varphi }^{\mathrm{T}}$. Addition of $r(t)>0$ in the denominator of (7) prevents division by zero in case $\varphi (t)=0$ for some t.

A formal proof of the fact that (4), with the free term chosen according to (7), is non-diverging even for the case of lack of excitation, can be found in Medvedev, 2003, Medvedev, 2004. However, stationary properties of the scheme are not considered there.

The SG algorithm can be seen as a generalization of the normalized least mean squares (N-LMS), a method that is well-known and widely used in engineering practice. In Ljung and Gunnarsson (1990), it is shown that the N-LMS can be obtained as a special case of the Kalman filter with the parameters

$$P_{\mathrm{d}}=\alpha I,\alpha \in R^{+};P(0)=P_{\mathrm{d}};r(t)=1$$

in Eqs. (4), (7). In the N-LMS, the Riccati equation becomes redundant since it is initiated at its stationary point. Thus, the resulting filter (2), (4), (7), (8) is insensitive to loss of excitation. Similarly, in the SG algorithm, once the Riccati equation has converged to the stationary point $P_{\mathrm{d}}$, it becomes robust against lack of excitation, in the sense that the solution does not diverge.

Interestingly, a directional tracking algorithm presented in Cao and Schwartz (2004), the one identified as Algorithm 1, is also very close to the N-LMS. The suggested choice of the free term in (4) is

$$Q(t)=\frac{\gamma \varphi (t){\varphi }^{\mathrm{T}}(t)}{\epsilon +{\varphi }^{\mathrm{T}}(t)\varphi (t)}\text{,}$$

where $\gamma >0$ and $\epsilon >0$ are arbitrary scalars. This algorithm becomes equivalent to the N-LMS with $\epsilon \gamma =1$ and being initiated at the stationary point of (4), i.e. $P(0)=\gamma I$. Thus, it is also a special case of the SG algorithm.

It is as well worth to note at this point that the proofs of boundedness of the recursive estimation algorithms in Cao and Schwartz (2004) are based on the assumption that all considered solutions to the Riccati equation are positive semidefinite. As mentioned before, this cannot be guaranteed under lack of excitation.

The main result of the paper is formulated in Proposition 4 and provides an explicit parametrization of all stationary solutions to the Riccati equation arising in the SG algorithm. Under sufficient excitation, defined in Section 5, the parametrization implies that the stationary point is unique and is pre-assigned by the matrix $P_{\mathrm{d}}$. When the excitation is insufficient, the parametrization defines the manifold of all possible solutions, including asymmetric ones.

The paper is organized as follows. First the mechanism behind Riccati equation windup is explained and the problem treated in the article is formulated. Then an equivalent linear time-varying form of the Riccati equation in the SG algorithm (4), (7) is provided. The equation itself was used before, see e.g. Stenlund and Gustafsson (2002), and Medvedev, 2003, Medvedev, 2004, but only a proof for the case when $P>0$ had been originally given in Stenlund and Gustafsson (2002). Then, using the linear form, stationary points of (4), (7) are investigated and some results on the behavior of the algorithm under insufficient excitation are presented.