On the lower smoothing bound in identification of time-varying systems

Abstract

In certain applications of nonstationary system identification the model-based decisions can be postponed, i.e. executed with a delay. This allows one to incorporate in the identification process not only the currently available information, but also a number of “future” data points. The resulting estimation schemes, which involve smoothing, are not causal. Assuming that the infinite observation history is available, the paper establishes the lower steady-state estimation bound for any noncausal estimator applied to a linear system with randomly drifting coefficients (under Gaussian assumptions). This lower bound complements the currently available one, which is restricted to causal estimators.

Introduction

Consider the problem of identification of a linear time-varying system governed by

$$y(t)={\mathit{\varphi }}^{\mathrm{T}}(t)\mathit{\theta }(t)+v(t)\text{,}$$

$$\mathit{\theta }(t)=\mathit{\theta }(t-1)+\mathbf{w}(t)\text{,}$$

where $y(t)$ denotes the system output, $\mathit{\varphi }(t)=[{\varphi }_1(t),\dots ,{\varphi }_n(t){]}^{\mathrm{T}}$ is a known regression vector, $v(t)$ denotes white measurement noise, $\mathit{\theta }(t)=[{\theta }_1(t),\dots ,{\theta }_n(t){]}^{\mathrm{T}}$ is the vector of unknown and time-varying system coefficients and $\mathbf{w}(t)$ stands for the one-step parameter change.

In this paper we will focus on systems with randomly drifting parameters, namely we will assume that $\{\mathbf{w}(t)\}$ is a white noise sequence. Under such assumption (2) becomes the so-called random-walk (RW) model. Systems with RW parameter changes have been extensively studied in the literature on identification of nonstationary processes as they allow one to arrive at analytical results. Therefore, even though from the practical viewpoint the RW model may be criticized as “unrealistic”, it is widely used as a benchmark for evaluation and comparison of tracking performance of different identification schemes, such as Kalman filter-based (KF) algorithms, exponentially weighted least squares (EWLS) algorithms and least mean squares (LMS) algorithms—see e.g. Guo and Ljung (1995), Macchi (1995), Haykin (1996), and Niedźwiecki (2000). Naturally, evaluation of universal estimation bounds for such a benchmark problem is interesting from the theoretical viewpoint.

Under Gaussian assumptions the steady-state value of the Cramér–Rao type lower estimation bound, for the system (1), (2) with parameters evolving according to the RW model, was established in the paper of Ravikanth and Meyn (1999). Since the bound derived there was restricted to causal estimation schemes, used for parameter tracking, it will be further referred to as lower tracking bound (LTB). By causal we mean estimators that specify $\widehat{\mathit{\theta }}(t)$ in terms of the current and past data only: $y(s),\mathit{\varphi }(s),s⩽t$. While in the adaptive prediction and control problems, studied in Ravikanth and Meyn (1999), causality is an obvious requirement, there are some other important applications, such as adaptive noise canceling or adaptive channel equalization, where the parameter-based decisions can be executed with a delay of a certain number of sampling intervals. In cases like this, the estimate of $\mathit{\theta }(t)$ can be based not only on all past, but also on a number of future data points: $y(s),\mathit{\varphi }(s),s>t$. Since estimation accuracy of such noncausal estimation schemes, which incorporate smoothing, exceeds accuracy of their causal counterparts, it is important to know what is the possible margin of improvement. To address this problem, we will derive expression for the steady-state value of the lower smoothing bound (LSB). Since LSB specifies the best achievable accuracy of any parameter estimation scheme (whether causal or not) for a time-varying system at hand, in some sense it may be considered a more fundamental limitation than LTB.

The paper is organized as follows. Section 2 summarizes the current state of knowledge about estimation bounds applicable to time-varying systems. Section 3 presents the optimal noncausal estimation scheme for identification of linear systems with parameters drifting according to the RW model. The LSB for such systems is established in Section 4. Section 5 describes a computationally inexpensive smoothing procedure with sub-LTB performance. Finally, Section 6 concludes.