A bias-corrected estimator for nonlinear systems with output-error type model structures

doi:10.1016/j.automatica.2014.07.021

Automatica

Volume 50, Issue 9, September 2014, Pages 2373-2380

https://doi.org/10.1016/j.automatica.2014.07.021 Get rights and content

Abstract

Parametric identification of linear time-invariant (LTI) systems with output-error (OE) type of noise model structures has a well-established theoretical framework. Different algorithms, like instrumental-variables based approaches or prediction error methods (PEMs), have been proposed in the literature to compute a consistent parameter estimate for linear OE systems. Although the prediction error method provides a consistent parameter estimate also for nonlinear output-error (NOE) systems, it requires to compute the solution of a nonconvex optimization problem. Therefore, an accurate initialization of the numerical optimization algorithms is required, otherwise they may get stuck in a local minimum and, as a consequence, the computed estimate of the system might not be accurate. In this paper, we propose an approach to obtain, in a computationally efficient fashion, a consistent parameter estimate for output-error systems with polynomial nonlinearities. The performance of the method is demonstrated through a simulation example.

Introduction

Parametric identification of linear time-invariant (LTI) systems with output-error (OE) noise models enjoys a well-established theoretical framework. Different identification techniques have been proposed in the literature to compute a consistent estimate of the system parameters, like instrumental variables based approaches (Söderström & Stoica, 1983); prediction-error methods (PEMs) (Diversi et al., 2007, Ljung, 1999) and bias-compensated least-squares algorithms, where the standard least square (LS) estimate is properly modified in order to remove the bias introduced by the noise (Beghelli et al., 1990, Hong et al., 2007, Stoica and Söderström, 1982, Zheng, 1998, Zheng, 2002). Among the aforementioned identification algorithms, only the PEM approach is guaranteed to provide a consistent estimate of the parameters of nonlinear systems with an output-error noise model. Specifically, in the PEM, the system parameters are estimated by minimizing the 2-norm of the difference between the measured output of the data-generating system and the simulated model output. This leads, also in the linear case, to a nonconvex optimization problem. Although, under mild assumptions, the global minimum of the minimized cost function is a consistent estimate of the systems parameters, the numerical optimization algorithms (e.g., gradient methods) can get trapped in local minima, which might lead to an inaccurate estimate of the system, in particular when the initial conditions of the optimization algorithm are not “close” to the global minimum or when complex nonlinear models have to be estimated (see, e.g., Ljung, 2010).

Significant efforts have been spent in recent years to develop numerical efficient algorithms for parametric identification of nonlinear output-error (NOE) systems. In particular, an instrumental-variable based approach providing a consistent estimate for linear-parameter-varying systems under zero-mean colored noise conditions, e.g., output-error or Box–Jenkins setting, is proposed by Laurain et al. in Laurain, Gilson, Tóth, and Garnier (2010). In the context of block-oriented identification, different algorithms for parametric identification of Hammerstein-like and Wiener-like structures with output-error noise models are presented in Cerone et al., 2012, Cerone et al., 2013, Ding, Shi, and Chen (2007), Laurain, Gilson, and Garnier (2012), Wang, Zhang, and Ljung (2009) and Zhu (2002). In the more general framework of nonlinear errors-in-variables (EIV) models (i.e., when all the regressor variables are contaminated by error or measurement noise), identification schemes for systems described by continuous nonlinear functions are presented in Baran (2000), Fazekas and Kukush (1997) and Vandersteen, Rolain, Schoukens, and Pintelon (1996). In these contributions, every moment of the noise is assumed to be a-priori known. In Vajk and Hetthéssy (2003), a generalization of the Koopmans–Levin’s method (Fernando & Nicholson, 1985), originally developed for EIV linear system identification, is properly extended to handle identification of static systems described by polynomial functions, under the assumption that the structure of the noise covariance matrix is known. In Jun and Bernstein (2007), Jun and Bernstein propose a method which is able to consistently estimate the parameters of nonlinear systems described by third or lower order polynomials without assuming that the noise covariance is known.

In this paper, we present a novel approach to consistently estimate the parameters of polynomial output-error systems with Gaussian-distributed measurement noise. One of the main benefits of the algorithm proposed in this paper is its ability to compute a consistent estimate of the system parameters with a modest computational complexity and without assuming to know the variance of the noise corrupting the data. The paper is organized as follows. In Section 2, the considered estimation problem is introduced. A consistent estimate of the system parameters is derived in Section 3 under the assumption that the variance of the noise affecting the output measurements is a-priori known. The latter assumption is relaxed in Section 4 in order to extend the applicability of the proposed method to a more general setting. The effectiveness of the presented identification procedure is shown in Section 5 through a simulation example.

Section snippets

Problem description

Consider a discrete-time, single-input single-output (SISO) data-generating system $S_{o}$ described by the nonlinear output-error (NOE) structure: $x (t) = h^{o} (x (t - 1), \dots, x (t - n_{a}), u (t), \dots, u (t - n_{b})),$ $y (t) = x (t) + e_{o} (t),$ where $u (t)$ is the measured input at time instant $t$ , $x (t)$ and $y (t)$ are the noise-free and the noise-corrupted output, respectively, and $e_{o} (t)$ is a stationary white Gaussian noise, independent of $x (t)$ and $u (t)$ , with zero mean and finite variance $σ_{e}^{2}$ . The function $h^{o} (\cdot)$ is a real-valued multivariate

A bias-corrected LS estimate

In order to correct the noise-induced bias introduced by the LS estimate, first note that $B_{Δ} (θ_{o}, Φ, Δ Φ)$ depends on the true parameters $θ_{o}$ and the noise-free regressor matrix $Φ_{o}$ (see the definition of $Δ Φ$ in (9a)). As a consequence, such a bias cannot be computed based on the observed input/output data and thus it cannot be directly subtracted from the LS estimate ${\hat{θ}}_{LS}$ .

Inspired by (10), the following corrected LS estimate is introduced: ${\tilde{θ}}_{CLS} = {\hat{θ}}_{LS} - B_{Δ} ({\tilde{θ}}_{CLS}, Φ, Δ Φ),$ with $B_{Δ} ({\tilde{θ}}_{CLS}, Φ, Δ Φ)$ being $B_{Δ} ({\tilde{θ}}_{CLS}, Φ,$

Estimation with unknown noise variance

In order to compute a relation between the noise variance $σ_{e}^{2}$ and the system parameters $θ_{o}$ , let us rewrite the minimal value of the loss function $V (θ, D_{N})$ as $V ({\hat{θ}}_{LS}, D_{N}) = \frac{1}{N} {‖ Y - Φ {\hat{θ}}_{LS} ‖}_{2}^{2} = \frac{1}{N} {‖ Φ_{o} θ_{o} + E_{o} - Φ {\hat{θ}}_{LS} ‖}_{2}^{2} = \frac{1}{N} ({‖ E_{o} ‖}_{2}^{2} + {‖ Φ_{o} θ_{o} - Φ {\hat{θ}}_{LS} ‖}_{2}^{2} - 2 E_{o}^{⊤} (Φ_{o} θ_{o} - Φ {\hat{θ}}_{LS})) .$ As the number of measurements $N$ goes to infinity, the term $\frac{1}{N} {‖ E_{o} ‖}_{2}^{2}$ converges (w.p. 1) to $σ_{e}^{2}$ , while $\frac{1}{N} E_{o}^{⊤} (Φ_{o} θ_{o} - Φ {\hat{θ}}_{LS})$ converges (w.p. 1) to 0 because of the independence of $e_{o} (t)$ from the noise-free and noise-corrupted regressors $φ_{o} (t)$ and $φ (t)$ . Based

Numerical example

The capabilities of the estimation scheme proposed in the paper are now shown through a simulation example.

Conclusion

In this paper, we have proposed a method for computing a consistent parameter estimate for output-error systems with polynomial nonlinearities. The noise corrupting the output measurements has been assumed to be white and Gaussian with unknown variance. The underlying idea of the proposed approach is to estimate, from the measured data, the bias introduced by the LS approach. The estimated bias is guaranteed to asymptotically converge to the true one as the number of measurements increases, and

References (23)

S. Beghelli et al.
The frisch scheme in dynamic system identification
Automatica
(1990)
V. Cerone et al.
Bounded error identification of Hammerstein systems through sparse polynomial optimization
Automatica
(2012)
F. Ding et al.
Auxiliary model-based least-squares identification methods for Hammerstein output-error systems
Systems & Control Letters
(2007)
R. Diversi et al.
Maximum likelihood identification of noisy input–output models
Automatica
(2007)
I. Fazekas et al.
Asymptotic properties of an estimator in nonlinear functional errors-in-variables models with dependent error terms
Computers and Mathematics with Applications
(1997)
M. Hong et al.
Accuracy analysis of bias-eliminating least squares estimates for errors-in-variables systems
Automatica
(2007)
V. Laurain et al.
Refined instrumental variable methods for identification of LPV Box—Jenkins models
Automatica
(2010)
L. Ljung
Perspectives on system identification
Annual Reviews in Control
(2010)
I. Vajk et al.
Identification of nonlinear errors-in-variables models
Automatica
(2003)
J. Wang et al.
Revisiting Hammerstein system identification through the two-stage algorithm for bilinear parameter estimation
Automatica
(2009)

Y. Zhu

Estimation of an N–L–N Hammerstein–Wiener model

Automatica

(2002)

Cited by (16)

A bias-correction modeling method of Hammerstein–Wiener systems with polynomial nonlinearities using noisy measurements
2024, Mechanical Systems and Signal Processing
Modeling of Hammerstein–Wiener nonlinear systems has received a lot of attention in the signal processing community. However, all existing model identification methods may fail to provide a consistent parameter estimate for Errors-In-Variables (EIV) Hammerstein–Wiener systems, where both input–output data are contaminated by measurement white noises. In this paper, a bias-correction Least-Squares (LS) algorithm for consistent identification of EIV Hammerstein–Wiener systems with polynomial nonlinearities using noisy measurements is proposed. Firstly, the analytic expression for the estimated bias of the LS algorithm using noisy measurements for EIV Hammerstein–Wiener systems with polynomial nonlinearities is derived, which is caused by the correlation between the input–output signals and measurement noises. Secondly, a consistent estimation method for the bias-correcting term, including a recursive step and a cross-validation step based on the available noisy measurements only, is then proposed to estimate the unknown terms of noises variances and noise-free measurements in the estimated bias. The effectiveness of the proposed algorithm is demonstrated through a simulated example and a robot arm system.
On identification of nonlinear ARX models with sparsity in regressors and basis functions
2021, IFAC-PapersOnLine
We present techniques for minimal order, sparse identification of Nonlinear ARX models. We consider two notions of sparsity - in the number of regressors used and in the number of basis functions employed by their regressor-to-output maps. We propose two regularized formulations for the sparse estimation problem with the additional constraint on maximum lag. The estimation is performed using proximal gradient descent methods. A bootstrapping technique in regressor space is proposed for tuning the regularization hyperparameters. We then present an extension to the basic NARX structure that guarantees BIBO stability and thus helps improve the generalizability and long-term forecasting ability of the model. The extension exploits the atomic representation of linear systems, and the associated minimization technique, to identify model parameters under sparsity constraints.
A bias-correction approach for the identification of piecewise affine output-error models
2020, IFAC-PapersOnLine
The paper presents an algorithm for the identification of PieceWise Affine Output-Error (PWA-OE) models, which involves the estimation of the parameters defining affine submodels as well as a partition of the regressor space. For the estimation of affine submodel parameters, a bias-correction scheme is presented to correct the bias in the least squares estimates which is caused by the output-error noise structure. The obtained bias-corrected estimates are proven to be consistent under suitable assumptions. The bias-correction method is then combined with a recursive estimation algorithm for clustering the regressors. These clusters are used to compute a partition of the regressor space by employing linear multi-category discrimination. The effectiveness of the proposed methodology is demonstrated via a simulation case study.
Particle filtering based parameter estimation for systems with output-error type model structures
2019, Journal of the Franklin Institute
Citation Excerpt :
For bias corrected parameter estimation, a bias correction or bias compensation is devoted to eliminate estimation bias caused by colored noise [22]. For nonlinear systems with OE type model structures, Piga and Tóth presented a bias-corrected estimator [23]. A bias correction method was discussed by Zheng for the identification of linear dynamic errors-in-variables models [24].
The output-error model structure is often used in practice and its identification is important for analysis of output-error type systems. This paper considers the parameter identification of linear and nonlinear output-error models. A particle filter which approximates the posterior probability density function with a weighted set of discrete random sampling points is utilized to estimate the unmeasurable true process outputs. To improve the convergence rate of the proposed algorithm, the scalar innovations are grouped into an innovation vector, thus more past information can be utilized. The convergence analysis shows that the parameter estimates can converge to their true values. Finally, both linear and nonlinear results are verified by numerical simulation and engineering.
A bias-correction method for closed-loop identification of Linear Parameter-Varying systems
2018, Automatica
Citation Excerpt :
The main idea underlying bias-correction methods is to eliminate the bias from ordinary Least Squares (LS) to obtain a consistent estimate of the model parameters. Bias-correction methods have been used in the past for the identification of LTI systems both in the open-loop (Hong, Söderström, & Zheng, 2007; Zheng, 2002) and closed-loop setting (Gilson & Van den Hof, 2001; Zheng & Feng, 1997), as well as for open-loop identification of nonlinear (Piga & Tóth, 2014) and LPV systems from noisy scheduling variable observations (Piga, Cox, Tóth, & Laurain, 2015). The main idea behind the closed-loop identification algorithm proposed in this paper is to quantify, based on the available measurements, the asymptotic bias due to the correlation between the plant input and the measurement noise.
Due to safety constraints and unstable open-loop dynamics, system identification of many real-world processes often requiresgathering data from closed-loop experiments. In this paper, we present a bias-correction scheme for closed-loop identification of Linear Parameter-Varying Input–Output (LPV-IO) models, which aims at correcting the bias caused by the correlation between the input signal exciting the process and output noise. The proposed identification algorithm provides a consistent estimate of the open-loop model parameters when both the output signal and the scheduling variable are corrupted by measurement noise. The effectiveness of the proposed methodology is tested in two simulation case studies.
LPV Model Order Selection from Noise-corrupted Output and Scheduling Signal Measurements
2017, IFAC-PapersOnLine
In parametric identification of Linear Parameter-Varying (LPV) systems, it is important to achieve a low variance of the model estimate by limiting the number of parameters to be identified. This is the well known “model order selection” problem, which consists of selecting the number of input and output delays and the basis functions characterizing the dependence of the LPV model parameters on the scheduling signal. Ignoring the effect of noise on the observations of the scheduling signals may lead to a bias in the final estimate and, as a consequence, also to an incorrect selection of the model order. In this paper, we introduce a “bias-corrected cost function” for the identification of LPV systems from noise-corrupted observations of the output and scheduling variable. The introduced cost function provides a bias-free parameter estimation along with model order selection. The proposed identification approach has two main advantages: (i) the problem of model order selection can be handled by adding a LASSO-like penalty term to the bias-corrected cost function; (ii) it provides a bias-free cost as a criterion to tune some hyper-parameters influencing the final parameter estimate.

View all citing articles on Scopus

Dario Piga received the Master’s degree in mechatronics engineering and the Ph.D. in systems engineering both from the Politecnico di Torino, Italy, in 2008 and 2012, respectively. He was a Postdoctoral Researcher at the Delft Center for Systems and Control (DCSC), Delft University of Technology, The Netherlands, in 2012, and at the Control Systems Group, Eindhoven University of Technology, The Netherlands, in 2013. Currently, he is a Researcher at the Istituto Dalle Molle di Studi sull’Intelligenza Artificiale (IDSIA), Scuola Universitaria Professionale della Svizzera Italiana, Lugano, Switzerland. His main research interests include system identification, machine learning and LMI-relaxation for polynomial optimization problems.

Roland Tóth was born in 1979 in Miskolc, Hungary. He received the B.Sc. degree in electrical engineering and the M.Sc. degree in information technology in parallel at the University of Pannonia, Veszprém, Hungary, in 2004, and the Ph.D. degree (cum laude) from the Delft Center for Systems and Control (DCSC), Delft University of Technology (TUDelft), Delft, The Netherlands, in 2008. He was a Post-Doctoral Research Fellow at DCSC, TUDelft, in 2009 and at the Berkeley Center for Control and Identification, University of California, Berkeley, in 2010. He held a position at DCSC, TUDelft, in 2011–2012. Currently, he is an Assistant Professor at the Control Systems Group, Eindhoven University of Technology (TU/e). He is an Associate Editor of the IEEE Conference Editorial Board. His research interest is in linear parameter-varying (LPV) and nonlinear system identification, modeling and control, machine learning, process modeling and control, and behavioral system theory. Dr. Tóth received the TUDelft Young Researcher Fellowship Award in 2010.

^☆: This work was supported by the Netherlands Organization for Scientific Research (NWO, Grant No. 639.021.127). The material in this paper was not presented at any conference. This paper was recommended for publication in revised form by Associate Editor Andrea Garulli under the direction of Editor Torsten Söderström.

¹: Tel.: +41 0 58 666 6664; fax: +41 0 58 666 6661.

View full text

Brief paperA bias-corrected estimator for nonlinear systems with output-error type model structures☆

Abstract

Introduction

Section snippets

Problem description

A bias-corrected LS estimate

Estimation with unknown noise variance

Numerical example

Conclusion

Automatica

Automatica

Systems & Control Letters

Automatica

Computers and Mathematics with Applications

Automatica

Automatica

Annual Reviews in Control

Automatica

Automatica

Automatica

Brief paper
A bias-corrected estimator for nonlinear systems with output-error type model structures☆