Hierarchical gradient parameter estimation algorithm for Hammerstein nonlinear systems using the key term separation principle

doi:10.1016/j.amc.2014.09.070

Applied Mathematics and Computation

Volume 247, 15 November 2014, Pages 1202-1210

https://doi.org/10.1016/j.amc.2014.09.070 Get rights and content

Abstract

In this paper, we use the hierarchical identification principle to decompose a Hammerstein controlled autoregressive system into three subsystems, apply the key term separation principle to express the system output as a linear combination of the system parameters, and then derive a hierarchical gradient parameter estimation algorithm for identifying all subsystems. Finally, a multi-innovation stochastic gradient algorithm is presented to improve the estimation accuracy by making full of the identification innovation. The simulation results show that the proposed algorithm is effective.

Introduction

Nonlinear system identification has attracted much research attention [1], [2], [3], and the typical nonlinear systems include the Hammerstein models (linear time-invariant (LTI) block following a static nonlinear block) [4], [5], the Wiener models (LTI block preceding a static nonlinear block) [6] and their combinations [7]. Parameter estimation is basic for system analysis [8], controller design [9], [10], [11], state estimation and filtering [12], [13], [14] and predictive control [15], [16], [17].

Recursive or iterative methods have been used for the identification of Hammerstein nonlinear systems [18], [19], [20] and for finding the solutions of matrix equations [21], [22], [23]. Many iterative identification methods have been reported for nonlinear systems, e.g., the Newton iterative algorithm [24] and the least squares based iterative algorithm for Hammerstein nonlinear systems [25]. Recently, Bai and Li studied the convergence of the iterative algorithm for a general Hammerstein system identification [26]; Wang et al. presented a recursive and an iterative least squares algorithms for an input nonlinear system with a dynamic subspace state space model to generate parameter estimates and state estimates by using the hierarchical identification principle and by replacing the unknown state variables with their estimates [27].

The key term separation principle, which was proposed first by Jozef Vörös, is important for studying identification of nonlinear systems, and has been used for discontinuous Hammerstein systems [28], Hammerstein systems with two-segment nonlinearities [29] and with multi-segment piecewise-linear characteristics [30], and the Hammerstein systems with discontinuous nonlinearities containing dead-zones [31]. The basic idea is to use the key term separation principle to express the output of the Hammerstein system as a linear combination of the system parameters.

The decomposition based hierarchical identification principle is to decompose a system into several subsystems with smaller dimensions and fewer variables and can be used for the identification of nonlinear systems with complex structures. This paper uses the key term separation principle and the hierarchical identification principle to derive the hierarchical stochastic gradient identification algorithm and a hierarchical multi-innovation stochastic gradient algorithm for Hammerstein controlled autoregressive (H-CAR) systems based on the multi-innovation identification theory [32], [33]. This paper extends the multi-innovation identification methods from linear systems to nonlinear systems and this work differs from the previous ones in [32], [33].

The rest of this paper is organized as follows. Section 2 expresses the output of a Hammerstein nonlinear system as a linear combination of the system parameters using the key term separation principle. Section 3 develops a hierarchical stochastic gradient algorithm for the H-CAR system. A multi-innovation stochastic gradient algorithm is presented for comparisons in Section 4. Section 5 provides a simulation example to illustrate the effectiveness of the proposed algorithm. Finally, some concluding remarks are given in Section 6.

Section snippets

System description and identification model

Let us introduce some notation. “ $A = : X$ ” or “ $X ≔ A$ ” stands for “A is defined as X”. The symbol $I$ ( $I_{n}$ ) stands for an identity matrix of appropriate size ( $n \times n$ ); $1_{n}$ represents an n-dimensional column vector whose elements are all 1; the superscript T denotes the matrix transpose; the norm of a matrix $X$ is defined by $‖ X ‖^{2} ≔ tr [{XX}^{T}]$ . Let $\hat{X} (t)$ be the estimate of $X$ at time t.

Consider a Hammerstein nonlinear system, whose linear part is a controlled autoregressive model, $A (z) y (t) = B (z) \bar{u} (t) + v (t),$ where $y (t)$ is

The hierarchical stochastic gradient algorithm

In order to show the advantage of the proposed multi-innovation gradient algorithm, the following simply derives a hierarchical stochastic gradient algorithm based on the hierarchical identification principle. The basic idea is to decompose a Hammerstein system into several subsystems with smaller dimension and fewer variables, and then to identify the parameter vector of each subsystem. The details are as follows.

The multi-innovation stochastic gradient algorithm

In this section, we expand the scalar innovation $e (t) ≔ y (t) - φ_{a}^{T} (t) \hat{a} (t - 1) - {\hat{φ}}_{b}^{T} (t) \hat{b} (t - 1) - f^{T} (t) \hat{c} (t - 1) \in R$ to a multi-innovation vector $E (p, t)$ , and derive a multi-innovation stochastic gradient identification algorithm in order to improve the parameter accuracy of the hierarchical SG algorithm.

Refer to the method in [33], consider p data from $j = t - p + 1$ to $j = t$ and define the multi-innovation cost functions $J_{1} (a) ≔ \sum_{j = 0}^{p - 1} {[y_{a} (t - j) - φ_{a}^{T} (t - j) a]}^{2},$ $J_{2} (b) ≔ \sum_{j = 0}^{p - 1} {[y_{b} (t - j) - φ_{b}^{T} (t - j) b]}^{2},$ $J_{3} (c) ≔ \sum_{j = 0}^{p - 1} [y_{c} (t - j) - f^{T} (t - j) c$

Example

Consider the following H-CAR system, $y (t) + a_{1} y (t - 1) + a_{2} y (t - 2) = \bar{u} (t) + b_{1} \bar{u} (t - 1) + b_{2} \bar{u} (t - 2) + v (t), b_{0} = 1,$ $\bar{u} (t) = c_{1} f_{1} (u (t)) + c_{2} f_{2} (u (t)) = c_{1} u (t) + c_{2} u^{2} (t),$ $a = {[a_{1}, a_{2}]}^{T} = {[- 1.35, 0.75]}^{T},$ $b = {[b_{1}, b_{2}]}^{T} = {[0.25, 0.10]}^{T},$ $c = {[c_{1}, c_{2}]}^{T} = {[0.50, 0.90]}^{T},$ $θ = {[a_{1}, a_{2}, b_{1}, b_{2}, c_{1}, c_{2}]}^{T} = {[- 1.35, 0.75, 0.25, 0.10, 0.50, 0.90]}^{T} .$

In simulation, the input ${u (t)}$ is taken as an uncorrelated persistent excitation signal sequence with zero mean and unit variance, ${v (t)}$ as a white noise sequence with zero mean and variance $σ^{2} = {0.30}^{2}$ , respectively. Applying the

Conclusions

This paper proposes a multi-innovation stochastic gradient algorithm for a Hammerstein nonlinear controlled autoregressive system using the key term separation principle and the hierarchical identification principle. The proposed hierarchical MISG algorithm has a higher accuracy compared with the hierarchical SG algorithm, and can be applied in dealing with the parameter identification of other linear systems [34], [35], [36], [37], [38], nonlinear systems [39] and multirate sampled systems [40]

References (49)

J. Sjöberg et al.
Nonlinear black-box modeling in system identification: a unified overview
Automatica
(1995)
D.Q. Wang et al.
Data filtering based recursive least squares algorithm for Hammerstein systems using the key-term separation principle
Inf. Sci.
(2013)
Y. Liu et al.
Iterative identification of Hammerstein systems
Automatica
(2007)
D.Q. Wang et al.
Least squares based and gradient based iterative identification for Wiener nonlinear systems
Signal Process.
(2011)
Y. Shi et al.
Robust mixed H-2/H-infinity control of networked control systems with random time delays in both forward and backward communication links
Automatica
(2011)
L. Xu
A proportional differential control method for a time-delay system using the Taylor expansion approximation
Appl. Math. Comput.
(2014)
F. Ding
Combined state and least squares parameter estimation algorithms for dynamic systems
Appl. Math. Modell.
(2014)
F. Ding
State filtering and parameter identification for state space systems with scarce measurements
Signal Process.
(2014)
H. Li et al.
Event-triggered robust model predictive control of continuous-time nonlinear systems
Automatica
(2014)
H. Li et al.
Distributed receding horizon control of large-scale nonlinear systems: handling communication delays and disturbances
Automatica
(2014)

J.H. Li

Parameter estimation for Hammerstein CARARMA systems based on the Newton iteration

Appl. Math. Lett.

(2013)

M. Dehghan et al.

An iterative algorithm for the reflexive solutions of the generalized coupled Sylvester matrix equations and its optimal approximation

Appl. Math. Comput.

(2008)

L. Xie et al.

Gradient based and least squares based iterative algorithms for matrix equations AXB + CXTD = F

Appl. Math. Comput.

(2010)

M. Dehghan et al.

An iterative algorithm for the reflexive solutions of the generalized coupled Sylvester matrix equations and its optimal approximation

Appl. Math. Comput.

(2008)

F. Ding et al.

Identification methods for Hammerstein nonlinear systems

Digital Signal Process.

(2011)

F. Ding et al.

Identification of Hammerstein nonlinear ARMAX systems

Automatica

(2005)

E.W. Bai et al.

Convergence of the iterative algorithm for a general Hammerstein system identification

Automatica

(2010)

J. Vörös

Parameter identification of discontinuous Hammerstein systems

Automatica

(1997)

Y.J. Liu et al.

Multi-innovation stochastic gradient algorithm for multiple-input single-output systems using the auxiliary model

Appl. Math. Comput.

(2009)

F. Ding et al.

Performance analysis of multi-innovation gradient type identification methods

Automatica

(2007)

Y.B. Hu

Iterative and recursive least squares estimation algorithms for moving average systems

Simul. Modell. Pract. Theory

(2013)

F. Ding

Hierarchical parameter estimation algorithms for multivariable systems using measurement information

Inf. Sci.

(2014)

Y.B. Hu et al.

A multi-innovation generalized extended stochastic gradient algorithm for output nonlinear autoregressive moving average systems

Appl. Math. Comput.

(2014)

J. Ding et al.

Auxiliary model based parameter estimation for dual-rate output error systems with colored noise

Appl. Math. Modell.

(2013)

Cited by (65)

Design of auxiliary model based normalized fractional gradient algorithm for nonlinear output-error systems
2022, Chaos, Solitons and Fractals
A new avenue of fractional calculus applications has emerged that investigates the design of fractional gradient based novel iterative methods for analyzing fractals and nonlinear dynamics in solving engineering and applied sciences problems. The most discussed algorithm in this regard is fractional least mean square (FLMS) algorithm. This study presents an auxiliary model based normalized variable initial value FLMS (AM-NVIV-FLMS) algorithm for input nonlinear output error (INOE) system identification. First, NVIV-FLMS is presented to automatically tune the learning rate parameter of VIV-FLMS and then the AM-NVIV-FLMS is introduced by incorporating the auxiliary model idea that replaces the unknown values of the information vector with the output of auxiliary model. The proposed AM-NVIV-FLMS scheme is accurate, convergent, robust and reliable for INOE system identification. Simulation results validate the significance and efficacy of the proposed scheme.
Design of fractional hierarchical gradient descent algorithm for parameter estimation of nonlinear control autoregressive systems
2022, Chaos, Solitons and Fractals
Citation Excerpt :
Ding et al., proposed various gradient descent and least-squares algorithms based on multi-innovation approach, auxiliary model theory and hierarchical identification principle (HIP) for nonlinear system identification [46–51]. The HIP based gradient descent (HGD) algorithm decomposes the system into sub-systems with smaller dimension and variables, thus reducing the overall complexity and making the HGD more efficient than the standard GD method [48]. The good performance of the fractional gradient methods over the standard counterparts motivate to design the fractional variant of HGD that generalizes the HGD to fractional order.
The trend of developing fractional gradient based iterative adaptive strategies is evolved in the recent years through effectively exploring the fractional and fractal dynamics. In this study, fractional hierarchical gradient descent (FHGD) is proposed by generalizing the standard hierarchical gradient descent (HGD) to fractional order for effectively solving nonlinear system identification problem. The FHGD is effectively to applied to estimate the parameters of nonlinear control autoregressive (NCAR) systems under different fractional order and noise conditions. The fractional order greater than 1 provides faster convergence speed, less than 1 gives better steady state performance and equal to 1 reduces the FHGD to HGD. The accurate estimation of NCAR system parameters representing electrically stimulated muscle model validates the efficacy and robustness of the proposed FHGD in comparison with the standard HGD.
System identification of hammerstein models by using backward shift algorithm
2022, Applied Mathematics and Computation
Citation Excerpt :
Researchers have proposed many methods for identification of Hammerstein models. Among them, there are blind method [3], stochastic methods [13,30–32], recursive methods [10,28,34,35,39], key term separation technique [16], nonparametric approaches [26,33], Hierarchical parameter estimation algorithm [4,7–9,11,17,29,36–38] and so on. All methods have their advantages on some conditions and are efficient to some special kind of Hammerstein systems.
In this paper, a new identification method for discrete-time Hammerstein systems is proposed. The method is a joint use of discrete Fourier transform, backward shift method, and the least squares method. The frequency responses are obtained with sampled input and output data in the time domain through discrete Fourier transform. It is followed by the backward shift algorithm that was originally developed for estimating poles of linear time-invariant systems. After poles of linear subsystem are estimated, coefficients of linear and nonlinear subsystems are respectively determined by using the least squares (LS) method. The robustness of the backward shift algorithm guarantees the effectiveness of the proposed algorithm. Simulation results show that the poles of linear subsystem are well located. Thus, it is practical to identify discrete Hammerstein systems.
Fully parametric identification for continuous time fractional order Hammerstein systems
2020, Journal of the Franklin Institute
This paper mainly investigates the issue of parameters identification for continuous time fractional order Hammerstein systems. When the commensurate order of linear part in system is prior information, two estimation algorithms are proposed to identify the parameters. Both methods use filters to obtain the derivative of the continuous time signal: one based on the Poisson filter and the other using the auxiliary model to construct the filter. Afterwards, the commensurate order update law is given, which contains the relationship between parameters and orders. In addition, due to the existence of fractional order terms, an irrational item will be introduced, and this paper gives a detailed mathematical derivation to discuss this issue. The effectiveness of proposed methods is demonstrated by the numerical simulations.
Backtracking search optimization heuristics for nonlinear Hammerstein controlled auto regressive auto regressive systems
2019, ISA Transactions
In this work, novel application of evolutionary computational heuristics is presented for parameter identification problem of nonlinear Hammerstein controlled auto regressive auto regressive (NHCARAR) systems through global search competency of backtracking search algorithm (BSA), differential evolution (DE) and genetic algorithms (GAs). The mean squared error metric is used for the fitness function of NHCARAR system based on difference between actual and approximated design variables. Optimization of the cost function is conducted with BSA for NHCARAR model by varying degrees of freedom and noise variances. To verify and validate the worth of the presented scheme, comparative studies are carried out with its counterparts DE and GAs through statistical observations by means of weight deviation factor, root of mean squared error, and Thiel’s inequality coefficient as well as complexity measures.
Fitting the exponential autoregressive model through recursive search
2019, Journal of the Franklin Institute
This paper focuses on the recursive parameter estimation methods for the exponential autoregressive (ExpAR) model. Applying the negative gradient search and introducing a forgetting factor, a stochastic gradient and a forgetting factor stochastic gradient algorithms are presented. In order to improve the parameter estimation accuracy and the convergence rate, the multi-innovation identification theory is employed to derive a forgetting factor multi-innovation stochastic gradient algorithm. A simulation example is provided to test the effectiveness of the proposed algorithms.

View all citing articles on Scopus

^☆: This work was supported by the National Natural Science Foundation of China (No. 61203111) and the PAPD of Jiangsu Higher Education Institutions.

View full text

Hierarchical gradient parameter estimation algorithm for Hammerstein nonlinear systems using the key term separation principle☆

Abstract

Introduction

Section snippets

System description and identification model

The hierarchical stochastic gradient algorithm

The multi-innovation stochastic gradient algorithm

Example

Conclusions

Automatica

Inf. Sci.

Automatica

Signal Process.

Automatica

Appl. Math. Comput.

Appl. Math. Modell.

Signal Process.

Automatica

Automatica

Appl. Math. Lett.

Appl. Math. Comput.

Appl. Math. Comput.

Appl. Math. Comput.

Digital Signal Process.

Automatica

Automatica

Automatica

Appl. Math. Comput.

Automatica

Simul. Modell. Pract. Theory

Inf. Sci.

Appl. Math. Comput.

Appl. Math. Modell.