Hierarchical least squares algorithms for nonlinear feedback system modeling☆
Introduction
Recently, identification methods have been widely adopted in signal processing [1], [2], [3], system modeling [4], [5], [6] and solving matrix problems [7], [8], [9], [10], [11]. For example, Li and Ding considered the identification problem of Hammerstein finite impulse response moving average system and presented a maximum likelihood multi-innovation stochastic gradient algorithm by using the maximum likelihood principle and stochastic gradient method based on the key term separation technique [12].
A Wiener model can be described as a linear dynamic block followed by a static nonlinear function [13], [14]. Some researches have been taken in the identification issues of Wiener models. Main contribution includes, i.e., Wang and Ding derived a least squares-based algorithm and a gradient-based iterative identification algorithm for Wiener nonlinear systems by separating one bilinear cost function into two linear cost functions without re-parameterization to generate redundant estimates [15]; Bai presented a blind approach to the sampled Hammerstein–Wiener model identification by using the blind approach [16] and proposed an optimal two-stage identification algorithm for Hammerstein–Wiener systems [17]. In the identification area of nonlinear systems, recursive least squares algorithms have attracted much attention [18], [19], [20], [21], [22], [23]. For example, Ding et al. derived a recursive least squares algorithm for estimating the parameters of the nonlinear systems based on the model decomposition [24].
Wang and Tang developed an auxiliary model based recursive least squares algorithm for identifying the parameters of the linear-in-parameters output error moving average system [25]; Mu and Chen derived a stochastic approximation algorithm incorporated with the deconvolution kernel functions for errors-in-variables Wiener–Hammerstein system [26]. Many researchers focus on deriving the online identification methods, such as Wang and Zhang proposed an improved least squares algorithm to identify the parameters of the multivariable Hammerstein system by using the Taylor expansion on a least squares quadratic criterion function [27]; Ding et al. proposed a two-stage gradient based algorithm and a least squares based iterative parameter estimation algorithm for controlled autoregressive moving average systems [28]; Hu and Ding investigated a multistage least squares based iterative algorithm to estimate the parameters of nonlinear systems with moving average noise from input–output data [29], but the proposed least squares based iterative algorithm in [28], [29] is widely used in offline identification field and the first step of running the iterative algorithm is collecting all the input–output data at once.
The hierarchical method is emerging as a type of decomposition based identification tool in recent years. For instance, Wang et al. derived a decomposition-based recursive least squares algorithm and combined the decomposition principle and the least squares method after using the filtering idea [30], but the proposed algorithm in [30] is not a direct identification method that needs two different steps, the first is to transform the finite impulse response moving average model to a controlled autoregressive model, then the least squares algorithm can be used. Thus, the method in [30] has an intricate data processing. Hu et al. derived an auxiliary model based least squares parameter estimation algorithm [31], however, this method is still a two-stage algorithm and can only be used in the online identification.
Compared with the works in [30], [31], the main contribution of this paper is to propose a hierarchical recursive least squares algorithm that avoids the complex steps of filtering algorithm and to derive a hierarchical least squares iterative algorithm which is computationally efficient and can be used in the offline identification field, such as the waste water treatment process and other industry circle.
The rest of this paper is organized as follows. Section 2 gives a representation of a finite response impulse moving average Wiener nonlinear feedback system. Section 3 presents a hierarchical recursive least squares algorithm for the proposed Wiener feedback system. Section 4 derives a hierarchical least squares iterative algorithm. Section 5 provides an illustrative example for the results in this paper. Finally, some concluding remarks are offered in Section 6.
Section snippets
Problem formulation
Consider the following Wiener system depicted in Fig. 1:where the open-loop part is a finite response impulse moving average (FIR-MA) subsystem, r(t) and y(t) are the reference input and output sequences of the Wiener system, v(t) is the stochastic white noise with zero mean and variance σ2, B(z) and D(z) are polynomials in the unit backward shift operator z−1 [i.e. ]:
The hierarchical recursive least squares algorithm
To proceed further, here some notations are denoted. For convenience, t represents the current time and is regarded as the estimate of at time t; “” or “” denotes “A is defined as X”; E stands for the expectation operator; the symbol is defined as an identity matrix of appropriate sizes ; is denoted as the norm of a matrix or a column vector and defined by .
From Eq. (1), the identification model can be rewritten as
The hierarchical least squares iterative algorithm
Assume that the data length and define the stacked output vectors and , the stacked information matrices and , and the stacked white noise vector asThus, , . Define three criterion
Examples
The nonlinear Wiener system is commonly used in the industry field such as the waste water treatment process. The researchers often purify the waste water by a loop processing and the waste water can be poured into a tank and be purified before a looping process with more waste water. Consider the following Wiener FIR-MA system as the waste water treatment process model:
Conclusions
This paper studies the parameter estimation problems for Wiener FIR-MA feedback systems. A hierarchical recursive least squares algorithm and a hierarchical least squares iterative algorithm are presented to estimate the parameters of the feedback systems directly from the input–output data. The hierarchical methods can demonstrate high computational efficiency in dealing with the identification problem of the Wiener system. The proposed methods can be extended to identify other nonlinear
References (38)
- et al.
Parameter estimation for a multivariable state space system with d-step state-delay
J. Frankl. Inst. – Eng. Appl. Math.
(2013) - et al.
Time series AR modeling with missing observations based on the polynomial transformation
Math. Comput. Modell.
(2010) - et al.
A property of the eigenvalues of the symmetric positive definite matrix and the iterative algorithm for coupled Sylvester matrix equations
J. Frankl. Inst. – Eng. Appl. Math.
(2014) - et al.
An iterative algorithm for the reflexive solutions of the generalized coupled Sylvester matrix equations and its optimal approximation
Appl. Math. Comput.
(2008) - et al.
The general coupled matrix equations over generalized bisymmetric matrices
Linear Algebra Appl.
(2010) - et al.
Maximum likelihood stochastic gradient estimation for Hammerstein systems with colored noise based on the key term separation technique
Comput. Math. Appl.
(2011) - et al.
Least squares based and gradient based iterative identification for Wiener nonlinear systems
Signal Process.
(2011) A blind approach to the Hammerstein–Wiener model identification
Automatica
(2002)An optimal two-stage identification algorithm for Hammerstein–Wiener nonlinear systems
Automatica
(1998)Iterative and recursive least squares estimation algorithms for moving average systems
Simul. Modell. Pract. Theory
(2013)
Recursive estimation for continuous time stochastic volatility models
Appl. Math. Lett.
Recursive least squares estimation algorithm applied to a class of linear-in-parameters output error moving average systems
Appl. Math. Lett.
Recursive identification of errors-in-variables Wiener–Hammerstein systems
Eur. J. Control
Hierarchical gradient based and hierarchical least squares based iterative parameter identification for CARARMA systems
Signal Process.
Auxiliary model based least squares parameter estimation algorithm for feedback nonlinear systems using the hierarchical identification principle
J. Frankl. Inst.
Hierarchical parameter estimation for a class of MIMO Hammerstein systems based on the reframed models
Appl. Math. Lett.
An efficient hierarchical identification method for general dual-rate sampled-data systems
Automatica
Genetic adaptive state estimation with missing input/output data
Proc. Inst. Mech. Eng. Part I – J. Syst. Control Eng.
Nonlinearity identification by time-domain-only signal processing
Int. J. Non-Linear Mech.
Cited by (9)
Computationally efficient identification of continuous-time Lur'e-type systems with stability guarantees
2022, AutomaticaCitation Excerpt :In the system identification literature, Lur’e-type systems are also referred to as nonlinear feedback systems (Giri & Bai, 2010), and NonLinear Linear Fractional Representation (NL-LFR) systems (Schoukens & Tóth, 2020; Vanbeylen, 2013). Different approaches for the identification of Lur’e-type systems have been proposed in literature (Hu & Ding, 2013; Mulders, Vanbeylen, & Usevich, 2014; Paduart, Horvath, & Schoukens, 2004; Schoukens, Nemeth, Crama, Rolain, & Pintelon, 2003; Schoukens & Tiels, 2017; Schoukens & Tóth, 2020; Van Mulders, Schoukens, & Vanbeylen, 2013; Van Pelt & Bernstein, 2001; Vanbeylen, 2013; Wang, Shen, Wu, & Ji, 2016). What most of these approaches have in common is that, as a final step in the identification procedure, a non-convex cost function has to be minimized using gradient-based optimization routines.
Orthotopic-filtering-based fault diagnosis algorithms for nonlinear systems with slowly varying faults
2020, Journal of the Franklin InstituteCitation Excerpt :Mathematical models and their parameter estimation of dynamical systems are basic for control system analysis and design. Some iterative estimation methods have been proposed [29–34] and can be used to identify the parameters of linear and nonlinear stochastic systems [35–38]. Case 1: Fault-free simulation
Robust hierarchical identification of Wiener systems in the presence of dynamic disturbances
2020, Journal of the Franklin InstituteCitation Excerpt :For identification of Wiener-Hammerstein systems with dead-zone, a hierarchical RLS algorithm is proposed by decomposing the model into two subsystems, which can avoid the redundant parameters estimation and reduce the computational burden [38]. A hierarchical RLS algorithm and a hierarchical least squares iterative algorithm are presented for identification of Wiener finite response impulse moving average feedback systems [39]. However, the existing hierarchical identification algorithm cannot be used for identification wiener systems in the presence of the dynamic disturbance
A novel two-stage estimation algorithm for nonlinear Hammerstein–Wiener systems from noisy input and output data
2017, Journal of the Franklin InstituteCitation Excerpt :This method can also be used in solving the parameter estimation problem of other nonlinear systems [19,20], such as Hammerstein and Wiener systems [21–24], block-oriented nonlinear systems [25–27].
Advances in Fault Detection and Diagnosis Using Filtering Analysis
2021, Advances in Fault Detection and Diagnosis Using Filtering AnalysisHierarchical Maximum Likelihood Stochastic Gradient Identification Algorithm for Feedback Nonlinear Systems Using the Data Filtering Technique
2018, Chinese Control Conference, CCC
- ☆
This work was supported in part by the National Natural Science Foundation of China (Nos. 61572237, 61573167), the National High Technology Research and Development Program of China (863 Program) (No. 2014AA041505), and the Fundamental Research Funds for the Central Universities (Nos. JUSRP115A30, JUSRP31106).