Hierarchical least squares algorithms for nonlinear feedback system modeling

https://doi.org/10.1016/j.jfranklin.2016.03.015Get rights and content

Abstract

A hierarchical recursive least squares algorithm and a hierarchical least squares iterative algorithm are presented for Wiener feedback finite impulse response moving average model. By combining the least squares idea and hierarchical principle, the finite impulse response moving average model can be decomposed into three subsystems. The simulation results illustrate the effectiveness of the proposed two algorithms.

Introduction

Recently, identification methods have been widely adopted in signal processing [1], [2], [3], system modeling [4], [5], [6] and solving matrix problems [7], [8], [9], [10], [11]. For example, Li and Ding considered the identification problem of Hammerstein finite impulse response moving average system and presented a maximum likelihood multi-innovation stochastic gradient algorithm by using the maximum likelihood principle and stochastic gradient method based on the key term separation technique [12].

A Wiener model can be described as a linear dynamic block followed by a static nonlinear function [13], [14]. Some researches have been taken in the identification issues of Wiener models. Main contribution includes, i.e., Wang and Ding derived a least squares-based algorithm and a gradient-based iterative identification algorithm for Wiener nonlinear systems by separating one bilinear cost function into two linear cost functions without re-parameterization to generate redundant estimates [15]; Bai presented a blind approach to the sampled Hammerstein–Wiener model identification by using the blind approach [16] and proposed an optimal two-stage identification algorithm for Hammerstein–Wiener systems [17]. In the identification area of nonlinear systems, recursive least squares algorithms have attracted much attention [18], [19], [20], [21], [22], [23]. For example, Ding et al. derived a recursive least squares algorithm for estimating the parameters of the nonlinear systems based on the model decomposition [24].

Wang and Tang developed an auxiliary model based recursive least squares algorithm for identifying the parameters of the linear-in-parameters output error moving average system [25]; Mu and Chen derived a stochastic approximation algorithm incorporated with the deconvolution kernel functions for errors-in-variables Wiener–Hammerstein system [26]. Many researchers focus on deriving the online identification methods, such as Wang and Zhang proposed an improved least squares algorithm to identify the parameters of the multivariable Hammerstein system by using the Taylor expansion on a least squares quadratic criterion function [27]; Ding et al. proposed a two-stage gradient based algorithm and a least squares based iterative parameter estimation algorithm for controlled autoregressive moving average systems [28]; Hu and Ding investigated a multistage least squares based iterative algorithm to estimate the parameters of nonlinear systems with moving average noise from input–output data [29], but the proposed least squares based iterative algorithm in [28], [29] is widely used in offline identification field and the first step of running the iterative algorithm is collecting all the input–output data at once.

The hierarchical method is emerging as a type of decomposition based identification tool in recent years. For instance, Wang et al. derived a decomposition-based recursive least squares algorithm and combined the decomposition principle and the least squares method after using the filtering idea [30], but the proposed algorithm in [30] is not a direct identification method that needs two different steps, the first is to transform the finite impulse response moving average model to a controlled autoregressive model, then the least squares algorithm can be used. Thus, the method in [30] has an intricate data processing. Hu et al. derived an auxiliary model based least squares parameter estimation algorithm [31], however, this method is still a two-stage algorithm and can only be used in the online identification.

Compared with the works in [30], [31], the main contribution of this paper is to propose a hierarchical recursive least squares algorithm that avoids the complex steps of filtering algorithm and to derive a hierarchical least squares iterative algorithm which is computationally efficient and can be used in the offline identification field, such as the waste water treatment process and other industry circle.

The rest of this paper is organized as follows. Section 2 gives a representation of a finite response impulse moving average Wiener nonlinear feedback system. Section 3 presents a hierarchical recursive least squares algorithm for the proposed Wiener feedback system. Section 4 derives a hierarchical least squares iterative algorithm. Section 5 provides an illustrative example for the results in this paper. Finally, some concluding remarks are offered in Section 6.

Section snippets

Problem formulation

Consider the following Wiener system depicted in Fig. 1:y(t)=B(z)u(t)+D(z)υ(t),u(t)=r(t)y¯(t),y¯(t)=f(y(t))=i=1nccifi(y(t)),where the open-loop part is a finite response impulse moving average (FIR-MA) subsystem, r(t) and y(t) are the reference input and output sequences of the Wiener system, v(t) is the stochastic white noise with zero mean and variance σ2, B(z) and D(z) are polynomials in the unit backward shift operator z−1 [i.e. z1y(t)=y(t1)]: B(z)b1z1+b2z2++bnbznb,D(z)1+d1z1+d2z

The hierarchical recursive least squares algorithm

To proceed further, here some notations are denoted. For convenience, t represents the current time and θ^(t) is regarded as the estimate of θ at time t; “AX” or “XA” denotes “A is defined as X”; E stands for the expectation operator; the symbol I(In) is defined as an identity matrix of appropriate sizes (n×n); X is denoted as the norm of a matrix or a column vector and defined by X2tr[XXT].

From Eq. (1), the identification model can be rewritten asy(t)=B(z)u(t)+D(z)υ(t)=B(z)[r(t)y¯(t)]+D(z

The hierarchical least squares iterative algorithm

Assume that the data length Ln and define the stacked output vectors Y,Y1,Y2 and Y3, the stacked information matrices Φ1,Φ2 and Φn, and the stacked white noise vector V asY[y(1),y(2),,y(L)]TRL,Y1[y1(1),y1(2),,y1(L)]TRL,Y2[y2(1),y2(2),,y2(L)]TRL,Y3[y3(1),y3(2),,y3(L)]TRL,Φ1[φ1(1),φ1(2),,φ1(L)]TRL×nb,Φ2[φ2(1),φ2(2),,φ2(L)]TRL×nc,Φn[φn(1),φn(2),,φn(L)]TRL×nd,Φs[φs(1),φs(2),,φs(L)]TRL×nb,V[υ(1),υ(2),,υ(L)]RL.Thus, Y1Φ1b+V,Y2Φ2c+V, Y3Φnd+V. Define three criterion

Examples

The nonlinear Wiener system is commonly used in the industry field such as the waste water treatment process. The researchers often purify the waste water by a loop processing and the waste water can be poured into a tank and be purified before a looping process with more waste water. Consider the following Wiener FIR-MA system as the waste water treatment process model:y(t)=B(z)[r(t)y¯(t)]+D(z)υ(t),y¯(t)=c1f1(y(t))+c2f2(y(t))=0.6sin2(y(t))+0.8sin3(y(t)),B(z)=b1z1+b2z2=1.617z1+1.05z2,D(z)=1

Conclusions

This paper studies the parameter estimation problems for Wiener FIR-MA feedback systems. A hierarchical recursive least squares algorithm and a hierarchical least squares iterative algorithm are presented to estimate the parameters of the feedback systems directly from the input–output data. The hierarchical methods can demonstrate high computational efficiency in dealing with the identification problem of the Wiener system. The proposed methods can be extended to identify other nonlinear

References (38)

  • H. Gong et al.

    Recursive estimation for continuous time stochastic volatility models

    Appl. Math. Lett.

    (2009)
  • C. Wang et al.

    Recursive least squares estimation algorithm applied to a class of linear-in-parameters output error moving average systems

    Appl. Math. Lett.

    (2014)
  • B.Q. Mu et al.

    Recursive identification of errors-in-variables Wiener–Hammerstein systems

    Eur. J. Control

    (2014)
  • F. Ding et al.

    Hierarchical gradient based and hierarchical least squares based iterative parameter identification for CARARMA systems

    Signal Process.

    (2014)
  • P.P. Hu et al.

    Auxiliary model based least squares parameter estimation algorithm for feedback nonlinear systems using the hierarchical identification principle

    J. Frankl. Inst.

    (2013)
  • D.Q. Wang

    Hierarchical parameter estimation for a class of MIMO Hammerstein systems based on the reframed models

    Appl. Math. Lett.

    (2016)
  • Y.J. Liu et al.

    An efficient hierarchical identification method for general dual-rate sampled-data systems

    Automatica

    (2014)
  • H. Fang et al.

    Genetic adaptive state estimation with missing input/output data

    Proc. Inst. Mech. Eng. Part I – J. Syst. Control Eng.

    (2010)
  • P.F. Pai et al.

    Nonlinearity identification by time-domain-only signal processing

    Int. J. Non-Linear Mech.

    (2013)
  • Cited by (9)

    • Computationally efficient identification of continuous-time Lur'e-type systems with stability guarantees

      2022, Automatica
      Citation Excerpt :

      In the system identification literature, Lur’e-type systems are also referred to as nonlinear feedback systems (Giri & Bai, 2010), and NonLinear Linear Fractional Representation (NL-LFR) systems (Schoukens & Tóth, 2020; Vanbeylen, 2013). Different approaches for the identification of Lur’e-type systems have been proposed in literature (Hu & Ding, 2013; Mulders, Vanbeylen, & Usevich, 2014; Paduart, Horvath, & Schoukens, 2004; Schoukens, Nemeth, Crama, Rolain, & Pintelon, 2003; Schoukens & Tiels, 2017; Schoukens & Tóth, 2020; Van Mulders, Schoukens, & Vanbeylen, 2013; Van Pelt & Bernstein, 2001; Vanbeylen, 2013; Wang, Shen, Wu, & Ji, 2016). What most of these approaches have in common is that, as a final step in the identification procedure, a non-convex cost function has to be minimized using gradient-based optimization routines.

    • Orthotopic-filtering-based fault diagnosis algorithms for nonlinear systems with slowly varying faults

      2020, Journal of the Franklin Institute
      Citation Excerpt :

      Mathematical models and their parameter estimation of dynamical systems are basic for control system analysis and design. Some iterative estimation methods have been proposed [29–34] and can be used to identify the parameters of linear and nonlinear stochastic systems [35–38]. Case 1: Fault-free simulation

    • Robust hierarchical identification of Wiener systems in the presence of dynamic disturbances

      2020, Journal of the Franklin Institute
      Citation Excerpt :

      For identification of Wiener-Hammerstein systems with dead-zone, a hierarchical RLS algorithm is proposed by decomposing the model into two subsystems, which can avoid the redundant parameters estimation and reduce the computational burden [38]. A hierarchical RLS algorithm and a hierarchical least squares iterative algorithm are presented for identification of Wiener finite response impulse moving average feedback systems [39]. However, the existing hierarchical identification algorithm cannot be used for identification wiener systems in the presence of the dynamic disturbance

    • A novel two-stage estimation algorithm for nonlinear Hammerstein–Wiener systems from noisy input and output data

      2017, Journal of the Franklin Institute
      Citation Excerpt :

      This method can also be used in solving the parameter estimation problem of other nonlinear systems [19,20], such as Hammerstein and Wiener systems [21–24], block-oriented nonlinear systems [25–27].

    • Advances in Fault Detection and Diagnosis Using Filtering Analysis

      2021, Advances in Fault Detection and Diagnosis Using Filtering Analysis
    View all citing articles on Scopus

    This work was supported in part by the National Natural Science Foundation of China (Nos. 61572237, 61573167), the National High Technology Research and Development Program of China (863 Program) (No. 2014AA041505), and the Fundamental Research Funds for the Central Universities (Nos. JUSRP115A30, JUSRP31106).

    View full text