Elsevier

Signal Processing

Volume 81, Issue 2, February 2001, Pages 357-379
Signal Processing

Nonlinear system identification using autoregressive quadratic models

https://doi.org/10.1016/S0165-1684(00)00213-9Get rights and content

Abstract

In this paper, we study the identification of a special class of nonlinear systems, Quadratic AutoRegressive Moving Average systems, QARMA. In the first part, we discuss the relationship between this model and the Volterra models and also the property of stability of these systems. The second part is devoted to the derivation of the two equation sets needed for a possibly time-variant QARMA identification. The equation sets use higher-order moments and the first set is derived under the assumption of finite length correlation of the input data. The coefficients of this first system depend on a mixed set of third- and fourth-order moments. The second set of equations assumes only unskewed input data and the equation coefficients are a linear combination of moments from the third up to the sixth order with the system coefficients at previous lags. In order to validate the identification methods and to numerically verify the accuracy of the estimated coefficients for both equation sets, the QARMA methods were applied to the deconvolution of L-PAM symbols, the rate of good estimation of these symbols allowing a numerical comparison between the respective performances of both equation sets. Another application presented in this paper is a Second-Order Volterra Model (SOVM) identification although the QARMA model cannot be strictly equal to a SOVM.

Introduction

A linear system driven by a Gaussian i.i.d. (independent identically distributed) noise generally provides a good modeling for stochastic time series and exhibits some properties which allow easy signal analysis. For instance, time series spectral analysis can always be performed by linear modeling, the AR/ARMA models being known to be high-resolution spectral estimators [15], [26]. However, this modeling cannot convey all the time series statistical properties. As detailed in Section 2.1, phase coupling (i.e. a linear relationship between the time series Fourier component phases) cannot be modeled by such systems driven by an i.i.d. noise. Phase coupling analysis is relevant for detecting nonlinear Fourier components (i.e. created by nonlinear interaction since the linear relationship is induced by the system nonlinearity) of the output of a possibly nonlinear system and consequently for identifying artifacts due to the system nonlinearity. As an example, Synthetic Aperture Radar (SAR) images of the sea surface can present some spectral components which do not exist in the original signal (sea surface) and which are generated by the nonlinear SAR process [21]. These “spurious” components have to be clearly identified for a coherent geophysical analysis. Such a problem is often encountered as time series nonlinear modeling and nonlinear system applications are used in many domains: for instance in adaptive control [28], economics [7], biology [18], [19], sonar [34], etc. Our main aim in this paper is to study the possibility of modeling a time series with a time variant (or invariant) Quadratic AutoRegressive Moving Average model which is a special class of NARMA (Nonlinear Autoregressive Moving Average) model introduced by Leontaritis and Billings in [23], [24]. We choose an autoregressive model for several reasons:

  • The nonlinear (and especially polynomial) autoregressive models have already been studied and some important theoretical points have already been developed (see for instance [1], [2], [7], [11]).

  • Due to the autoregressive structure of the filter, these systems can be identified with only exact knowledge of the output data and with only few assumptions on the input data.

  • The parametric approach of the quadratic autogressive moving average model implies that the algorithms presented are high-resolution methods of nonlinear system identification, as linear models are for spectrum estimation [15], [26]. These models thus represent an alternative solution with regard to conventional time-frequency transient nonlinear signal detection (Higher Order Wigner-Ville transform, see [8]). Indeed, some important features can be considered as nonstationary and nonlinear processes, such as the radar signature of an internal wave on the sea surface [3].

Section 3 presents the theoretical limitations of quadratic autoregressive moving average models for time series modeling and especially for Volterra system output modeling. Volterra models are quite common nonlinear systems and many nonlinear processes can be modeled by such systems [4], [19], [32]. These models are indeed a generalization of the Taylor expansion of general nonlinear transfer function. However the output of these models being a polynomial of input data, exact knowledge of the input data is needed for estimating Volterra kernels. In some applications, these data are not available (radar, sonar) and nonlinear autoregressive models are an alternative solution for time series nonlinear modeling. The expression of the Volterra models corresponding to the quadratic autoregressive moving average models given in Section 3.1 allows the autoregressive models to be linked with previous works and to specify the time series modeling limitations by defining the structure of Volterra models which can be approximated by a quadratic autoregressive moving average model.

The second limitation on the modeling possibilities due to the stability of the model is described in Section 3.2 through a sufficient condition linking the kernel model and the maximum bound of the input data (condition close to the one derived for bilinear systems).

In Section 4, two equation sets are derived under two different statistical hypotheses on the input data. As mentioned above, knowledge of input data is not necessary for quadratic autoregressive moving average model identification unlike some fundamental papers on this subject, Koh and Powers identify the cross correlation of input and output data in [17]. Kim and Powers [16] orthogonalize the input data for identifying Volterra kernels in the frequency domain. The algorithm proposed by Korenberg et al. [25], although using a general nonlinear autoregressive model, also needs the input data knowledge for parameter estimation (also by orthogonalization methods). Swain and Billing [33] identify autoregressive models driven by complex data, but they also need input for their identification algorithm. In case of non-zero exogenous data, these input–output identification methods cannot be used for identifying a nonlinear transfer function, but remains obviously valid for system identification without exogenous input. This paper deals with identification of nonlinear systems driven by unavailable nonzero exogenous data. This theme is a nonlinear blind source separation problem.

After the equation set derivation, we have thus to verify the accuracy of the derived equation sets since these sets use HOS, known to have strong estimation variance. In Section 5.1, we choose to numerically verify this accuracy on the simple problem of recovering quantified real-valued symbols passing through a quadratic autoregressive model. A theoretical study of the system estimation variance remains difficult. The accuracy is verified for both equation sets on the coefficient estimation accuracy as well as on the rate of well recovered symbols. This approach being close to the one used for (minimal phase/nonminimal phase) linear system blind deconvolution (since the input data are noninvolved in the model estimation algorithm), a comparison of the results of the same problem under the hypothesis of nonlinear or linear transmission system is possible. Finally, time series modeling is tested on a Second Order Volterra Model (SOVM) driven by a Gaussian white noise. As shown in Section 3.1, a quadratic autoregressive moving average model cannot be strictly equal to a finite order Volterra model. But such models being able to simulate and describe many nonlinear processes, it is of prime importance to verify (also numerically) the validity of the approximation of finite order Volterra modeling by a quadratic autoregressive moving average model (see Section 5.2). In Section 6, the advantages and limitations of the presented methods are summarized and results obtained on simulations are discussed. Further improvements for time series modeling are also proposed in the same section.

Section snippets

Time series nonlinear modeling

In this first section, we state the problem of the nonlinear modeling of an observed time series. As explained below, the nonlinearity of a random time series is modeled through the phase coupling relation. Let there be the following signal:Y(n)=A(ω1)cos1n+ϕ1)+A(ω2)cos2n+ϕ2)+A(ω21)cos((ω12)n+ϕ3)where A(ωi) is a random coefficient and ϕi a random phase distributed over [0,2π]. If we compute the third-order moment of this signalM3Y(n1,n2)=E{Y(n)Y(n+n1)Y(n+n2)}then we have to distinguish

Quadratic auto regressive moving average (QARMA) model

The aim of this section is to introduce QARMA models (which are a special subclass of NARMA models) and to study their properties for modeling stochastic time series. If we assume that a stochastic process depends nonlinearly on a finite number of input samples:Y(n)=F(X(n+n),…,X(n+n+),n)where n and n+ are two integers, then we seek to model the nonlinear transfer function as:i=qq+bi(n)X(n)+N(n)=i=0pai(n)Y(n−i)+i=0rj=0rcij(n)Y(n−i)Y(n−j)where Y(n) is the output system, X(n) is a nonzero

Equation sets

The two equation sets presented below have been derived using third-order and fourth-order moments for the first set and from the third up to the sixth order moment for the second set of equations. The equation sets presented in 4.1 Data with finite length correlation, 4.2 Unskewed input data use two different assumptions, respectively, a finite correlation length of the input data and unskewed input data (i.e. with a null third-order moment); the input data are assumed to be zero mean in these

Results

In this section, we develop three examples of QARMA model identification. These simulations allow us to verify the validity of both equation sets and the variance of the nonlinear model coefficient estimates. The first two examples are nonlinear deconvolution of L-PAM symbols having passed through a QARMA model. The third example is the identification of a SOVM driven by a Gaussian white noise. The QARMA identification in the applications mentioned above were made with moments estimated over a

Discussion

From the theoretical point of view, the QARMA system cannot model the output signal of all nonlinear systems. From the different results of 5.1 Nonlinear deconvolution, 5.2 SOVM identification, we can conclude that nonlinear deconvolution is much more difficult than linear. SOVM identification is also verified to be limited. However, the limited assumption on the input data and the simplicity of the methods make the algorithm presented one of the most universal methods of SOVM identification,

Conclusion

In this communication, we have studied time series nonlinear modeling by a QARMA model. From the theoretical point of view we have shown that there are some limitations for this modeling. We have thus derived two linear equation sets for identifying a QARMA model. The assumptions on the input data for deriving both equation sets are general and exact knowledge of these data is not required in the identification algorithm. In the last part of this paper, we have verified the accuracy of these

Acknowledgements

Special thanks to Janet Omrod and Gosette Hemery for their help with writing this paper.

References (36)

  • A.K. Swain et al.

    Weighted complex orthogonal estimator for identifying linear and non-linear continuous time models from generalized frequency response functions

    Mech. Systems Signal Process.

    (1998)
  • K.K. Aase

    Recursive estimation in nonlinear time series models of autoregressive type

    JRSS Ser. B

    (1983)
  • L.A. Aguirre et al.

    Nonlinear chaotic systems: approaches and implications for science and engineering-a survey

    Appl. Signal Process.

    (1995)
  • W.R. Alpers

    Theory of radar imaging of internal waves

    Nature

    (1985)
  • J.S. Bendat

    Nonlinear System Analysis and Identification

    (1990)
  • S. Chen et al.

    Representation of non-linear systems: the narmax model

    Int. J. Control

    (1989)
  • H-H. Chiang et al.

    Adaptative deconvolution and identification of nonminimum phase FIR systems based on cumulants

    IEEE Trans. Automat. Control

    (1990)
  • R.F. Engle

    Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom Inflation

    Econometrika

    (1982)
  • J.R. Fonollosa et al.

    Wigner higher order moment spectra: definition, properties, computation and application to transient signal analysis

    IEEE Trans. Signal Process.

    (1993)
  • G.B. Giannakis

    Cumulants: a powerful tool in signal processing

    Proc. IEEE

    (1987)
  • G.B. Giannakis et al.

    On estimating noncausal nonminimum phase ARMA models of non-gaussian processes

    IEEE Trans. Acoust. Speech Signal Process.

    (1990)
  • V. Haggan et al.

    A study of application of modeling nonlinear random vibrations using an amplitude time dependent autoregressive time series models

    Biometrika

    (1981)
  • K. Hasselmann et al.

    On the nonlinear mapping of an ocean wave spectrum into a synthetic aperture radar image spectrum and its inversion

    J. Geophys. Res.

    (1991)
  • D. Hatzanikos et al.

    Blind equalization using a tricepstrum-based algorithm

    IEEE Trans. Commun.

    (1991)
  • S. Haykin

    Adaptative Filter Theory

    (1986)
  • S.M. Kay

    Modern Spectral Estimation, Theory and Application

    (1988)
  • K.I. Kim et al.

    A digital method of modeling quadratically nonlinear systems with a general random input

    IEEE Trans. Acoust. Speech Signal Process.

    (1988)
  • T. Koh et al.

    Second-order volterra filtering and its application to nonlinear system identification

    IEEE Trans. Acoust. Speech Signal Process.

    (1985)
  • Cited by (20)

    • Spectral inversion of second order Volterra models based on the blind identification of Wiener models

      2011, Signal Processing
      Citation Excerpt :

      The practical implementation is also limited to systems with few coefficients. In [27], the author proposes the identification of second order Volterra models (SOVMs) (and then the inversion as seen in Section 4), using NARMA (nonlinear autoregressive moving average) models. However, Volterra models with a finite number of kernels cannot strictly be equivalent to NARMA models with a finite number of kernels.

    View all citing articles on Scopus
    View full text