Nonlinear system identification using autoregressive quadratic models
Introduction
A linear system driven by a Gaussian i.i.d. (independent identically distributed) noise generally provides a good modeling for stochastic time series and exhibits some properties which allow easy signal analysis. For instance, time series spectral analysis can always be performed by linear modeling, the AR/ARMA models being known to be high-resolution spectral estimators [15], [26]. However, this modeling cannot convey all the time series statistical properties. As detailed in Section 2.1, phase coupling (i.e. a linear relationship between the time series Fourier component phases) cannot be modeled by such systems driven by an i.i.d. noise. Phase coupling analysis is relevant for detecting nonlinear Fourier components (i.e. created by nonlinear interaction since the linear relationship is induced by the system nonlinearity) of the output of a possibly nonlinear system and consequently for identifying artifacts due to the system nonlinearity. As an example, Synthetic Aperture Radar (SAR) images of the sea surface can present some spectral components which do not exist in the original signal (sea surface) and which are generated by the nonlinear SAR process [21]. These “spurious” components have to be clearly identified for a coherent geophysical analysis. Such a problem is often encountered as time series nonlinear modeling and nonlinear system applications are used in many domains: for instance in adaptive control [28], economics [7], biology [18], [19], sonar [34], etc. Our main aim in this paper is to study the possibility of modeling a time series with a time variant (or invariant) Quadratic AutoRegressive Moving Average model which is a special class of NARMA (Nonlinear Autoregressive Moving Average) model introduced by Leontaritis and Billings in [23], [24]. We choose an autoregressive model for several reasons:
- •
The nonlinear (and especially polynomial) autoregressive models have already been studied and some important theoretical points have already been developed (see for instance [1], [2], [7], [11]).
- •
Due to the autoregressive structure of the filter, these systems can be identified with only exact knowledge of the output data and with only few assumptions on the input data.
- •
The parametric approach of the quadratic autogressive moving average model implies that the algorithms presented are high-resolution methods of nonlinear system identification, as linear models are for spectrum estimation [15], [26]. These models thus represent an alternative solution with regard to conventional time-frequency transient nonlinear signal detection (Higher Order Wigner-Ville transform, see [8]). Indeed, some important features can be considered as nonstationary and nonlinear processes, such as the radar signature of an internal wave on the sea surface [3].
The second limitation on the modeling possibilities due to the stability of the model is described in Section 3.2 through a sufficient condition linking the kernel model and the maximum bound of the input data (condition close to the one derived for bilinear systems).
In Section 4, two equation sets are derived under two different statistical hypotheses on the input data. As mentioned above, knowledge of input data is not necessary for quadratic autoregressive moving average model identification unlike some fundamental papers on this subject, Koh and Powers identify the cross correlation of input and output data in [17]. Kim and Powers [16] orthogonalize the input data for identifying Volterra kernels in the frequency domain. The algorithm proposed by Korenberg et al. [25], although using a general nonlinear autoregressive model, also needs the input data knowledge for parameter estimation (also by orthogonalization methods). Swain and Billing [33] identify autoregressive models driven by complex data, but they also need input for their identification algorithm. In case of non-zero exogenous data, these input–output identification methods cannot be used for identifying a nonlinear transfer function, but remains obviously valid for system identification without exogenous input. This paper deals with identification of nonlinear systems driven by unavailable nonzero exogenous data. This theme is a nonlinear blind source separation problem.
After the equation set derivation, we have thus to verify the accuracy of the derived equation sets since these sets use HOS, known to have strong estimation variance. In Section 5.1, we choose to numerically verify this accuracy on the simple problem of recovering quantified real-valued symbols passing through a quadratic autoregressive model. A theoretical study of the system estimation variance remains difficult. The accuracy is verified for both equation sets on the coefficient estimation accuracy as well as on the rate of well recovered symbols. This approach being close to the one used for (minimal phase/nonminimal phase) linear system blind deconvolution (since the input data are noninvolved in the model estimation algorithm), a comparison of the results of the same problem under the hypothesis of nonlinear or linear transmission system is possible. Finally, time series modeling is tested on a Second Order Volterra Model (SOVM) driven by a Gaussian white noise. As shown in Section 3.1, a quadratic autoregressive moving average model cannot be strictly equal to a finite order Volterra model. But such models being able to simulate and describe many nonlinear processes, it is of prime importance to verify (also numerically) the validity of the approximation of finite order Volterra modeling by a quadratic autoregressive moving average model (see Section 5.2). In Section 6, the advantages and limitations of the presented methods are summarized and results obtained on simulations are discussed. Further improvements for time series modeling are also proposed in the same section.
Section snippets
Time series nonlinear modeling
In this first section, we state the problem of the nonlinear modeling of an observed time series. As explained below, the nonlinearity of a random time series is modeled through the phase coupling relation. Let there be the following signal:where A(ωi) is a random coefficient and ϕi a random phase distributed over [0,2π]. If we compute the third-order moment of this signalthen we have to distinguish
Quadratic auto regressive moving average (QARMA) model
The aim of this section is to introduce QARMA models (which are a special subclass of NARMA models) and to study their properties for modeling stochastic time series. If we assume that a stochastic process depends nonlinearly on a finite number of input samples:where n− and n+ are two integers, then we seek to model the nonlinear transfer function as:where Y(n) is the output system, X(n) is a nonzero
Equation sets
The two equation sets presented below have been derived using third-order and fourth-order moments for the first set and from the third up to the sixth order moment for the second set of equations. The equation sets presented in 4.1 Data with finite length correlation, 4.2 Unskewed input data use two different assumptions, respectively, a finite correlation length of the input data and unskewed input data (i.e. with a null third-order moment); the input data are assumed to be zero mean in these
Results
In this section, we develop three examples of QARMA model identification. These simulations allow us to verify the validity of both equation sets and the variance of the nonlinear model coefficient estimates. The first two examples are nonlinear deconvolution of L-PAM symbols having passed through a QARMA model. The third example is the identification of a SOVM driven by a Gaussian white noise. The QARMA identification in the applications mentioned above were made with moments estimated over a
Discussion
From the theoretical point of view, the QARMA system cannot model the output signal of all nonlinear systems. From the different results of 5.1 Nonlinear deconvolution, 5.2 SOVM identification, we can conclude that nonlinear deconvolution is much more difficult than linear. SOVM identification is also verified to be limited. However, the limited assumption on the input data and the simplicity of the methods make the algorithm presented one of the most universal methods of SOVM identification,
Conclusion
In this communication, we have studied time series nonlinear modeling by a QARMA model. From the theoretical point of view we have shown that there are some limitations for this modeling. We have thus derived two linear equation sets for identifying a QARMA model. The assumptions on the input data for deriving both equation sets are general and exact knowledge of these data is not required in the identification algorithm. In the last part of this paper, we have verified the accuracy of these
Acknowledgements
Special thanks to Janet Omrod and Gosette Hemery for their help with writing this paper.
References (36)
- et al.
Weighted complex orthogonal estimator for identifying linear and non-linear continuous time models from generalized frequency response functions
Mech. Systems Signal Process.
(1998) Recursive estimation in nonlinear time series models of autoregressive type
JRSS Ser. B
(1983)- et al.
Nonlinear chaotic systems: approaches and implications for science and engineering-a survey
Appl. Signal Process.
(1995) Theory of radar imaging of internal waves
Nature
(1985)Nonlinear System Analysis and Identification
(1990)- et al.
Representation of non-linear systems: the narmax model
Int. J. Control
(1989) - et al.
Adaptative deconvolution and identification of nonminimum phase FIR systems based on cumulants
IEEE Trans. Automat. Control
(1990) Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom Inflation
Econometrika
(1982)- et al.
Wigner higher order moment spectra: definition, properties, computation and application to transient signal analysis
IEEE Trans. Signal Process.
(1993) Cumulants: a powerful tool in signal processing
Proc. IEEE
(1987)
On estimating noncausal nonminimum phase ARMA models of non-gaussian processes
IEEE Trans. Acoust. Speech Signal Process.
A study of application of modeling nonlinear random vibrations using an amplitude time dependent autoregressive time series models
Biometrika
On the nonlinear mapping of an ocean wave spectrum into a synthetic aperture radar image spectrum and its inversion
J. Geophys. Res.
Blind equalization using a tricepstrum-based algorithm
IEEE Trans. Commun.
Adaptative Filter Theory
Modern Spectral Estimation, Theory and Application
A digital method of modeling quadratically nonlinear systems with a general random input
IEEE Trans. Acoust. Speech Signal Process.
Second-order volterra filtering and its application to nonlinear system identification
IEEE Trans. Acoust. Speech Signal Process.
Cited by (20)
Threshold autoregressive model blind identification based on array clustering
2021, Signal ProcessingFusion of hypothesis testing for nonlinearity detection in small time series
2013, Signal ProcessingSpectral inversion of second order Volterra models based on the blind identification of Wiener models
2011, Signal ProcessingCitation Excerpt :The practical implementation is also limited to systems with few coefficients. In [27], the author proposes the identification of second order Volterra models (SOVMs) (and then the inversion as seen in Section 4), using NARMA (nonlinear autoregressive moving average) models. However, Volterra models with a finite number of kernels cannot strictly be equivalent to NARMA models with a finite number of kernels.
Time series nonlinearity modeling: A Giannakis formula type approach
2003, Signal ProcessingAnalysis of the sar imaging process of the ocean surface using volterra models
2002, IEEE Journal of Oceanic Engineering