Nonlinear system identification using quasi-perfect periodic sequences

Highlights

• The paper discusses functional link polynomial (FLiP) nonlinear filters.

• Quasi-perfect periodic sequences (QPPSs) are introduced for their identification.

• QPPSs have discrete level samples and block-diagonal sparse autocorrelation matrix.

• A simple and fast combinatorial rule is provided for generating the QPPSs.

Abstract

Perfect periodic sequences are currently used for modeling linear and nonlinear systems. A periodic sequence, applied as input to a linear or nonlinear system, is called perfect if the basis functions of the modeling filter are orthogonal to each other, and thus the auto-correlation matrix is diagonal. In this paper, we introduce quasi-perfect periodic sequences for a sub-class of linear-in-the-parameters nonlinear filters, called functional link polynomial filters, which is derived by using the constructive rule of Volterra filters. A periodic sequence is defined as quasi-perfect for a nonlinear filter if the resulting auto-correlation matrix is block-diagonal and highly sparse. Moreover, the samples of the sequence are represented by only a few discrete levels. It is shown in the paper that quasi-perfect periodic sequences for third-order systems can be obtained by means of a simple combinatorial rule. The derived sequences, which are the same for all functional link polynomial filters, allow an efficient implementation of the least-squares approximation method. Simulation results and a real-world experiment show good performance of the proposed identification method.

Introduction

Identification methods are usually based on the choice of a linear combination of basis functions that are able to represent the unknown system within a given accuracy. If the basis functions are all mutually orthogonal for appropriate input signals, then one of the simplest and most effective identification methods is the so-called cross-correlation method [1]. Indeed, in this situation, the parameters of the model can be identified by computing the cross-correlations between each basis function and the output of the unknown system. The drawbacks of the method, when using stochastic inputs, are the required accuracy of the properties of the input signal, and, most importantly, the very high number of input samples necessary to find reliable approximations. To overcome this difficulty, it is possible to resort to some deterministic signal able to guarantee the orthogonality of the basis functions on a finite interval. As a matter of fact, in the field of linear filters, perfect periodic sequences (PPSs) have been exploited, as shown in [2], [3], [4]. A periodic sequence is perfect for a given filter if all cross-correlations between two different basis functions, estimated over a period of N samples, are equal to zero. In this case, the linear basis functions $x(n-i)$, with i ranging from 0 to $N-1$, form an orthogonal set. It has been shown in the literature that PPSs guarantee the optimization of the convergence speed of the normalized least mean-square (NLMS) algorithm [5], [6]. In this case, without output noise, the NLMS algorithm is able to identify a linear system within N samples when excited by a PPS of period N. The approach has been then extended to the identification of multichannel linear systems [7], [8]. In [9], identification algorithms using PPSs that require only a multiplication, an addition and a subtraction per sample have been developed. These algorithms have been extended in [10] to a generic periodic sequence, within the constraint that its discrete Fourier transform has no zero components. Other examples of application of PPSs can be found in the area of information theory [11], [12], communications [13], [14], [15], [16] and acoustics [5], [17].

Periodic sequences have been also considered for the identification of nonlinear systems, as for example, Volterra filters. The Volterra filter is the most popular member of the class of nonlinear filters, characterized by the property that their outputs depend linearly on the filter coefficients, called linear-in-the-parameters (LIP) nonlinear filters. LIP nonlinear filters constitute a class of great interest since, according to their defining property, they admit an optimal least-squares (LS) solution of the approximation problem. Moreover, it is possible to extend the adaptation algorithms valid for linear filters to any member of the LIP class. As an example, a comprehensive approach to nonlinear regression yielding an overall piecewise linear regressor has been considered in [18]. In contrast to other members of the LIP class, Volterra filters are able to arbitrarily well approximate any causal, time-invariant, finite-memory, continuous, nonlinear system, according to the Stone–Weierstrass theorem [19], and thus can be considered as universal approximators. However, their basis functions, formed by products of delayed input samples, are in general not mutually orthogonal. In fact, the basis functions become orthogonal for Gaussian input signals only after the orthogonalization procedure due to Wiener. The interest for Volterra filters is documented even by recent contributions on efficient model computation [20], [21], [22], [23], [24], [25], [26], [27], [28], nonlinear active noise control [29], [30], [31], [32], nonlinear echo cancellation [33], [34], [35], [36], [37], among others. With reference to periodic sequences, pseudorandom multilevel periodic sequences have been studied in [38] and used for the least-square estimation of the coefficients of Volterra and extended Volterra filters. A Wiener model has been estimated in [39] using a multilevel sequence. Periodic sequences have been applied in [40], [41] to derive an efficient algorithm for the identification of LIP nonlinear filters.

Recently, two new members of the class of LIP nonlinear filters have been introduced, the even mirror Fourier nonlinear (EMFN) filter [42], [43], [44] and the Legendre filter [45], [46]. The first one has been so-called since its basis functions are even mirror symmetric, as the waveforms defining the discrete cosine transform, while the second one is based on Legendre polynomials. It has been shown in [42], [43], [45], [46] that both filters are universal approximators, as the Volterra filter, for causal, time-invariant, finite-memory, continuous, nonlinear systems, according to the Stone–Weierstrass theorem. Moreover, in contrast to Volterra filters, their basis functions are mutually orthogonal for white uniform input signals in the interval $[-1,+1]$. As a consequence, EMFN and Legendre models can be simply estimated by means of the cross-correlation method. Moreover, it has been argued that deterministic quasi-uniform sequences able to guarantee the same orthogonality property on a finite period can also exist. Indeed, it has been shown in [46], [47], [48], [49] that PPSs can be developed for EMFN and Legendre filters, so that the cross-correlation method can be directly applied to the identification of nonlinear systems, avoiding the use of long stochastic input sequences. The PPSs are computed by solving an undetermined system of nonlinear equations involving the EMFN or Legendre basis functions, using the Newton–Raphson method. Since the number of equations increases exponentially with the order P and geometrically with the memory N of the filter, only kernels of order $P=0,1,2$ and 3 are usually considered. It has been also shown in [48] that the number of nonlinear equations can be reduced exploiting symmetry and oddness conditions. As a consequence, the Newton–Raphson method becomes feasible, but at the expense of an increased length L of the PPS. It is worth noting that, in general, a PPS for an EMFN filter is not a PPS for a Legendre filter, and viceversa. However, by using a larger system of equations, and thus increasing the period of the PPS, it is also possible to obtain sequences that are perfect for both filters. PPSs for EMFN and Legendre filters of order P=2 and P=3 are available at [50].

In this paper, we consider an alternative strategy for the generation of periodic sequences for nonlinear system identification. These sequences, in contrast to PPSs, are the same not only for EMFN and Legendre filters but also for all filters whose basis functions are derived by applying the multiplicative rule of Volterra filters. These filters constitute a sub-class of the LIP filters and here are called functional link polynomial (FLiP) filters, since their basis functions are polynomials of nonlinear expansions of delayed input samples. An interesting property of these filters is their ability to arbitrarily well approximate any causal, time-invariant, finite-memory, continuous, nonlinear system, whose input–output relationship can be expressed by a nonlinear function f of the N most recent input samples.

Motivated by the fact that previous research studies for linear and Volterra filters, as indicated above, often deal with multilevel sequences, we assume as our goal the derivation of periodic sequences composed of only a few levels. This constraint is useful for the implementation of identification procedures exploiting finite precision arithmetic. To reach this objective, however, we need to relax some of the properties that characterize PPSs. More specifically, we define a quasi-perfect periodic sequence (QPPS) as a sequence characterized by an auto-correlation matrix that is close to the diagonal one that characterizes a PPS sequence. In our case, the auto-correlation matrix of the proposed QPPSs has a highly sparse diagonal block structure. The QPPSs are generated according to a simple rule, so avoiding the need to solve a system of nonlinear equations. As mentioned above, the QPPSs are the same for all FLiP filters. However, since the auto-correlation matrix of QPPSs is no longer diagonal, the cross-correlation method cannot be exploited for system identification. Alternatively, we need to resort to the optimal solution of an LS problem. Therefore, the inverse of the auto-correlation matrix should be computed. However, since the input signal is deterministic, the auto-correlation matrix and its inverse need to be computed only once. This calculation is not computationally intensive because of the sparsity of the auto-correlation matrix. In addition, its entries can be obtained using small look-up tables for the computation of the values of the basis functions. It is worth noting that QPPSs offer an exact LS solution with a mean-square error (MSE) almost coincident with that of the infinite-precision PPSs, whereas the PPSs of [46], [47], [48], [49], rounded to the finite precision of even eight or sixteen bits, give a remarkably larger MSE. On the other hand, QPPSs cannot be used to rank the basis functions according to given information criteria as PPSs [48], unless impractical orthogonalization procedures are applied.

The paper is organized as follows. In Section 2, definition and basic notions on the FLiP filters are given. In Section 3, the generation rule of the QPPSs for the FLiP filters, of order P up to 3, is introduced, and further relevant details and properties are discussed. Experimental results are presented in Section 4. Conclusions follow in Section 5.

The following notation is used throughout the paper. Intervals are represented with square brackets, $\mathbb{N}$ is the set of natural numbers, $\mathbb{R}$ the set of real numbers, ${\mathbb{R}}_1$ is the unit interval $[-1,+1]$, ${〈x\left(n\right)〉}_L$ indicates time average over L successive samples of x(n), $\left(\genfrac{}{}{0em}{}{n}{m}\right)$ denotes the number of combinations of n things taken m at a time, the operator $\lceil ·\rceil$ (ceiling) indicates the smallest integer greater than or equal to the real argument.