Time delay Chebyshev functional link artificial neural network

Abstract

In real applications, a time delay in the parameter update of the neural network is sometimes required. In this paper, motivated by the Chebyshev functional link artificial neural network (CFLANN), a new structure based on the time delay adaptation is developed for nonlinear system identification. Particularly, we present a new CFLANN-delayed recursive least square (CFLANN-DRLS), as an online learning algorithm for parameter adaptation in CFLANN. The CFLANN-DRLS algorithm exploits the time delayed error signal with the gain vector delayed by D cycles to form the weight increment term, which provides potential implementation in the filter with pipelined structure. However, it suffers from the instability problems under imperfect network delay estimate. To overcome this problem, we further propose a modified CFLANN-DRLS (CFLANN-MDRLS) algorithm by including a compensation term to the error signal. We analyze the stability and convergence of the proposed algorithm. Simulations in nonlinear system identification contexts reveal that the newly proposed CFLANN-MDRLS algorithm can effectively compensate the time delay of system and it is even superior to the algorithm without delay in some cases.

Introduction

System identification is the art and science of establishing a mathematical model for an unknown system through the input–output relationship, which plays a significant role in control theory [1], [2]. In previous studies, many approaches have been proposed in the context of linear system identification [3], [4]. However, the real-world system is not always linear. For instance, different types of artificial noise in active noise control systems [5], [6], [7] and impulsive noises in communication systems [8], [9], can be described more accurately using nonlinear models. In these cases, linear system identification technique performs poorly. Therefore, it is reasonable to take the nonlinear system identification technique into account.

Several nonlinear approaches have been proposed to identify nonlinear dynamic systems [10], [11], [12], [13]. Among these, the artificial neural network (ANN) has attracted more and more attention because of the learning ability and excellent approximating performance [13], [14]. Most of the ANN-based system identification techniques are based on the feedforward networks such as the radial basis function neural networks (RBFNNs) [15], [16], [17] and the multilayer perceptron (MLP) trained with backpropagation (BP) [18], [19], [20], [21]. In [15], an effective procedure based on the RBFNN was proposed to detect the harmonic amplitudes of the measured signal. However, the main bottleneck of the RBFNNs is their essentially static input-to-output maps, which reduces the ability of modeling nonlinear systems [22], [23]. The algorithm in [21] was developed for estimating carbon price of European Union Emissions Trading Scheme. Although the reliable performance of the MLP network model is achieved, such neural network does not consider the time delay in network and compensate the delay for performance improvement.

As an effective alternative to the multilayer ANN, the functional link ANN (FLANN) was initially proposed on neural networks [24]. By removing the hidden layer, the FLANN becomes a single layer network. It employs the point-wise functional expansions of the current input pattern and then it generates the output by linear combination. Previous studies have shown that the FLANN can simplify the learning algorithms and unify the architecture for all types of networks [25]. So far, a large number of FLANN algorithms have been proposed based upon different basis functions [5], [26], [27], [28]. By making use of the Fourier expansions, even mirror Fourier nonlinear (EMFN) algorithm was introduced in [27], which accelerates the convergence rate as compared with its Volterra expansion counterpart. The trigonometric polynomial is another popular expansion, which is an effective means for nonlinear active noise control (NANC) and provides a computational advantage over existing algorithms [5], [29]. Furthermore, many orthogonal basis functions, such as Hermite polynomial [30], Legendre polynomial [31], and Chebyshev polynomial [32] were also extensively investigated under the FLANN framework. Due to the merits of FLANN, it has been applied to the solution of various practical problems, such as nonlinear acoustic echo cancellation [26], [33], nonlinear channel equalization [34], and NANC [5], [29].

The Chebyshev polynomial expansions satisfy all the requirements of the Stone–Weierstrass approximation theorem, and therefore it has powerful nonlinear modeling capabilities as compared to the MLP and can be used for many practical applications [32], [35], [36]. These Chebyshev polynomials form an orthogonal basis, which converge faster than expansions in other sets of polynomials [37], [38]. The Chebyshev functional link artificial neural network (CFLANN) combines the benefits of the Chebyshev polynomials and the FLANN, which is a well-known improvement to solve the nonlinear system identification problem. It has been proved that the CFLANN shares some similar characteristics to the EMFN algorithm and the Legendre nonlinear (LN) algorithm, and outperforms these algorithms in various environments [32]. At present, a vast number of CFLANN-based algorithms have been proposed in diverse fields [35], [36], [39], [40], [41]. Particularly, in [36], a CFLANN-based recursive least square (CFLANN-RLS) algorithm was proposed for the ideal dynamic system identification in online mode. This algorithm obtains good identification accuracy with low computational cost, which makes it very well-suited for applications in the design of online adaptive identification models.

In many practical situations, the updating step of the algorithm can be performed only after a fixed time delay. For instance, in many dynamic systems [42], [43], decision-directed adaptive channel equalizers [44] and implementation of the adaptive filter [45], [46], time delay widely exists. In [47], a practical identification algorithm for the time delayed linear system with incomplete measurement was proposed, which has a good quality of noise resistance. The algorithm in [48] provides a distortion correction method to identify the strongly nonlinear stiffness of the equivalent model and time delay in the absorber system. Following a different direction, the adaptive algorithm was proposed for system identification with time delay. Such research dates back to 1989 (delayed-least mean square, DLMS algorithm) [49], and so far several works of the DLMS algorithm have been developed [50], [51], [52], [53], [54]. But few algorithms aimed at enhancing the stability of DLMS algorithm were investigated. In [53], a modified leaky delayed least-mean-square (MLDLMS) algorithm was proposed under the imperfect system delay estimates. Nevertheless, this algorithm has the risk of instability in cases where the unknown system is nonlinear.

Motivated by the advantages of CFLANN, we first consider the time delay problem in CFLANN for nonlinear system identification. Before that, some time delay neural networks have been proposed for nonlinear system identification. However, these efforts rely on the priori information or have high computational complexity [23], [55], which are hard to implement. To facilitate its practical use, the delayed-recursive least sqaure (DRLS) algorithm is proposed as an online training algorithm in CFLANN, resulting in the CFLANN-DRLS algorithm. The proposed CFLANN-DRLS is a recursive algorithm, which converges fast even when the eigenvalue spread of the input signal correlation matrix is large. Besides, algorithms of the recursive type have excellent performance when implementing to time-varying environments [56]. To further enhance the performance of the CFLANN-DRLS algorithm, a modified CFLANN-DRLS (CFLANN-MDRLS) algorithm that does not need any a priori information about the noise statistical characteristics and nonlinear system, is developed motivated by the method in [54]. This method makes it possible to estimate the time delay from the nonlinear system, thereby enabling the newly proposed CFLANN-MDRLS algorithm to achieve an improved performance. In summary, our main contributions are depicted as the following points:

• The CFLANN with time delay model is first proposed for nonlinear dynamic system identification.

• The CFLANN-DRLS algorithm is proposed for online adaptation of the parameter in CFLANN.

• The CFLANN-MDRLS algorithm is developed for performance improvement.

• The convergence analysis of the CFLANN-MDRLS algorithm is performed and supported by simulations.

The rest of the paper is organized as follows. In Section 2, we briefly introduce the nonlinear system model. In Section 3, we present the proposed algorithms in detail, including CFLANN-DRLS and CFLANN-MDRLS algorithms. Then, the convergence analysis of the CFLANN-MDRLS algorithm is performed. Simulation results are shown in Section 4 to illustrate the effectiveness and advantages of the proposed algorithms. Finally, Section 5 presents the conclusions and future lines of research.