Time delay Chebyshev functional link artificial neural network
Introduction
System identification is the art and science of establishing a mathematical model for an unknown system through the input–output relationship, which plays a significant role in control theory [1], [2]. In previous studies, many approaches have been proposed in the context of linear system identification [3], [4]. However, the real-world system is not always linear. For instance, different types of artificial noise in active noise control systems [5], [6], [7] and impulsive noises in communication systems [8], [9], can be described more accurately using nonlinear models. In these cases, linear system identification technique performs poorly. Therefore, it is reasonable to take the nonlinear system identification technique into account.
Several nonlinear approaches have been proposed to identify nonlinear dynamic systems [10], [11], [12], [13]. Among these, the artificial neural network (ANN) has attracted more and more attention because of the learning ability and excellent approximating performance [13], [14]. Most of the ANN-based system identification techniques are based on the feedforward networks such as the radial basis function neural networks (RBFNNs) [15], [16], [17] and the multilayer perceptron (MLP) trained with backpropagation (BP) [18], [19], [20], [21]. In [15], an effective procedure based on the RBFNN was proposed to detect the harmonic amplitudes of the measured signal. However, the main bottleneck of the RBFNNs is their essentially static input-to-output maps, which reduces the ability of modeling nonlinear systems [22], [23]. The algorithm in [21] was developed for estimating carbon price of European Union Emissions Trading Scheme. Although the reliable performance of the MLP network model is achieved, such neural network does not consider the time delay in network and compensate the delay for performance improvement.
As an effective alternative to the multilayer ANN, the functional link ANN (FLANN) was initially proposed on neural networks [24]. By removing the hidden layer, the FLANN becomes a single layer network. It employs the point-wise functional expansions of the current input pattern and then it generates the output by linear combination. Previous studies have shown that the FLANN can simplify the learning algorithms and unify the architecture for all types of networks [25]. So far, a large number of FLANN algorithms have been proposed based upon different basis functions [5], [26], [27], [28]. By making use of the Fourier expansions, even mirror Fourier nonlinear (EMFN) algorithm was introduced in [27], which accelerates the convergence rate as compared with its Volterra expansion counterpart. The trigonometric polynomial is another popular expansion, which is an effective means for nonlinear active noise control (NANC) and provides a computational advantage over existing algorithms [5], [29]. Furthermore, many orthogonal basis functions, such as Hermite polynomial [30], Legendre polynomial [31], and Chebyshev polynomial [32] were also extensively investigated under the FLANN framework. Due to the merits of FLANN, it has been applied to the solution of various practical problems, such as nonlinear acoustic echo cancellation [26], [33], nonlinear channel equalization [34], and NANC [5], [29].
The Chebyshev polynomial expansions satisfy all the requirements of the Stone–Weierstrass approximation theorem, and therefore it has powerful nonlinear modeling capabilities as compared to the MLP and can be used for many practical applications [32], [35], [36]. These Chebyshev polynomials form an orthogonal basis, which converge faster than expansions in other sets of polynomials [37], [38]. The Chebyshev functional link artificial neural network (CFLANN) combines the benefits of the Chebyshev polynomials and the FLANN, which is a well-known improvement to solve the nonlinear system identification problem. It has been proved that the CFLANN shares some similar characteristics to the EMFN algorithm and the Legendre nonlinear (LN) algorithm, and outperforms these algorithms in various environments [32]. At present, a vast number of CFLANN-based algorithms have been proposed in diverse fields [35], [36], [39], [40], [41]. Particularly, in [36], a CFLANN-based recursive least square (CFLANN-RLS) algorithm was proposed for the ideal dynamic system identification in online mode. This algorithm obtains good identification accuracy with low computational cost, which makes it very well-suited for applications in the design of online adaptive identification models.
In many practical situations, the updating step of the algorithm can be performed only after a fixed time delay. For instance, in many dynamic systems [42], [43], decision-directed adaptive channel equalizers [44] and implementation of the adaptive filter [45], [46], time delay widely exists. In [47], a practical identification algorithm for the time delayed linear system with incomplete measurement was proposed, which has a good quality of noise resistance. The algorithm in [48] provides a distortion correction method to identify the strongly nonlinear stiffness of the equivalent model and time delay in the absorber system. Following a different direction, the adaptive algorithm was proposed for system identification with time delay. Such research dates back to 1989 (delayed-least mean square, DLMS algorithm) [49], and so far several works of the DLMS algorithm have been developed [50], [51], [52], [53], [54]. But few algorithms aimed at enhancing the stability of DLMS algorithm were investigated. In [53], a modified leaky delayed least-mean-square (MLDLMS) algorithm was proposed under the imperfect system delay estimates. Nevertheless, this algorithm has the risk of instability in cases where the unknown system is nonlinear.
Motivated by the advantages of CFLANN, we first consider the time delay problem in CFLANN for nonlinear system identification. Before that, some time delay neural networks have been proposed for nonlinear system identification. However, these efforts rely on the priori information or have high computational complexity [23], [55], which are hard to implement. To facilitate its practical use, the delayed-recursive least sqaure (DRLS) algorithm is proposed as an online training algorithm in CFLANN, resulting in the CFLANN-DRLS algorithm. The proposed CFLANN-DRLS is a recursive algorithm, which converges fast even when the eigenvalue spread of the input signal correlation matrix is large. Besides, algorithms of the recursive type have excellent performance when implementing to time-varying environments [56]. To further enhance the performance of the CFLANN-DRLS algorithm, a modified CFLANN-DRLS (CFLANN-MDRLS) algorithm that does not need any a priori information about the noise statistical characteristics and nonlinear system, is developed motivated by the method in [54]. This method makes it possible to estimate the time delay from the nonlinear system, thereby enabling the newly proposed CFLANN-MDRLS algorithm to achieve an improved performance. In summary, our main contributions are depicted as the following points:
- (1)
The CFLANN with time delay model is first proposed for nonlinear dynamic system identification.
- (2)
The CFLANN-DRLS algorithm is proposed for online adaptation of the parameter in CFLANN.
- (3)
The CFLANN-MDRLS algorithm is developed for performance improvement.
- (4)
The convergence analysis of the CFLANN-MDRLS algorithm is performed and supported by simulations.
The rest of the paper is organized as follows. In Section 2, we briefly introduce the nonlinear system model. In Section 3, we present the proposed algorithms in detail, including CFLANN-DRLS and CFLANN-MDRLS algorithms. Then, the convergence analysis of the CFLANN-MDRLS algorithm is performed. Simulation results are shown in Section 4 to illustrate the effectiveness and advantages of the proposed algorithms. Finally, Section 5 presents the conclusions and future lines of research.
Section snippets
Problem formulation
Fig. 1 shows the structure of nonlinear system identification based on the CFLANN with time delay, where u(k) denotes the input signal, z(k) denotes the nonlinear system output, d(k) denotes the desired signal, v(k) is the additive noise, D is the time delay related to the nonlinear system (nonlinear ANN), is the time delay estimate, y(k) is the output of the FLANN, and e(k) is the error signal between y(k) and d(k). Here, we consider the single-input–single-output (SISO) and
Chebyshev basis function
The ANN structure considered in time delay model is a single layer Chebyshev neural network based on the Chebyshev polynomials. The Chebyshev polynomials hold two-fold characteristics as follows. (1) Chebyshev polynomials are a family of orthogonal polynomials. (2) Chebyshev polynomials converge faster than expansions in other sets of polynomials. The Chebyshev polynomials are given by a generating function [32]where and n is the order of Chebyshev polynomials
Simulation results
In this section, we evaluate the performance of the proposed algorithms on some numerical simulations in the case of SISO system, MIMO system, and Box and Jenkins’ identification problem. To test the performance of the proposed algorithms, we compare them with the CFLANN-RLS algorithm. Here, the CFLANN-RLS algorithm is used for nonlinear system without time delay, i.e., . It can be regarded as an ‘ideal state’ of the algorithm, that is to say, all the online learning algorithm and unknown
Conclusion
In this paper, two CFLANN delayed-based RLS algorithms have been proposed for nonlinear system identification problem. By considering a time delay in the coefficient adaptation, the CFLANN-DRLS algorithm has been proposed. However, it has a worse convergence performance than the CFLANN-RLS algorithm (without time delay). To solve this problem, the CFLANN-MDRLS algorithm has been developed by introducing compensation term in the error signal, which can achieve a better performance under the
Acknowledgments
We thank Dr. Zongsheng Zheng (School of Electrical Engineering, Southwest Jiaotong University, China), he gives us many precious suggestions and discussions. Meanwhile, the authors would like to thank all the anonymous reviewers for their valuable comments and suggestions for improving the quality of this work.
This work was partially supported by the National Science Foundation of P.R. China under Grant no. 61701327.
Lu Lu was born in Chengdu, China, in 1990. He received the Ph.D. degree in the field of signal and information processing at the School of Electrical Engineering, Southwest Jiaotong University, Chengdu, China, in 2018. From January 2017 to January 2018, he was a visiting Ph.D. student with the Electrical and Computer Engineering at McGill University, Montreal, QC, Canada. He is currently a postdoctoral fellow with the College of Electronics and Information Engineering, Sichuan University,
References (67)
- et al.
Collaborative adaptive Volterra filters for nonlinear system identification in α-stable noise environments
J. Frankl. Inst.
(2016) - et al.
Sparse normalized subband adaptive filter algorithm with l0-norm constraint
J. Frankl. Inst.
(2016) - et al.
Nonlinear active noise control using spline adaptive filters
Appl. Acoust.
(2015) - et al.
Adaptive Volterra filter with continuous lp-norm using a logarithmic cost for nonlinear active noise control
J. Sound Vib.
(2016) Nonlinear least lp-norm filters for nonlinear autoregressive α-stable processes
Digit. Signal Process.
(2002)Differential evolution-based nonlinear system modeling using a bilinear series model
Appl. Soft Comput.
(2012)- et al.
A new detection approach of transient disturbances combining wavelet packet and Tsallis entropy
Neurocomputing
(2014) - et al.
Self-organization of a recurrent RBF neural network using an information-oriented algorithm
Neurocomputing
(2017) - et al.
Artificial neural networks (the multilayer perceptron) – a review of applications in the atmospheric sciences
Atmos. Environ.
(1998) - et al.
Robust on-line nonlinear systems identification using multilayer dynamic neural networks with two-time scales
Neurocomputing
(2013)
Chaotic characteristic identification for carbon price and an multi-layer perceptron network prediction model
Expert Syst. Appl.
A linear recurrent kernel online learning algorithm with sparse updates
Neural Netw.
Adaptive neural network tracking control for a class of switched strict-feedback nonlinear systems with input delay
Neurocomputing
Improving nonlinear modeling capabilities of functional link adaptive filters
Neural Netw.
Fourier nonlinear filters
Signal Process.
Functional link artificial neural network filter based on the q-gradient for nonlinear active noise control
J. Sound Vib.
A semi-analytical formula for estimating peak wind load effects based on Hermite polynomial model
Eng. Struct.
Hierarchical theories of structures based on Legendre polynomial expansions with finite element applications
Int. J. Mech. Sci.
A study about Chebyshev nonlinear filters
Signal Process.
A functional link artificial neural network for adaptive channel equalization
Signal Process.
On-line system identification of complex systems using Chebyshev neural networks
Appl. Soft Comput.
A channel equalizer using reduced decision feedback Chebyshev functional link artificial neural networks
Inf. Sci.
Time-delay systems: an overview of some recent advances and open problems
Automatica
An improved phase method for time-delay estimation
Automatica
Time delay identifiability and estimation for the delayed linear system with incomplete measurement
J. Sound Vib.
Adaptive time delay neural network structures for nonlinear system identification
Neurocomputing
Multiple recurrent neural networks for stable adaptive control
Neurocomputing
A comparative study of adaptation algorithms for nonlinear system identification based on second order Volterra and bilinear polynomial filters
Measurement
Memory improved proportionate M-estimate affine projection algorithm
Electron. Lett.
Active mitigation of nonlinear noise processes using a novel filtered-s LMS algorithm
IEEE Trans. Speech Audio Process.
Simplified Volterra filters for acoustic echo cancellation in GSM receivers
Proceedings of the Tenth Conference on European Signal Processing
Continuous-time bilinear system identification using single experiment with multiple pulses
Nonlinear Dyn.
Polynomial Signal Processing
Cited by (24)
Robust kernel adaptive filtering for nonlinear time series prediction
2023, Signal ProcessingFinite-time active fault-tolerant attitude control for flexible spacecraft with vibration suppression and anti-unwinding
2023, Advances in Space ResearchCitation Excerpt :The adaptive strategy negates the need for this prior bound knowledge and can estimate the bounds of the uncertainties. However, it has two major shortcomings: weak robustness to unmodeled dynamics and assumption of linearity in uncertain dynamics’ parameters (Lu et al., 2019). Propitiously, neural networks (NN) offer a solution to these shortcomings (Chen and Ge, 2015; Liu et al., 2015).
A class of augmented complex-value FLANN adaptive algorithms for nonlinear systems
2023, NeurocomputingCitation Excerpt :Thus, the input signals of the FLN filters are mapped into a high dimension by point-wise orthogonal polynomial expansions, and then are combined with a linear adaptive filter to get output signals. There are too many polynomial expansions of the FLN, including conventional trigonometric functional expansion, Chebyshev orthogonal polynomial, Legendre expansion, Hermite functional expansion, and Euler functional expansion [30–34]. However, the traditional linear finite impulse response (FIR) filters have a long computational load and slow convergence rate.
An improved time delay neural network model for predicting dynamic heat and mass transfer characteristics of a packed liquid desiccant dehumidifier
2022, International Journal of Thermal SciencesCitation Excerpt :And TDNN based methods could also capture complicate temporal characteristics of environmental sounds with transient, intermittent and continuous temporal structures. Lu at el [22]. proposed an online training algorithm (delayed-recursive least square, DRLS) to facilitate practical use of TDNN, which could achieve excellent identification performance when priori information of nonlinear dynamic process is hard to be obtained.
Lu Lu was born in Chengdu, China, in 1990. He received the Ph.D. degree in the field of signal and information processing at the School of Electrical Engineering, Southwest Jiaotong University, Chengdu, China, in 2018. From January 2017 to January 2018, he was a visiting Ph.D. student with the Electrical and Computer Engineering at McGill University, Montreal, QC, Canada. He is currently a postdoctoral fellow with the College of Electronics and Information Engineering, Sichuan University, Chengdu, China. His research interests include adaptive signal processing, kernel methods and distributed estimation.
Yi Yu was born in Sichuan Province, China, in 1989. He received the B.E. degree at Xihua University, Chengdu, China, in 2011, and the M.S. degree and Ph.D. degree at Southwest Jiaotong University, Chengdu, China, in 2014 and 2018, respectively. From December 2016 to December 2017, he was a visiting Ph.D. student with the Department of Electronic Engineering, University of York, United Kingdom. His research interests include adaptive signal processing, distributed estimation, and compressive sensing.
Xiaomin Yang is currently an Associate Professor in College of Electronics and Information Engineering, Sichuan University. She received her B.S. degree from Sichuan University in 2002, and received her Ph.D. degree in communication and information system from Sichuan University in 2007. From October 2009 to October 2010, she worked at the University of Adelaide as a Postdoctoral for one year. At present, she has authored or co-authored of more than 50 research articles in international journals and conferences. Her research interests fall under the umbrella of image processing, particularly image enlargement, super-resolution, image enhancement as well as computational intelligence.
Wei Wu is currently a Professor in College of Electronics and Information Engineering, Sichuan University. He received his B.S. degree from Tianjin University, in 1998. He received M.S. and Ph.D. degrees in communication and information system from Sichuan University, in 2003 and 2008, respectively. From October 2009 to October 2010, he worked in a National Research Council Canada as a post doctorate for one year. He has also edited journals special issues in the area of image enlargement, cloud computing, and embedding. At present, he has authored or co-authored of more than 70 research article in international journals and conferences. His research interests fall under the umbrella of image processing, particularly image enlargement, super-resolution, image enhancement as well as computational intelligence. Wu has served as an active reviewer for several IEEE Transactions, and other international journals.