Gradient-based iterative identification method for multivariate equation-error autoregressive moving average systems using the decomposition technique☆
Introduction
System identification is the foundation of constructing the mathematical models of systems from input–output data [1], [2], and parameter estimation is essential for system identification [3], [4]. So exploring new parameter estimation methods is an eternal theme of system identification [5], [6], and many identification methods have been applied in different fields, such as system control [7], information filtering [8] and signal processing [9]. As a consequence of the wide variety of applications, some identification methods have been reported in the literature. For example, Chaudhary et al. exploited a strength of momentum least mean square algorithm for nonlinear system identification problems represented with Hammerstein model [10], [11]. Yu and Verhaegen studied the identification of a network comprised of interconnected clusters of linear time-invariant systems and a subspace identification method was proposed for identifying each single cluster using only local input and output data [12]. Gan et al. presented a variable projection approach for estimation of radial basis function network-based autoregressive with exogenous inputs models by using the orthogonal projection [13].
Parameter estimation methods can be applied to many areas [14], [15], [16], [17]. Since multivariable systems widely exist in practical control systems, especially in modern large-scale industrial systems, it has drawn a great deal of attention of many researchers in multivariable system identification. At the same time, a series of methods have been proposed for multivariable systems, including gradient-based methods [18], the least squares methods [19], [20]. Different from the signal-input signal-output systems, multivariable systems has the characteristics of complex structure, multiple variables and non-linearity [21]. In addition, the interaction between multiple inputs and outputs of multivariate systems results in the complexity and coupling of the structures [22]. An effective method to identify multivariate systems is to use the decomposition technique [23]. The decomposition can transform a large-scale system into several small subsystems with fewer variables, thus the dimensions of the involved covariance matrices in each subsystem become smaller compared with the original system. By means of the decomposition technique, the computational burden of identification algorithms can be effectively reduced, and the computational efficiency can also be improved. Recently, Ma et al. proposed a recursive least squares identification method for multivariate pseudolinear systems using the decomposition technique and the multi-innovation theory [24]. Ding developed a hierarchial gradient based iterative algorithm and a hierarchical least squares based iterative algorithm for the identification problem of multi-input-output-error autoregressive systems by using the hierarchial identification principle [25]. The mentioned algorithms can give more accurate parameter estimates compared with the recursive or iterative based algorithms. However, in order to obtain convergent and consistent estimates, these methods are inevitably dependent upon the accuracy of the noise model.
In recent years, the iterative identification methods play an important role in the field of system identification because it can make full use of all input–output data. These methods have been successfully applied to state space systems [26], linear systems and nonlinear systems [27]. The iterative identification algorithms can be derived by means of defining and minimizing an output error criterion function, and many iterative identification algorithms were reported for different systems. For example, Xu and Ding proposed the Newton iterative algorithm and the gradient iterative algorithm for the parameter estimation problems of dynamical systems by means of the impulse response measurement data [28]. Li and Liu used the filtering technique and iterative methods to study the parameter estimation problems of a bilinear system with autoregressive noise [29]. Liu and Alleyne presented an iterative learning identification approach for the parameter estimates of linear time-varying systems [30]. However, these methods are usually for single-input single-output systems. The issue of how to efficiently apply the iterative identification methods and the decomposition technique for multivariate equation-error autoregressive moving average systems is a potential area of research. There are the motivation factors for the authors to investigate in exploring the identification problem of multivariate systems.
Different from the previous work in [31] which researches the recursive least squares identification methods, this paper uses the gradient-based iterative identification methods based on the decomposition technique. Compared with the recursive algorithm, the iterative algorithm uses a batch of data to refresh parameter estimation and improves identification accuracy. The key of the decomposition technique is to transform a multivariate equation-error autoregressive moving average system into two subsystems, where one contains a system model parameter vector and a measured information matrix, and the other contains a noise model parameter matrix and an unmeasured information vector, and then to estimate each subsystem by using the negative gradient search. The main contributions of this paper are as follows.
- •
A decomposition-based gradient iterative (D-GI) algorithm is developed for the multivariate equation-error autoregressive moving average system based on the decomposition technique.
- •
The D-GI identification method can effectively reduce the computational burden compared with the gradient-based iterative (GI) algorithm.
- •
The proposed D-GI algorithm can generate more accurate parameter estimates and has higher computational efficiency than the GI algorithm.
The proposed algorithms in this paper can combine the multi-innovation method [32], the neural network methods [33] and the kernel methods [34], [35] to study parameter identification of different systems [36], [37].
The rest of this paper is organized as follows. Section 2 describes an identification model of multivariate equation-error autoregressive moving average systems. Section 3 introduces the GI algorithm for comparisons. Section 4 presents a D-GI algorithm. Section 5 gives the convergence analysis of the two proposed algorithms. Section 6 provides a simulation example to show the effective of the proposed algorithms in this paper. Finally, we offer some concluding remarks in Section 7.
Section snippets
System description and identification model
Let us introduce some notations first.
Symbols Meaning : An -dimensional column vector whose entries are all 1. or : The identity matrix of appropriate sizes or . : The trace of the square matrix . : The transpose of the vector or matrix . : The determinant of the square matrix : . : The norm of a matrix : : is defined by . : is defined by . : The estimate of at iteration .
Consider the following multivariate equation-error autoregressive
The gradient-based iterative algorithm
For the purpose of showing the advantage of the proposed algorithm, we give a gradient-based iterative (GI) algorithm for comparison in this section. Assume that L is the data length. According to the identification model in Eq. (7), consider the input–output data from to and define a static criterion functionDefine the stacked output vector Y(L) and the stacked information matrix Ψ(L) asFrom Eq. (8),
The decomposition-based gradient iterative algorithm
Although the GI algorithm can improve the parameter estimation accuracy, the disadvantage is that it needs heavy computational load for large-scale systems. In order to improve the computational efficiency of the GI algorithm, we derive a decomposition-based gradient iterative (D-GI) algorithm in this section.
Introduce two intermediate variables:According to the hierarchical identification principle, the system in Eq. (6) can be decomposed into the
Convergence analysis
The convergence analysis of the two proposed algorithms is simply illustrated as follows.
Let and represent the estimates of θ, Φn(t) and θn, respectively. The output prediction is given byDefine the prediction error optimization criterionAccording to the methods in [50], suppose that the information vectors Φ(t) and are persistently exciting. Obviously, the independent and identically distributed noise assumption on the
Example
Consider the following multivariate equation-error autoregressive moving average system:
In simulation, the inputs u1(t) and u2(t) are taken as two independent
Conclusions
This paper discusses the parameter estimation problems of multivariate equation-error autoregressive moving average systems and derives a D-GI algorithm based on the decomposition technique. Compared with the GI algorithm, the proposed D-GI algorithm requires less computational effort, and has higher accuracy than the GI algorithm. The simulation example is given to show the effectiveness of the D-GI algorithm. In addition, the proposed algorithms can be combined with other identification
References (58)
- et al.
Momentum fractional LMS for power signal parameter estimation
Signal Process.
(2018) - et al.
Design of normalized fractional adaptive algorithms for parameter estimation of control autoregressive autoregressive systems
Appl. Math. Model.
(2018) - et al.
Robust adaptive parameter estimation of sinusoidal signals
Automatica
(2015) - et al.
A new computing approach for power signal modeling using fractional adaptive algorithms
ISA Trans.
(2017) - et al.
Parallel processing algorithm for railway signal fault diagnosis data based on cloud computing
Future Gener. Comput. Syst.
(2018) - et al.
A hierarchical least squares identification algorithm for Hammerstein nonlinear systems using the key term separation
J. Frankl. Inst.
(2018) - et al.
Combined state and parameter estimation for a bilinear state space system with moving average noise
J. Frankl. Inst.
(2018) - et al.
The least squares based iterative algorithms for parameter estimation of a bilinear system with autoregressive noise using the data filtering technique
Signal Process.
(2018) Several multi-innovation identification methods
Digital Signal Process.
(2010)- et al.
An optimal reproducing kernel method for linear nonlocal boundary value problems
Appl. Math. Lett.
(2018)
A new reproducing kernel collocation method for nonlocal fractional boundary value problems with non-smooth solutions
Appl. Math. Lett.
H-index for discrete-time stochastic systems with Markovian jump and multiplicative noise
Automatica
On the regularity of maximal operators supported by submanifolds
J. Math. Anal. Appl.
Exit problems for jump processes with applications to dividend problems
J. Comput. Appl. Math.
Optimal dividend problem with a terminal value for spectrally positive levy processes
Insur. Math. Econ.
Sparsity-aware transmit Beamspace design for FDA-MIMO radar
Signal Process.
On some separated algorithms for separable nonlinear squares problems
IEEE Trans. Cybern.
Subspace identification of local systems in one-dimensional homogeneous networks
IEEE Trans. Control Syst. Technol.
Iterative parameter estimation for signal models based on measured data
Circuits Syst. Signal Process.
State filtering-based least squares parameter estimation for bilinear systems using the hierarchical identification principle
IET Control Theory Appl.
Improving transient performance of adaptive control via a modified reference model and novel adaptation
Int. J. Robust Nonlinear Control
Adaptive gradient-based iterative algorithm for multivariate controlled autoregressive moving average systems using the data filtering technique
Complexity
Design of momentum LMS adaptive strategy for parameter estimation of hammerstein controlled autoregressive systems
Neural Comput. Appl.
Modified volterra LMS algorithm to fractional order for identification of hammerstein nonlinear system
IET Signal Process.
Subspace identification of distributed clusters of homogeneous systems
IEEE Trans. Autom. Control
A variable projection approach for efficient estimation of RBF-ARX model
IEEE Trans. Cybern.
Standard analysis for transfer delay in CTCS-3
Chin. J. Electron.
Performance evaluation with improved receiver design for asynchronous coordinated multipoint transmissions
Chin. J. Electron.
Optimization of information interaction protocols in cooperative vehicle-infrastructure systems
Chin. J. Electron.
Cited by (34)
A new multivariate equation-error autoregressive moving average system with conditional heteroscedastic noise: Maximum likelihood identification
2021, Digital Signal Processing: A Review JournalCitation Excerpt :In the past decade, the MIMO system identification researches have mainly concentrated on improving parameter estimation accuracy and computational efficiency. For example, hierarchical least-squares identification [7], adaptive gradient-based iterative algorithm [8] or decomposition-based techniques [9–11] are used to transform a large-scale system into several small subsystems with fewer variables. However, the proposed MIMO models are not very realistic and fail to appropriately reflect the characteristics of actual MIMO systems.
Robust adaptive identification for sandwich systems with unknown time-delay
2020, ISA TransactionsRobust hierarchical identification of Wiener systems in the presence of dynamic disturbances
2020, Journal of the Franklin InstituteGeneralization of the gradient method with fractional order gradient direction
2020, Journal of the Franklin InstituteCitation Excerpt :Gradient method, as a classical tool in optimization theory, because of its intuitive principle, simple structure and easy implementation, has attracted much attention and developed rapidly since its inception. Now it prevalently used in many fields, such as automatic control [1], system identification [2], machine learning [3], and image denoising [4]. Besides, many effective algorithms are based on it or inspired by it [5].
Fitting the exponential autoregressive model through recursive search
2019, Journal of the Franklin InstituteCitation Excerpt :This paper aims to derive new recursive identification algorithms for the ExpAR model. The gradient-based identification is an essential parameter estimation method [34–36]. Recently, Wang and Ding derived a gradient-based iterative algorithm for nonlinear systems with moving average noise [37].