Elsevier

Automatica

Volume 97, November 2018, Pages 92-103
Automatica

Prediction-error identification of LPV systems: A nonparametric Gaussian regression approach

https://doi.org/10.1016/j.automatica.2018.07.032Get rights and content

Abstract

In this paper, a Bayesian nonparametric approach is introduced to estimate multi-input multi-output (MIMO) linear parameter-varying (LPV) models under the general noise model structure of Box–Jenkins (BJ) type. The approach is based on the estimation of the one-step-ahead predictor of general LPV-BJ structures. Parts of the predictors associated with the input and output signals are modeled as asymptotically stable infinite impulse response (IIR) models. Then, these IIR models are identified in a completely nonparametric sense: not only the coefficients are estimated as functions, but also the whole time evolution of the impulse response w.r.t. the scheduling signal of the LPV system. In this Bayesian setting, the estimate of the one-step-ahead predictor is a realization from a zero-mean Gaussian random field, where the covariance function is a multidimensional Gaussian kernel that encodes both the possible structural dependencies and the stability of the predictor. Two different kernel formulations are presented for the LPV setting, namely a diagonal (DI) like and tuned/correlated (TC) like kernels, where the TC-like kernel is able to describe the correlation between coefficient functions associated with different time indices. The unknown hyperparameters that parameterize the DI or TC kernel are tuned by maximizing the marginal likelihood w.r.t. the observed data. Moreover, we provide a nonparametric realization scheme to recover the original process and noise IIRs from the identified one-step-ahead predictor. The performance of the presented identification approach is tested on a MIMO LPV-BJ simulation example by means of an extensive Monte-Carlo study.

Introduction

Linear parameter-varying (LPV) systems, introduced in Shamma and Athans (1990), have received considerable attention Mohammadpour and Scherer (2011), Tóth (2010), as they offer an attractive modeling framework to capture nonlinear and/or non-stationary behavior of physical and chemical processes Bachnas et al. (2014), Wassink et al. (2004). Most of the existing LPV identification (ID) approaches are formulated in discrete-time (DT) (Tóth, 2010) to identify state-space or linear-fractional representation forms (e.g., dos Santos et al. (2008), Sznaier and Mazzaro (2003), van Wingerden and Verhaegen (2009), Wills and Ninness (2011)); series-expansion based models (e.g., Tóth, Heuberger, & Van den Hof, 2009); and various input–output (IO) model structures (e.g., Bachnas et al. (2014), Bamieh and Giarré (2002), Tóth et al. (2012)).

Identification of LPV-IO models gained popularity, as prediction-error minimization (PEM) methods have been successfully extended to LPV models, providing a well-understood framework for consistency and stochastic interpretation of the estimates together with low computational complexity of the resulting identification procedures (Tóth et al., 2012). Moreover, the PEM framework is well suited to identify a large variety of noise and plant models, see Tóth (2010) for an overview. Although LPV-IO models cover a variety of process and noise representations, where the Box–Jenkins (BJ) model is the most general form, PEM identification of BJ models leads to a nonlinear optimization problem (Tóth et al., 2012), which is sensitive to local minima. Alternatively, the instrumental variable (IV) method provides an attractive approach that deals with the general noise scenario and avoids the nonlinear optimization (Laurain, Gilson, Tóth, & Garnier, 2010). Another important issue in the identification of LPV-IO models is capturing the structural dependency on the scheduling signal. In the parametric case, the structural dependency is generally characterized by using a pre-specified set of basis functions, which either require significant prior knowledge of the underlying system or tedious repetitive execution of methods to synthesize an acceptable basis (Tóth et al., 2012). In addition, the choice of the number of these bases is challenging as it induces a bias/variance trade-off, i.e., by using fewer basis functions, the under-modeling (bias) error increases while increasing their number results in an increase of the variance of the parameters of the estimated models.

The so-called nonparametric methods offer attractive alternative approaches to capture the underlying dependencies directly from data without specifying any parameterization in terms of fixed basis functions. The main approaches of LPV nonparametric identification in the literature are: (i) the dispersion function method (Hsu, Vincent, & Poolla, 2008), (ii) the least squares-support vector machine (LS-SVM) methods (e.g., Piga and Tóth (2013), Tóth et al. (2011)), and (iii) the Bayesian setting based approaches (e.g., Golabi et al. (2014), Golabi et al. (2017)). However, in (i)–(iii) the considered noise models are restricted to output error type (LPV finite impulse response (FIR) model) and autoregressive type (LPV autoregressive with exogenous input (ARX) model). Additionally, both LS-SVM and Bayesian approaches have roots in the reproducing kernel Hilbert space (RKHS) theory (Aronszajn, 1950) and admit an 2-regularization interpretation (Chen, Ohlsson, & Ljung, 2012), such that consistency and convergence notions of the resulting estimator can be formulated.

This work is inspired by recent advances in nonparametric identification of linear time-invariant (LTI) models in the PEM setting (Pillonetto, Chiuso, & De Nicolao, 2011) and novel results for optimal kernel design (Pillonetto, Dinuzzo, Chen, De Nicolao, & Ljung, 2014). Here, we aim at formulating a nonparametric estimator of the one-step-ahead predictor for an LPV-BJ model, preserving the generality of the noise class and the asymptotic optimality of PEM. More specifically, we consider the one-step-ahead predictor as the summation of two sub-predictors associated with the input and output signals, where these sub-predictors are modeled as asymptotically stable LPV infinite impulse response (IIR) models. These LPV-IIR sub-predictors are identified in a nonparametric sense, where not only the coefficients are estimated as functions, but also the whole time evolution of the impulse response.

We follow a Bayesian approach for the nonparametric estimation by modeling the sub-predictors as realizations of zero-mean Gaussian random fields, which can be completely characterized by covariance (kernel) functions that implicitly act as a basis generator to describe both the functional dependencies and the time evolution of the impulse response of the sub-predictors. To this end, inspired by Pillonetto, Quang, and Chiuso (2011), we introduce a multidimensional Gaussian kernel which encodes: (i) the possible structural dependencies on the scheduling signal by using radial basis functions (RBF) and (ii) the stability of the predictor by including a decay term, which models the vanishing influence of the past input-scheduling-output pairs on the predicted output. The hyperparameters that parameterize the kernel can be efficiently estimated from data by maximizing the marginal likelihood w.r.t. the observations (MacKay, 2003). A preliminary work in this direction can be found in Darwish, Cox, Pillonetto, and Tóth (2015), however, here we provide the following extensions:

  • (1)

    Kernel formulation for the multi-input multi-output (MIMO) case;

  • (2)

    Enriching the kernel to take into account (nominal) LTI dynamics of the model, independent of the scheduling variables;

  • (3)

    Introduce a tuned/correlated (TC)-like kernel for the LPV setting to encode correlation between coefficient functions associated with different time indices;

  • (4)

    Introducing a nonparametric realization scheme to recover the original process and noise IIRs from the identified one-step-ahead sub-predictors.

The paper is organized as follows. In Section 2, the considered model structure is defined and the corresponding optimal one-step-ahead predictor is derived. The considered Gaussian regression framework is reviewed in Section 3. In Section 4, Bayesian identification of LPV-IO models and the problem of estimating the predictor and the hyperparameters of the kernel from data are presented. This is followed by a realization approach to get a nonparametric estimate for the process and noise dynamics from the identified predictor in Section 5. In Section 6, the effectiveness of the introduced approach is demonstrated by means of an extensive Monte Carlo study. Finally, the conclusions of the paper are given in Section 7.

Section snippets

LPV-BJ model

Consider a MIMO data-generating LPV system described in DT1 by the following difference equations: (A0(q1)p)ky̆(k)=(B

Gaussian regression framework

In this section, the Gaussian process (GP) regression framework and the connection to function estimation in RKHS are reviewed.

Bayesian identification of LPV-IO models

In this section, the Gaussian regression framework of Section 3 will be applied to the estimation of the LPV-IIR representation (11). First, the identification of (11) will be formulated as shown in Section 3. Second, an appropriate kernel K will be designed. Finally, the estimation of the unknown structural dependencies will be introduced.

The covariance on the noise e(k) is assumed to be diagonal, i.e., Σe=diag([σ12σny2]). Hence, the (λ)th output channel of (11) can be written as: y(k)λ=fλ(x(k

Estimate of the process and noise models

In this section, we present a novel method to construct a nonparametric estimate of the process (9a) and noise (9b) model based on the estimated one-step-ahead predictor in Section 4.3. Such a representation of the model estimates is more useful for control synthesis (Formentin, Piga, Tóth, & Savaresi, 2016) or analyzing the dynamics of the deterministic part (1a).

To this end, we use the truncated nonparametric estimates hˆyi, hˆuj (41) of hyi, huj of order nfy, nfu (described in Section 4.3)

Numerical simulation

In this section, the performance of the presented nonparametric approach for the identification of LPV-BJ models based on their one-step-ahead predictor is shown by means of an extensive Monte-Carlo study.

Conclusion

In this paper, we have presented a nonparametric identification approach for MIMO LPV-BJ models. Similar to the LTI case, it has been shown that the one-step-ahead predictor of such models is a summation of two sub-predictors associated with the input and output signals, where under mild assumptions, these sub-predictors are shown to be convergent IIRs. To cope with issues associated with identifying such models, e.g., parameterization of parameter-varying matrix coefficient functions, a

Mohamed Abdelmonim Hassan Darwish was born on January 12, 1984, in Assiut, Egypt. He received his B.Sc. in Electrical Engineering (Computers and Control) from Assiut University, Assiut, Egypt in 2005. He stood the first among his colleagues and graduated with the highest distinction.

He had the honor of receiving the Excellence award from both the Egyptian Engineering Syndicate and the Government of Egypt during the Science Day in 2005 and 2006, respectively.

In 2006, he started at the same

References (44)

  • van WingerdenJ.W. et al.

    Subspace identification of Bilinear and LPV systems for open- and closed-loop data

    Automatica

    (2009)
  • WangL. et al.

    Use of PRESS residuals in dynamic system identification

    Automatica

    (1996)
  • WollnackS. et al.

    Fixed-structure LPV-IO controllers: An implicit representation based approach

    Automatica

    (2017)
  • AronszajnN.

    Theroy of reproducing kernels

    Transactions of the American Mathematical Society

    (1950)
  • BamiehB. et al.

    Identification of linear parameter varying models

    International Journal of Robust and Nonlinear Control

    (2002)
  • CarlinB.P. et al.

    Bayes and empirical bayes methods for data analysis

    (2000)
  • Darwish, M. A. H., Cox, P. B., Pillonetto, G., & Tóth, R. (2015). Bayesian identification of LPV Box-Jenkins models. In...
  • DarwishM.A.H. et al.

    Description of the Data Generating System Utilized in “Prediction-Error Identification of LPV Systems: A Nonparametric Gaussian Regression Approach”, TUE-CS-2017-001

    (2017)
  • Darwish, M. A. H., Lataire, J., & Tóth, R. (2017). Bayesian frequency domain identification of LTI systems with OBFs...
  • Darwish, M. A. H., Pillonetto, G., & Tóth, R. (2015). Perspectives of orthonormal basis functions based kernels in...
  • dos Santos, P. L., Ramos, J. A., & de Carvalho, J. L. M. (2008). Identification of LPV systems using successive...
  • Golabi, A., Meskin, N., Tóth, R., & Mohammadpour, J. (2014). A Bayesian approach for estimation of LPV...
  • Cited by (16)

    • Nonparametric identification of batch process using two-dimensional kernel-based Gaussian process regression

      2022, Chemical Engineering Science
      Citation Excerpt :

      In this study, we follow a Bayesian approach for the nonparametric estimation by modeling the impulse response of different time points as realizations of zero-mean GP. Previous work has focused only on the inheritance function design for batch process, an alternative approach is to encode the prior knowledge into the kernel design of GP, which is prevalent in GP machine learning (Ning and You, 2019; Seeger, 2004) and kernel-based identification (Birpoutsoukis et al., 2017; Chen, 2018; Darwish et al., 2018; Risuleo et al., 2017). Depending on the 2D correlation property of batch process, we propose the amplitude modulated 2D locally stationary kernel with two ways of combinations: addition and multiplication.

    • Frequency response function identification of periodically scheduled linear parameter-varying systems

      2021, Mechanical Systems and Signal Processing
      Citation Excerpt :

      In [43], a method is developed for input–output LPV models that employs a support vector machine approach to estimate the scheduling dependence nonparametrically, yet similarly requires prior knowledge of the dynamic order. The Bayesian approaches developed in [16,10] enable nonparametric estimation in both the dynamics and parameter dependence through the use of Gaussian processes. These approaches require careful tuning of the hyperparameters associated with the kernel functions and require approximate prior knowledge on the pole locations [6].

    • Sparse RKHS estimation via globally convex optimization and its application in LPV-IO identification

      2020, Automatica
      Citation Excerpt :

      The proposed approach is demonstrated on the identification problem of linear parameter-varying (LPV) systems (see Tóth, 2010), which corresponds to a generalization of the “sum of nonlinear functions” problem. It is shown that the proposed method allows joint reconstruction of the scheduling-variable dependencies and the model coefficient structure from data, extending the capabilities of previous non-parametric estimators developed for this model class (Darwish et al., 2018; Laurain, Tóth, Zheng, & Gilson, 2012; Lopes dos Santos, &, 2014). To introduce the preliminaries for the main results of the paper, a brief introduction to nonlinear function estimation via the RKHS theory is provided in this section.

    View all citing articles on Scopus

    Mohamed Abdelmonim Hassan Darwish was born on January 12, 1984, in Assiut, Egypt. He received his B.Sc. in Electrical Engineering (Computers and Control) from Assiut University, Assiut, Egypt in 2005. He stood the first among his colleagues and graduated with the highest distinction.

    He had the honor of receiving the Excellence award from both the Egyptian Engineering Syndicate and the Government of Egypt during the Science Day in 2005 and 2006, respectively.

    In 2006, he started at the same department as a teaching assistant, where he was responsible for teaching and supervising graduation projects. At the same time, he continued his studies at the same department and obtained his M.Sc. in Electrical Engineering (Systems and Control) in 2011. In 2011, he received a promotion and became an Assistant lecturer at the same department.

    In 2013, he obtained a fully funded Ph.D. scholarship to continue his graduate studies toward a Ph.D. degree at the Eindhoven University of Technology (TU/e).

    Since September 2013, he has been working as a Ph.D. student on a project entitled “Bayesian Identification of Linear Dynamic Systems: Synthesis of Kernels in the LTI Case and Beyond” at the Control Systems research group, Eindhoven University of Technology, The Netherlands, under the supervision of prof. dr. ir. P. M. J. Van den Hof and dr. ir. Roland Tóth. He successfully defended his Ph.D. in October 2017. During the period 2013–2016, he took graduate courses at the Dutch Institute of Systems and Control (DISC) and received the DISC certificate.

    His research interests include data-driven modeling of linear and nonlinear dynamic systems and machine learning.

    Pepijn Bastiaan Cox was born in Arnhem, The Netherlands, in 1989. He has received his B.Sc. degree in Mechanical Engineering (cum laude) in 2010 and his M.Sc. degree in Systems and Control Engineering (cum laude) in 2013, both at the Delft University of Technology (TUDelft), The Netherlands. In 2018, he obtained his Ph.D. degree in the Control Systems group at the Eindhoven University of Technology (TUe), The Netherlands. His Ph.D. topic was “Towards Efficient Identification of Linear Parameter-Varying State-Space Models” under the supervision of prof.dr.ir. P.M.J. Van den Hof and dr.ir. R. Tóth. Currently, he is a postdoctoral researcher in the Control Systems group at the TUe. Pepijn Cox’s main research interests are in linear parameter-varying (LPV) and nonlinear system modeling and identification.

    Ioannis Proimadis was born in 1989 in Tripolis, Greece. In 2012 he obtained his 5-year diploma in Electrical Engineering in the University of Patras, Greece and in 2015 he received the M.Sc. degree (cum laude) in Systems and Control at Delft University of Technology in the Netherlands. In 2015 he joined the Control Systems group at Eindhoven University of Technology as a Ph.D. student. Since then he has been working in the “Nanometer-Accurate Planar Actuation System (NAPAS)” project, under the supervision of prof. dr. ir. Paul M. J. Van den Hof, prof. dr.ir. Hans Butler and dr. ir. Roland Tóth. His research interests include data-driven identification of nonlinear systems, kernel-based identification, as well as control for flexible and position-dependent motion systems.

    Gianluigi Pillonetto was born on January 21, 1975 in Montebelluna (TV), Italy. He received the Doctoral degree in Computer Science Engineering cum laude from the University of Padova in 1998 and the Ph.D. degree in Bioengineering from the Polytechnic of Milan in 2002. In 2000 and 2002 he was visiting scholar and visiting scientist, respectively, at the Applied Physics Laboratory, University of Washington, Seattle. From 2002 to 2005 he was Research Associate at the Department of Information Engineering, University of Padova, becoming an Assistant Professor in 2005. He is currently an Associate Professor of Control and Dynamic Systems at the Department of Information Engineering, University of Padova. His research interests are in the field of system identification, estimation and machine learning. He currently serves as Associate Editor for Automatica and IEEE Transactions on Automatic Control. In 2003 he received the Paolo Durst award for the best Italian Ph.D. thesis in Bioengineering, and he was the 2017 recipient of the Automatica Prize.

    Roland Tóth was born in 1979 in Miskolc, Hungary. He received the B.Sc. degree in Electrical Engineering and the M.Sc. degree in Information Technology in parallel with distinction at the University of Pannonia, Veszprém, Hungary, in 2004, and the Ph.D. degree (cum laude) from the Delft Center for Systems and Control (DCSC), Delft University of Technology (TUDelft), Delft, The Netherlands, in 2008. He was a Post-Doctoral Research Fellow at DCSC, TUDelft, in 2009 and at the Berkeley Center for Control and Identification, University of California, Berkeley, in 2010. He held a position at DCSC, TUDelft, in 2011–12. Currently, he is an Assistant Professor at the Control Systems Group, Eindhoven University of Technology (TU/e). He is an Associate Editor of the IEEE Conference Editorial Board, the IEEE Transactions on Control Systems Technology and the International Journal of Robust and Nonlinear Control. His research interests are in linear parameter-varying (LPV) and nonlinear system identification, machine learning, process modeling and control, model predictive control and behavioral system theory. Dr. Tóth received the TUDelft Young Researcher Fellowship Award in 2010, the VENI award of The Netherlands Organisation for Scientific Research in 2011 and the Starting Grant of the European Research Council in 2016.

    This research has benefited from the financial support of the Student Mission, Ministry of Higher Education, Government of Egypt. The first three authors have contributed equally to the paper. The material in this paper was partially presented at the 54th IEEE Conference on Decision and Control, December 15–18, 2015, Osaka, Japan. This paper was recommended for publication in revised form by Associate Editor Cristian R. Rojas under the direction of Editor Torsten Söderström.

    View full text