Skip to main content
SpringerLink
Log in

On Convexification of System Identification Criteria

  • Topical Issue
  • Published:
Automation and Remote Control Aims and scope Submit manuscript

Abstract

System Identification is about estimating models of dynamical systems from measured input-output data. Its traditional foundation is basic statistical techniques, such as maximum likelihood estimation and asymptotic analysis of bias and variance and the like. Maximum likelihood estimation relies on minimization of criterion functions that typically are non-convex, and may cause numerical search problems and estimates trapped in local minima. Recent interest in identification algorithms has focused on techniques that are centered around convex formulations. This is partly the result of developments in semidefinite programming, machine learning and statistical learning theory. The development concerns issues of regular-ization for sparsity and for better tuned bias/variance trade-offs. It also involves the use of subspace methods as well as nuclear norms as proxies to rank constraints. A special approach is to look for difference-of-convex programming (DCP) formulations, in case a pure convex criterion is not found. Other techniques are based on Lagrangian relaxation and contraction theory. A quite different route to convexity is to use algebraic techniques to manipulate the model parameterizations. This article will illustrate all this recent development.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Ljung, L., System Identification—Theory for the User, Upper Saddle River: Prentice Hall, 1999, 2nd ed.

    MATH  Google Scholar 

  2. Soderstrom, T. and Stoica, P., System Identification, London: Prentice Hall, 1989.

    MATH  Google Scholar 

  3. Pintelon, R. and Schoukens, J., System Identification—A Frequency Domain Approach, New York: Wiley-IEEE Press, 2012, 2nd ed.

    Book  MATH  Google Scholar 

  4. Tao, P.D. and An, L.T.H., Convex Analysis Approach to D.C. Programming: Theory, Algorithms and Applications, ACTA Math. Vietnamica, 1997, vol. 22, no. 1, pp. 289–355.

    MathSciNet  MATH  Google Scholar 

  5. Horst, R. and Thoai, N.V., DC Programming: Overview, J. Optim. Theory Appl., 1999, vol. 103, no. 1, pp. 1–43.

    Article  MathSciNet  MATH  Google Scholar 

  6. Thoai, R.H.N., DC Programming: An Overview, J. Optim. Theory Appl., 1999, vol. 193(1), pp. 1–43.

    MathSciNet  MATH  Google Scholar 

  7. Ljung, L. and Glad, T., On Global Identifiability of Arbitrary Model Parameterizations, Automatica, 1994, vol. 30, no. 2, pp. 265–276.

    Article  MathSciNet  MATH  Google Scholar 

  8. Tobenkin, M.M., Manchester, I.R., and Megretski, A., Convex Paramterizations and Fidelity Bounds for Nonlinear Identification and Reduced-Order Modelling, IEEE Trans. Autom. Control, 2017, vol. AC-62, no. 7, pp. 3679–3686.

    Article  MATH  Google Scholar 

  9. Manchester, I.R., Tobenkin, M.M., and Megretski, A., Stable Nonliner System Identifiction: Convexity, Model Class and Consistency, Proc. 16 IFAC Sympos. Syst. Identificat., Brussels, Belgium, July 2012, pp. 328–333.

    Google Scholar 

  10. Umenberger, J. and Manchester, I.R., Convex Bounds for Equation Error in Stable Nonlinear Identification, IEEE Control Syst. Lett., 2019, vol. 3, no. 1, pp. 73–79.

    Article  MATH  Google Scholar 

  11. Van Overschee, P. and De Moor, B., N4SID: Subspace Algorithms for the Identification of Combined Deterministic-Stochastic Systems, Automatica (Special Issue), 1994, vol. 30, no. 1, pp. 75–93.

    MATH  Google Scholar 

  12. Verhaegen, M. and Dewilde, P., The Output-Error State-Space Model Identification Class of Algorithms. Int J. Control, 1992, vol. 56, no. 5, pp. 1187–1210.

    Article  MATH  Google Scholar 

  13. Ljung, L. and Wahlberg, B., Asymptotic Properties of the Least-Squares Method for Estimating Transfer Functions and Disturbance Spectra, Adv. Appl. Prob., 1992, vol. 24, no. 2, pp. 412–440.

    Article  MathSciNet  MATH  Google Scholar 

  14. Tikhonov, A.N. and Arsenin, V.Y., Solutions of Ill-posed Problems, Washington: Winston/Wiley, 1977.

    MATH  Google Scholar 

  15. Chen, T., Ohlsson, H., and Ljung, L., On the Estimation of Transfer Functions, Regularizations and Gaussian Processes—Revisited, Automatica, 2012, vol. 48, no. 8, pp. 1525–1535.

    Article  MathSciNet  MATH  Google Scholar 

  16. Carlin, B.P. and Lewis, T.A., Bayes and Empirical Bayes Methods for Data Analysis, London: Chapman and Hall, 1996.

    MATH  Google Scholar 

  17. Chen, T. and Ljung, L., Implementation of Algorithms for Tuning Parameters in Regularized Least Squares Problems in System Identification, Automatica, 2013, vol. 49, no. 7, pp. 2213–2220.

    Article  MathSciNet  MATH  Google Scholar 

  18. Carli, F.P., Chiuso, A., and Pillonetto, C., Efficient Algorithms for Large Scale Linear System Identification Using Stable Spline Estimators, Proc. 16 IFAC Sympos. Syst. Identificat (SYSID 2012), 2012, pp. 119–124.

    Google Scholar 

  19. Ljung, L., The System Identification Toolbox: The Manual, Natick: The MathWorks Inc., USA, 1986, 1st ed., 2012, 8th ed.

    Google Scholar 

  20. Chen, T., Ljung, L., Andersen, M., Chiuso, A., Carli, P.F., and Pillonetto, C., Sparse Multiple Kernels for Impulse Response Estimation with Majorization Minimization Algorithms, IEEE Conf. Decision and Control, Hawaii, 2012, pp. 1500–1505.

    Google Scholar 

  21. Chen, T., Andersen, M.S., Ljung, L., Chiuso, A., and Pillonetto, C., System Identification via Sparse Multiple Kernel-Based Regularization Using Sequential Convex Optimization Techniques, Autom. Control. IEEE Transact, 2014, vol. 59, no. 11, pp. 2933–2945.

    Article  MathSciNet  MATH  Google Scholar 

  22. Dinuzzo, F., Kernels for Linear Time Invariant System Identification, Manuscript, Max Planck Institute for Intelligent Systems, Tubingen, Germany, 2012.

    MATH  Google Scholar 

  23. Pillonetto, G. and De Nicolao, G., A New Kernel-Based Approach for Linear System Identification, Automatica, 2010, vol. 46, no. 1, pp. 81–93.

    Article  MathSciNet  MATH  Google Scholar 

  24. Pillonetto, G., Chiuso, A., and De Nicolao, G., Prediction Error Identification of Linear Systems: A Non-parametric Gaussian Regression Approach, Automatica, 2011, vol. 47, no. 2, pp. 291–305.

    Article  MathSciNet  MATH  Google Scholar 

  25. Pillonetto, G., Dinuzzo, F., Chen, T., De Nicolao, G., and Ljung, L., Kernel Methods in System Identification, Machine Learning and Function Estimation: A Survey, Automatica, 2014, vol. 50, no. 3, pp. 657–682.

    Article  MathSciNet  MATH  Google Scholar 

  26. Mu, B., Chen, T., and Ljung, L., On Asymptotic Properties of Hyperparameter Estimators for Kernel-Based Regularization Methods, Automatica, 2018, vol. 94, no. 8, pp. 381–395.

    Article  MathSciNet  MATH  Google Scholar 

  27. Parrilo, P. and Ljung, L., Initialization of Physical Parameter Estimates, in Proc. 13 IFAC Sympos. Syst. Identificat., van der Hof, P., Wahlberg, B., and Weiland, S., Eds., Rotterdam, The Netherlands, 2003, pp. 1524–1529.

    Google Scholar 

  28. Xie, L.L. and Ljung, L., Estimate Physical Parameters by Black Box Modeling, Proc. 21 Chinese Control Conf, Hangzhou, China, 2002, pp. 673–677.

    Google Scholar 

  29. Yu, C., Ljung, L., and Verhaegen, M., Identification of Structured State-Space Models, Automatica, 2018, vol. 90, no. 4, pp. 54–61.

    Article  MathSciNet  MATH  Google Scholar 

  30. Yu, C., Ljung, L., and Verhaegen, M., Gray Box Identification Using Difference of Convex Programming, Proc. IFAC World Congress, Toulouse, France, 2017.

    Google Scholar 

  31. Wills, A., Yu, C., Ljung, L., and Verhaegen, M., Affinely Parametrized State-Space Models: Ways to Maximize the Likelihood Function, Proc. IFAC Sympos. Syst. Identificat. SYSID18, Stockholm, July 2018.

    Google Scholar 

  32. Ritt, J.F., Differential Algebra, Providence: Am. Math. Soc., 1950.

    Book  MATH  Google Scholar 

  33. Glad, S.T., Implementing Ritt's Algorithm of Differential Algebra, IFAC Sympos. Control Systems Design, NOLCOS'92, Bordeaux, France, 1992, pp. 398–401.

    Google Scholar 

  34. Ljung, L. and Chen, T., Convexity Issues in System Identification, Proc. 10 IEEE Int. Conf. Control Automat. (ICCA), Hangzhou, China, 2013.

    Google Scholar 

  35. Ljung, L., Pespectives on System Identification, IFAC Annual Reviews, Spring Issue, 2010.

    Google Scholar 

  36. Ljung, L., Hjalmarsson, H., and Ohlsson, H., Four Encounters with System Identification, Eur. J. Control, 2011, vol. 17, no. 5–6, pp. 449–471.

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Linköping University, Linköping, Sweden

    L. Ljung

Authors
  1. L. Ljung
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to L. Ljung.

Additional information

Russian Text © The Author(s), 2019, published in Avtomatika i Telemekhanika, 2019, No. 9, pp. 45–63.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ljung, L. On Convexification of System Identification Criteria. Autom Remote Control 80, 1591–1606 (2019). https://doi.org/10.1134/S0005117919090030

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S0005117919090030

Keywords

Search

Navigation