Skip to main content
SpringerLink
Log in

Novel approaches for parameter estimation of local linear models for dynamical system identification

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

In this paper we introduce two novel techniques for local linear modeling of dynamical systems. As in the standard approach, we use vector quantization (VQ) algorithms, such as the Self-Organizing Map, to partition the joint input-output space into smaller regions. Then, to each neuron we associate a vector of parameters which must be suitably estimated. The first estimation technique uses the prototypes of the i-th neuron and its K nearest neighbors to build the corresponding local linear model. The second technique builds the i-th local linear model using the data vectors that are mapped into the regions comprised of the Voronoi cells of the i-th neuron and its K nearest neighbors. A comprehensive evaluation of the proposed techniques is carried out for the task of inverse identification of three benchmarking Single Input/Single Output (SISO) dynamical systems. Their performances are compared to those achieved by the Multilayer Perceptron and the Extreme Learning Machine networks. We also evaluate how robust are the proposed techniques with respect to the VQ algorithm used to partition the input-output space. The results show that proposed techniques consistently outperform standard approaches for all evaluated datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. NARX stands for Nonlinear Autoregressive model with exogenous inputs.

  2. That is, instead of using the recursive LMS rule, the D-MKSOM model uses the standard (non-recursive) least-squares method.

  3. http://www.esat.kuleuven.ac.be/sista/daisy/

  4. By the original MKSOM models, we mean the P- and D-MKSOM models that use the VQTAM method with the SOM as the VQ algorithm.

References

  1. Abonyi J (2003) Fuzzy Model Identification for Control. Birkhäuser

  2. Abonyi J, Nemeth S, Vincze C, Arva P (2003) Process analysis and product quality estimation by self-organizing maps with an application to polyethylene production. Comput Ind 52(3):221–234

    Article  Google Scholar 

  3. Ahalt S, Krishnamurthy A, Cheen P, Melton D (1990) Competitive learning algorithms for vector quantization. Neural Netw 3(3):277–290

    Article  Google Scholar 

  4. Andrášik A, Mészáros A, de Azevedo S (2004) On-line tuning of a neural PID controller based on plant hybrid modeling. Comput Chem Eng 28(8):1499–1509

    Article  Google Scholar 

  5. Azeem MF, Hanmandlu M, Ahmad N (2000) Generalization of adaptive neuro-fuzzy inference systems. IEEE Transactions on Neural Networks 11(6):1332–1346

    Article  Google Scholar 

  6. Babuška R, Verbruggen H (2003) Neuro-fuzzy methods for nonlinear system identification. Annu Rev Control 27:73–85

    Article  Google Scholar 

  7. Barreto GA, Aguayo L (2009). In: Príncipe JC, Miikkulainen R (eds) Time series clustering for anomaly detection using competitive neural networks. Springer, pp 28–36

  8. Barreto GA, Araújo AFR (2004) Identification and control of dynamical systems using the self-organizing map. IEEE Transactions on Neural Networks 15(5):1244–1259

    Article  Google Scholar 

  9. Barreto GA, Araújo AFR (2004) Identification and control of dynamical systems using the self-organizing map. IEEE Transactions on Neural Networks 15(5):1244–1259

    Article  Google Scholar 

  10. Barreto GA, Souza LGM (2006) Adaptive filtering with the self-organizing maps: A performance comparison. Neural Netw 19(6):785–798

    Article  MATH  Google Scholar 

  11. Barreto GA, Araújo AFR, Ritter HJ (2003) Self-organizing feature maps for modeling and control of robotic manipulators. J Intell Robot Syst 36(4):407–450

    Article  MATH  Google Scholar 

  12. Berglund E, Sitte J (2006) The parameterless self-organizing map algorithm. IEEE Transactions on Neural Networks 17(2):305–316

    Article  Google Scholar 

  13. Billings SA, Voon WSF (1983) Structure detection and model validity tests in the identification of nonlinear systems. IEE Proceedings, Part D, Control Theory and Applications 130(4):193–199

    Article  MATH  Google Scholar 

  14. Billings SA, Voon WSF (1986) Correlation based model validity tests for nonlinear models. Int J Control 44(1):235–244

    Article  MathSciNet  MATH  Google Scholar 

  15. Billings SA, Zhu QM (1994) Nonlinear model validation using correlation tests. Int J Control 60(6):1107–1120

    Article  MathSciNet  MATH  Google Scholar 

  16. Bittanti S, Piroddi L (1997) Nonlinear identification and control of a heat exchanger: A neural network approach. J Frankl Inst 334(1):135–153

    Article  Google Scholar 

  17. Chen JQ, Xi YG (1998) Nonlinear system modeling by competitive learning and adaptive fuzzy inference system. IEEE Trans Syst Man Cybern C 28(2):231–238

    Article  Google Scholar 

  18. Chen S, Billings SA, Grant PM (1990) Nonlinear system identification using neural networks. Int J Control 51:1191–1214

    Article  MathSciNet  MATH  Google Scholar 

  19. Cho J, Principe J, Erdogmus D, Motter M (2006) Modeling and inverse controller design for an unmanned aerial vehicle based on the self-organizing map. IEEE Transactions on Neural Networks 17(2):445–460

    Article  Google Scholar 

  20. Cho J, Principe J, Erdogmus D, Motter M (2007) Quasi-sliding mode control strategy based on multiple linear models. Neurocomputing 70(4-6):962–974

    Google Scholar 

  21. Daosud W, Thitiyasook P, Arpornwichanop A, Kittisupakorn P, Hussain MA (2005) Neural network inverse model-based controller for the control of a steel pickling process. Comput Chem Eng 29(10):2110–2119

    Article  Google Scholar 

  22. Darken C, Moody J (1990) Fast adaptive k-means clustering: Some empirical results. In: Proceedings of the international joint conference on neural networks (IJCNN’90), vol 2, pp 233–238

  23. Gan M, Peng H, Chen L (2012) A global-local optimization approach to parameter estimation of RBF-type models. Inf Sci 197:144–160

    Article  Google Scholar 

  24. Göppert J, Rosenstiel W (1993) Topology preserving interpolation in selforganizing maps. In: Proceedings of the NeuroNIMES’93, pp 425–434

  25. Göppert J, Rosenstiel W (1995) Topological interpolation in SOM by affine transformations. In: Proceedings of the european symposium on artificial neural networks (ESANN’95), pp 15–20

  26. Gregorčič G, Lightbody G (2007) Local model network identification with gaussian processes. IEEE Transactions on Neural Networks 18(5):1404–1423

    Article  Google Scholar 

  27. Gregorčič G, Lightbody G (2008) Nonlinear system identification: from multiple-model networks to gaussian processes. Eng Appl Artif Intell 21(7):1035–1055

    Article  Google Scholar 

  28. Hametner C, Jakubek S (2013) Local model network identification for online engine modelling. Inf Sci 220:210–225

    Article  Google Scholar 

  29. Huang G, Huang GB, Song S, You K (2015) Extreme learning machine: Theory and applications. Neural Netw 61(1):32–48

    Article  Google Scholar 

  30. Huang GB, Zhu QY, Ziew CK (2006) Extreme learning machine: Theory and applications. Neurocomputing 70(1–3):489–501

    Article  Google Scholar 

  31. Hunt KJ, Sbarbaro D, Zbikowski R, Gawthrop P (1992) Neural networks for control systems: a survey. Automatica 28(6):1083–1111

    Article  MathSciNet  MATH  Google Scholar 

  32. Hussain MA (1996) Inverse model control strategies using neural networks: Analysis, simulation and on-line implementation. PhD thesis, University of London

  33. Hussain MA, Kershenbaum LS (2000) Implementation of an inverse-model-based control strategy using neural networks on a partially simulated exothermic reactor. Chem Eng Res Des 78(2):299–311

    Article  Google Scholar 

  34. Kim E, Lee H, Park M, Park M (1998) A simply identified Sugeno-type fuzzy model via double clustering. Inf Sci 110(1–2):25–39

    Article  Google Scholar 

  35. Kohonen T (2013) Essentials of the self-organizing map. Neural Netw 37:52–65

    Article  Google Scholar 

  36. Kohonen TK, Oja E, Simula O, Visa A, Kangas J (1996) Engineering applications of the self-organizing map. Proc IEEE 84(10):1358–1384

    Article  Google Scholar 

  37. Li X, Yu W (2002) Dynamic system identification via recurrent multilayer perceptrons. Inf Sci 147 (1–4):45–63

    Article  MATH  Google Scholar 

  38. Lightbody G, Irwin GW (1997) Nonlinear control structures based on embedded neural system models. IEEE Transactions on Neural Networks 8(3):553–567

    Article  Google Scholar 

  39. Lima CAM, Coelho ALV, Von Zuben FJ (2007) Hybridizing mixtures of experts with support vector machines: Investigation into nonlinear dynamic systems identification. Inf Sci 177(10):2049–2074

    Article  Google Scholar 

  40. Liu J, Djurdjanovic D (2008) Topology preservation and cooperative learning in identification of multiple model systems. IEEE Transactions on Neural Networks 19(12):2065–2072

    Article  Google Scholar 

  41. Ljung L (1999) System Identification: Theory for the user, 2nd edn. Prentice-Hall, Englewood Cliffs

    Google Scholar 

  42. MacQueen J (1967) Some methods for classification and analysis of multivariate observations. In: Le Cam LM, Neyman J (eds) Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, vol 1, pp 281–297. University of California Press, Berkeley

    Google Scholar 

  43. Mu C, Sun C, Yu X (2011) Internal model control based on a novel least square support vector machines for MIMO nonlinear discrete systems. Neural Comput Applic 20(8):1159–1166

    Article  Google Scholar 

  44. Murray-Smith R, Gollee H (1994) A constructive learning algorithm for local model networks. In: IEEE workshop on computer-intensive methods in control and signal processing, pp 21–29

  45. Murray-Smith R, Hunt KJ (1995). In: Hunt KJ, Irwin GR, Warwick K (eds) Local model architectures for nonlinear modelling and control. Springer, Neural network engineering in dynamic control systems, pp 61–82

  46. Narendra KS (1996) Neural networks for control theory and practice. Proc IEEE 84(10):1385–1406

    Article  Google Scholar 

  47. Narendra KS, Lewis FL (2001) Special issue on neural networks feedback control. Automatica 37(8)

  48. Narendra KS, Parthasarathy K (1990) Identification and control of dynamical systems using neural networks. IEEE Transactions on Neural Networks 1(1):4–27

    Article  Google Scholar 

  49. Norgaard M, Ravn O (2000) Neural Networks for Modelling and Control of Dynamic Systems. Springer-Verlag, Hansen LK

    Book  Google Scholar 

  50. Papadakis SE, Kaburlasos VG (2010) Piecewise-linear approximation of non-linear models based on probabilistically/possibilistically interpreted intervals numbers (INs). Inf Sci 180(24):5060–5076

    Article  MATH  Google Scholar 

  51. Peng H, Nakano K, Shioya H (2007) A comprehensive review for industrial applicability of artificial neural networks. IEEE Trans Control Syst Technol 15(1):130–143

    Article  Google Scholar 

  52. Principe JC, Wang L, Motter MA (1998) Local dynamic modeling with self-organizing maps and applications to nonlinear system identification and control. Proc IEEE 86(11):2240–2258

    Article  Google Scholar 

  53. Rezaee B, Fazel Zarandi M (2010) Data-driven fuzzy modeling for TakagiSugenoKang fuzzy system. Inf Sci 180(2):241–255

    Article  Google Scholar 

  54. Rubio JJ (2009) SOFMLS: Online self-organizing fuzzy modified least-squares network. IEEE Trans Fuzzy Syst 17(6):1296–1309

    Article  MathSciNet  Google Scholar 

  55. Shorten R, Murray-Smith R, Bjørgan R, Gollee H (1999) On the interpretation of local models in blended multiple model structures. Int J Control 72(7–8):620–628

    Article  MATH  Google Scholar 

  56. Sjöberg J, Zhang Q, Ljung L, Benveniste A, Deylon B, Glorennec PY, Hjalmarsson H, Juditsky A (1995) Nonlinear black-box modeling in system identification: A unified overview. Automatica 31(12):1691–1724

    Article  MathSciNet  MATH  Google Scholar 

  57. Soong TT (2004) Fundamentals of Probability and Statistics for Engineers, 1st edn. Wiley, West Sussex

    MATH  Google Scholar 

  58. Souza LGM, Barreto GA (2010) On building local models for inverse system identification with vector quantization algorithms. Neurocomputing 73(10–12):1993–2005

    Article  Google Scholar 

  59. Takagi T, Sugeno M (1985) Fuzzy identification of systems and its application to modeling and control. IEEE Trans Syst Man Cybern 15(1):116–132

    Article  MATH  Google Scholar 

  60. Teslić L, Hartmann B, Nelles O, Škrjanc I (2011) Nonlinear system identification by gustafson-kessel fuzzy clustering and supervised local model network learning for the drug absorption spectra process. IEEE Transactions on Neural Networks 22(12):1941–1951

    Article  Google Scholar 

  61. Vasuki A, Vanathi PT (2006) A review of vector quantization techniques. IEEE Potentials 25(4):39–47

    Article  Google Scholar 

  62. Walter J, Ritter H, Schulten K (1990) Non-linear prediction with self-organizing map. In: Proceedings of the IEEE international joint conference on neural networks (IJCNN’90), vol 1, pp 587–592

  63. Yu W (2004) Nonlinear system identification using discrete-time recurrent neural networks with stable learning algorithm. Inf Sci 158:131–157

    Article  MATH  Google Scholar 

Download references

Acknowledgments

The first author thanks CNPq for the financial support through the grant no. 309841/2012-7. The second author thanks FUNCAP and CAPES for the scholarships granted along the development of this research. Both authors thanks NUTEC (Fundação Núcleo de Tecnologia Industrial do Ceará) for providing the laboratory infrastructure for the execution of the computer experiments reported in this paper.

Author information

Authors and Affiliations

  1. Department of Teleinformatics Engineering, Av. Mister Hull, S/N, Campus of Pici, Center of Technology, Federal University of Ceará, CEP 60455-970, Fortaleza, Ceará, Brazil

    Guilherme A. Barreto

  2. Department of Electrical Engineering, Av. Ministro Petrônio Portela, S/N - Campus of Ininga, Center of Technology, Federal University of Piauí, CEP 64049-550, Teresina, Piauí, Brazil

    Luís Gustavo M. Souza

Authors
  1. Guilherme A. Barreto
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Luís Gustavo M. Souza
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guilherme A. Barreto.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Barreto, G.A., M. Souza, L.G. Novel approaches for parameter estimation of local linear models for dynamical system identification. Appl Intell 44, 149–165 (2016). https://doi.org/10.1007/s10489-015-0699-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-015-0699-1

Keywords

Search

Navigation