Skip to main content

Advertisement

Springer Nature Link
Log in

An Outlier-Robust Growing Local Model Network for Recursive System Identification

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

In this paper, we develop a self-growing variant of the local model network (LMN) for recursive dynamical system identification. The proposed model has the following features: growing online structure, fast recursive updating rules, better memory use (no storage of covariance matrices is required), and outlier-robustness. In this regard, efficiency in performance and simplicity of implementation are the essential qualities of the proposed approach. The proposed growing version of the LMN model results from a synergistic amalgamation of two simple but powerful ideas. For this purpose, we adapt the neuron insertion strategy of the resource-allocating network to LMN model, and replaces the standard OLS rule for parameter estimation with outlier-robust recursive rules. A comprehensive evaluation involving three SISO and one MIMO benchmarking data sets corroborates the proposed approach’s superior predictive performance in outlier-contaminated scenarios compared to fixed-size LMN-based models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. www.gnu.org/software/octave/.

  2. http://homes.esat.kuleuven.be/~smc/daisy/.

References

  1. Angelov PP, Filev DP (2004) An approach to online identification of Takagi-Sugeno fuzzy models. IEEE Trans Syst Man Cybern Part B 34(1):484–497

    Google Scholar 

  2. Barreto GA, Araújo AFR (2004) Identification and control of dynamical systems using the self-organizing map. IEEE Trans Neural Netw 15(5):1244–1259

    Google Scholar 

  3. Barreto GA, Souza LGM (2016) Novel approaches for parameter estimation of local linear models for dynamical system identification. Appl Intell 44(1):149–165

    Google Scholar 

  4. Belz J, Münker T, Heinz TO, Kampmann G, Nelles O (2017) Automatic modeling with local model networks for benchmark processes. IFAC-PapersOnLine 50(1):470–475

    Google Scholar 

  5. Bessa JA, Barreto GA (2019) Recursive system identification using outlier-robust local models. IFAC-PapersOnLine 52(1):436–441

    Google Scholar 

  6. Billings SA (2013) Nonlinear system identification: NARMAX methods in the time, frequency, and spatio-temporal domains. Wiley, London

    MATH  Google Scholar 

  7. Bittanti S, Picci G (1996) Identification, adaptation, learning: the science of learning models from data

  8. Blažič S, Škrjanc I (2020) Incremental fuzzy C-regression clustering from streaming data for local-model-network identification. IEEE Trans Fuzzy Syst 28(4):758–767

    Google Scholar 

  9. Brunton SL, Kutz JN (2019) Data-driven science and engineering: machine learning, dynamical systems, and control. Cambridge Press, Cambridge

  10. Chan S-C, Zhou Y (2010) On the performance analysis of the least mean M-estimate and normalized least mean M-estimate algorithms with gaussian inputs and additive gaussian and contaminated Gaussian noises. J Signal Process Syst 60(1):81–103

    Google Scholar 

  11. Chen S, Billings SA, Cowan CFN, Grant PM (1990) Non-linear systems identification using radial basis functions. Int J Syst Sci 21(12):2513–2539

    MATH  Google Scholar 

  12. Chou CT, Maciejowski JM (1997) System identification using balanced parametrizations. IEEE Trans Autom Control 42(7):956–974

    MATH  Google Scholar 

  13. Chuang C-C (2007) Fuzzy weighted support vector regression with a fuzzy partition. IEEE Trans Syst Man Cybern Part B 37(3):630–640

    Google Scholar 

  14. Costa TV, Fileti AMF, Oliveira-Lopes LC, Silva FV (2015) Experimental assessment and design of multiple model predictive control based on local model networks for industrial processes. Evol Syst 6(4):243–253

    Google Scholar 

  15. Foss BA, Johansen TA (1993) , On local and fuzzy modelling. In: Third International Conference on Industrial Fuzzy Control and Intelligent Systems (IFIS’93), pp 80–87

  16. Fritzke B (1995) A growing neural gas network learns topologies. In: Advances in neural information processing systems 7. MIT Press, pp 625–632

  17. Fritzke B (1995) Incremental learning of local linear mappings. In: Proceedings of the international conference on artificial neural networks (ICANN’95’), pp. 217–222

  18. Fuangkhon P (2014) An incremental learning preprocessor for feed-forward neural network. Artif Intell Rev 41(2):183–210

    Google Scholar 

  19. Hampel FR, Ronchetti EM, Rousseeuw PJ, Stahel WA (2011) Robust statistics: the approach based on influence functions, vol. 114. Wiley

  20. Han H, Chen Q, Qiao J (2010) Research on an online self-organizing radial basis function neural network. Neural Comput Appl 19:667–676

    Google Scholar 

  21. Hassibi B, Sayed AH, Kailath T (1996) H\(_\infty \) optimality of the LMS algorithm. IEEE Trans Signal Process 44(2):267–280

    Google Scholar 

  22. He H, Chen S, Li K, Xu X (2011) Incremental learning from stream data. IEEE Trans Neural Netw 22(12):1901–1914

    Google Scholar 

  23. Huber P (1964) Robust estimation of a location parameter. Ann Math Stat 35(1):73–101

    MathSciNet  MATH  Google Scholar 

  24. Jain S, Bandyopadhyay S (2017) Resource allocation network for segregated targeting problems with dedicated sources. Ind Eng Chem Res 56(46):13831–13843

    Google Scholar 

  25. Johansen TA, Foss BA (1992) A NARMAX model representation for adaptive control based on local models. Model Identif Control 13(1):25

    Google Scholar 

  26. Jones RD, Lee YC, Barnes CW, Flake GW, Lee K, Lewis PS, Qian S (1990) Function approximation and time series prediction with neural networks. In: Proceedings of the 1990 international joint conference on neural networks (IJCNN’1990)’, pp 649–665

  27. Kiumarsi B, Lewis FL, Levine SD (2015) Optimal control of nonlinear discrete time-varying systems using a new neural network approximation structure. Neurocomputing 156:157–165

  28. Kohonen T (2013) Essentials of the self-organizing map. Neural Netw 37:52–65

    Google Scholar 

  29. König O, Hametner C, Prochart G, Jakubek S (2014) Battery emulation for power-HIL using local model networks and robust impedance control. IEEE Trans Ind Electron 61(2):943–955

    Google Scholar 

  30. Kovačević B, Banjac Z, Kovačević IK (2016) Robust adaptive filtering using recursive weighted least squares with combined scale and variable forgetting factors. EURASIP J Adv Signal Process 37:1–22

    Google Scholar 

  31. Liu J, Djurdjanovic D, Marko K, Ni J (2009) Growing structure multiple model systems for anomaly detection and fault diagnosis. J Dyn Syst Meas Contr 131(5):1–13

    Google Scholar 

  32. Ljung L (1999) System identification: theory for the user, 2nd edn. Prentice Hall

  33. Lu T, Pan L, Jiangs J, Zhang Y, Xiong Z (2017) Dlml: deep linear mappings learning for face super-resolution with nonlocal-patch. In: 2017 IEEE international conference on multimedia and expo (ICME), IEEE, pp 1362–1367

  34. Lv C, Zhang T, Ma F, Yue D (2018) A very short-term online forecasting model for photovoltaic power based on two-stage resource allocation network. In: 2018 International joint conference on neural networks (IJCNN), IEEE, pp 1–6

  35. Mahanand BS, Suresh S, Sundararajan N, Kumar MA (2012) Identification of brain regions responsible for Alzheimer’s disease using a self-adaptive resource allocation network. Neural Netw 32:313–322

    Google Scholar 

  36. Maier CC, Schirrer A, Kozek M (2018) Real-time capable nonlinear pantograph models using local model networks in state-space configuration. Mechatronics 50:292–302

    Google Scholar 

  37. Mandic DP, Kanna S, Constantinides AG (2015) On the intrinsic relationship between the least mean square and Kalman filters. IEEE Signal Process Mag 32(6):117–122

    Google Scholar 

  38. Martinetz T (1993) , Competitive hebbian learning rule forms perfectly topology preserving maps. In: ICANN’93, Springer, pp 427–434

  39. Mattos CLC, Dai Z, Damianou A, Barreto GA, Lawrence ND (2017) Deep recurrent gaussian processes for outlier-robust system identification. J Process Control 60:82–94

    Google Scholar 

  40. Moody J, Darken CJ (1989) Fast learning in networks of locally-tuned processing units. Neural Comput 1(2):281–294

    Google Scholar 

  41. Moshou D, Ramon H (1997) Extended self-organizing maps with local linear mappings for function approximation and system identification. In: Proceedings of the 1st workshop on self-organizing maps (WSOM’97), pp 1–6

  42. Münker T, Heinz TO, Nelles O (2017) , Hierarchical model predictive control for local model networks. In: Proceedings of the American Control Conference (ACC’2017), pp 5026–5031

  43. Narendra K, Parthasarathy K (1990) Identification and control of dynamical systems using neural networks. IEEE Trans Neural Netw 1(1):4–27

    Google Scholar 

  44. Pai PS, Nagabhushana T, Rao PR (2001) Tool wear estimation using resource allocation network. Int J Mach Tools Manuf 41(5):673–685

    Google Scholar 

  45. Platt J (1991) A resource-allocating network for function interpolation. Neural Comput 3(2):213–225

    MathSciNet  Google Scholar 

  46. Poggio T, Girosi F (1990) Networks for approximation and learning. Proc IEEE 78(9):1484–1487

    MATH  Google Scholar 

  47. Prasad G, Swidenbank E, Hogg BW (1998) A local model networks based multivariable long-range predictive control strategy for thermal power plants. Automatica 34(10):1185–1204

    MATH  Google Scholar 

  48. Ribeiro AH, Aguirre LA (2018) Parallel training considered harmful?: Comparing series-parallel and parallel feedforward network training. Neurocomputing 316:222–231

    Google Scholar 

  49. Schoukens J, Nemeth JG, Crama P, Rolain Y, Pintelon R (2003) Fast approximate identification of nonlinear systems. Automatica 39(7):1267–1274

    MathSciNet  MATH  Google Scholar 

  50. Sjöberg J, Zhang Q, Ljung L, Benveniste A, Delyon B, Glorennec P-Y, Hjalmarsson H, Juditsky A (1995) Nonlinear black-box modeling in system identification: a unified overview. Automatica 31(12):1691–1724

    MathSciNet  MATH  Google Scholar 

  51. Soong TT (2004) Fundamentals of probability and statistics for engineers. Wiley, London

    MATH  Google Scholar 

  52. Stokbro K, Umberger DK, Hertz JA (1990) Exploiting neurons with localized receptive fields to learn chaos. Complex Syst 4(3):603–622

    MATH  Google Scholar 

  53. Suresh S, Dong K, Kim HJ (2010) A sequential learning algorithm for self-adaptive resource allocation network classifier. Neurocomputing 73(16–18):3012–3019

    Google Scholar 

  54. Takagi T, Sugeno M (1985) Fuzzy identification of systems and its applications to modelling and control. IEEE Trans Syst Man Cybern 15(1):116–132

    MATH  Google Scholar 

  55. Vachkov G (2004) , Growing model algorithm for process identification based on neural-gas learning and local linear mapping. In: Fourth international conference on hybrid intelligent systems (HIS’04), IEEE, pp 222–227

  56. Walter J, Ritter H, Schulten K (1990) , Nonlinear prediction with self-organizing maps. In: Proceedings of the IEEE international joint conference on neural networks (IJCNN’90), pp 589–594

  57. Wang Q, Wang X, Sun B (2019) Resource allocation network and PID control based on automated guided vehicles. Acad J Manuf Eng 17(2)

  58. Widrow B (2005) Thinking about thinking: the discovery of the LMS algorithm. IEEE Signal Process Mag 22(1):100–106

    Google Scholar 

  59. Widrow B, Lehr MA (1990) 30 years of adaptive neural networks: perceptron, madaline and backpropagation. Proc IEEE 78(9):1415–1442

    Google Scholar 

  60. Yan L, Sundararajan N, Saratchandran P (2000) Nonlinear system identification using Lyapunov based fully tuned dynamic RBF networks. Neural Process Lett 12(3):291–303

    MATH  Google Scholar 

  61. Zhao Z-L, Qiu Z-C, Zhang X-M, Han J-D (2016) Vibration control of a pneumatic driven piezoelectric flexible manipulator using self-organizing map based multiple models. Mech Syst Signal Process 70:345–372

    Google Scholar 

  62. Zou Y, Chan S-C, Ng T-S (2000) Least mean M-estimate algorithms for robust adaptive filtering in impulse noise. IEEE Trans Circuits Syst II: Analog Digit Signal Process 47(12):1564–1569

    Google Scholar 

  63. Zou Y, Chan SC, Ng TS (2001) Robust M-estimate adaptive filtering. IEE Proc Vis Image Signal Process 148(4):289–294

    Google Scholar 

Download references

Acknowledgements

This study was financed by the following Brazilian research funding agencies: CAPES (Finance Code 001), CNPq Grants No. 309379/2019-9 (2nd author) and 311211/2017-8 (3rd author).

Author information

Authors and Affiliations

  1. Graduate Program in Teleinformatics Engineering, Center of Technology, Federal University of Ceará, Campus of Pici, Fortaleza, Ceará, Brazil

    Jéssyca A. Bessa, Guilherme A. Barreto & Ajalmar R. Rocha-Neto

Authors
  1. Jéssyca A. Bessa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guilherme A. Barreto
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ajalmar R. Rocha-Neto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jéssyca A. Bessa.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Availability of data and materials

The data used in the experiments reported in this paper are available for public use and can be freely downloaded for scientific purposes from the website indicated in the text.

Code availability

The Octave/Matlab codes of the proposed approach are available for scientific purposes and can be provided upon request.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bessa, J.A., Barreto, G.A. & Rocha-Neto, A.R. An Outlier-Robust Growing Local Model Network for Recursive System Identification. Neural Process Lett 55, 4257–4289 (2023). https://doi.org/10.1007/s11063-022-11040-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-022-11040-z

Keywords