Abstract
Evolving systems unfolds from the interaction and cooperation between systems with adaptive structures, and recursive methods of machine learning. They construct models and derive decision patterns from stream data produced by dynamically changing environments. Different components that assemble the system structure can be chosen, being rules, trees, neurons, and nodes of graphs amongst the most prominent. Evolving systems relate mainly with time-varying environments, and processing of nonstationary data using computationally efficient recursive algorithms. They are particularly appropriate for online, real-time applications, and dynamically changing situations or operating conditions. This paper gives an overview of evolving systems with focus on system components, learning algorithms, and application examples. The purpose is to introduce the main ideas and some state-of-the-art methods of the area as well as to guide the reader to the essential literature, main methodological frameworks, and their foundations.
Similar content being viewed by others
References
Abonyi J, Babuška R, Szeifert F (2002) Modified gath-geva fuzzy clustering for identification of takagi-sugeno fuzzy models. IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) 32(5):612–621
Agrawal R, Bala R (2008) Incremental bayesian classification for multivariate normal distribution data. Pattern Recognit Lett 29:1873–1876
Andonovski G, Angelov P, Blažič S, Škrjanc I (2016) A practical implementation of robust evolving cloud-based controller with normalized data space for heat-exchanger plant. Appl Soft Comput 48:29–38
Angelov PP (2002) Evolving rule-based models: a tool for design of flexible adaptive systems. Springer-Verlag, London
Angelov P (2012) Autonomous learning systems: from data streams to knowledge in real-time. Wiley, Hoboken
Angelov P, Filev D (2004) An approach to Online identification of Takagi–Suigeno fuzzy models. IEEE Trans Syst Man Cybern Part B-Cybern 34(1):484–498
Angelov P, Filev D, Kasabov N (2010) Evolving intelligent systems: methodology and applications. Wiley-IEEE Press, New Jersey
Angelov P, Kordon A (2010) Adaptive inferential sensors based on evolving fuzzy models. IEEE Trans Syst Man Cybern Part B Cybern 40(2):529–539
Angelov P, Lughofer E, Zhou X (2008) Evolving fuzzy classifiers using different model architectures. Fuzzy Sets Syst 159(23):3160–3182
Angelov P, Zhou X (2008) Evolving fuzzy-rule-based classifiers from data streams. IEEE Trans Fuzzy Syst 16:1462–1475
Angelov P, Ramezani R, Zhou X (2008) Autonomous novelty detection and object tracking in video streams using evolving clustering and takagi-sugeno type neuro-fuzzy system. In: IEEE international joint conference on neural networks (IJCNN), pp. 1456–1463
Angelov P, Yager R (2011) Simplified fuzzy rule-based systems using non-parametric antecedents and relative data density. In: 2011 IEEE Workshop on Evolving and Adaptive Intelligent Systems (EAIS), pp. 62–69
Angelov P, Zhou X (2006) Evolving fuzzy systems from data streams in real-time. In: 2006 International Symposium on Evolving Fuzzy Systems, pp. 29–35
Angelov P (2010) Evolving Intelligent Systems: Methodology and Applications, chap. Evolving Takagi-Sugeno Fuzzy Systems From Streaming Data (eTS+), pp. 21 – 50. Wiley, New Jersey
Angelov P (2010) Evolving takagi-sugeno fuzzy systems from streaming data, ets+. In: P. Angelov, D. Filev, N. Kasabov (eds.) Evolving Intelligent Systems: Methodology and Applications. Wiley-Interscience/IEEE Press
Angelov P, Gu X (2017) Mice: Multi-layer multi-model images classifier ensemble. In: Proceedings of the IEEE International Conference Cybernetics, pp. 1–8
Azeem MF, Hanmandlu H, Ahmad N (2003) Structure identification of generalized adaptive neurofuzzy inference systems. IEEE Trans Fuzzy Syst 11:666–681
Beliakov G, Pradera A, Calvo T (2007) Aggregation Functions: A Guide for Practitioners, 1st edn. Springer - Studies in Fuzziness and Soft Comput, Vol 21
Blažič S, Angelov P, Škrjanc I (2015) Comparison of approaches for identification of all-data cloud-based evolving systems. IFAC-PapersOnLine 48(10):129–134
Blažič S, Škrjanc I, Matko D (2014) A robust fuzzy adaptive law for evolving control systems. Evol Syst 5(1):3–10
Blažič S, Dovžan D, Škrjanc I (2014) Cloud-based identification of an evolving system with supervisory mechanisms. In: Proceedings of IEEE Control Systems Society Multiconference on Systems and Control, pp. 1906–1911
Bodyanskiy Y, Tyshchenko O, Kopaliani D (2016) Adaptive learning of an evolving cascade neo-fuzzy system in data stream mining tasks. Evol Syst 7(2):107–116
Bodyanskiy Y, Vynokurova O, Volkova V, Boiko O (2018) 2d-neo-fuzzy neuron and its adaptive learning. Inf Technol Manag Sci 21:24–28
Bordes A, Bottou L (2005) The huller: a simple and efficient online svm. In: Proceedings of the European Conf on Machine Learning. Springer, New York, pp. 505–512
Bordignon F, Gomide F (2014) Uninorm based evolving neural networks and approximation capabilities. Neurocomputing 127:13–20
Bueno L, Costa P, Mendes I, Cruz E, Leite D (2015) Evolving ensemble of fuzzy models for multivariate time series prediction. In: Proceedings of the IEEE Int Conf on Fuzzy Systems (FUZZ-IEEE), pp. 1–6
Doborjeh M, Kasabov N, Doborjeh ZG (2018) Evolving, dynamic clustering of spatio/spectro-temporal data in 3d spiking neural network models and a case study on eeg data. Evol Syst 9:195–211
Dovžan D, Logar V, Škrjanc I (2015) Implementation of an evolving fuzzy model (eFuMo) in a monitoring system for a waste-water treatment process. IEEE Trans Fuzzy Syst 23(5):1761–1776
Dovžan D, Škrjanc I (2011) Recursive clustering based on a Gustafson-Kessel algorithm. Evol Syst 2(1):15–24
Ferdaus M, Pratama M, Anavatti S, Garratt M (2019) Palm: An incremental construction of hyperplanes for data stream regression. IEEE Transactions on Fuzzy Systems, 15p., https://doi.org/10.1109/TFUZZ.2019.2893565
Filev D, Tseng F (2006) Novelty detection based machine health prognostics. In: 2006 International Symposium on Evolving Fuzzy Systems, pp. 193–199
Fritzke B (1994) Growing cell structures: a self-organizing network for unsupervised and supervised learning. Neural Netw 7:1441–1460
Garcia C, Leite D, Škrjanc I (2019) Incremental missing-data imputation for evolving fuzzy granular prediction. IEEE Transactions on Fuzzy Systems p. 15p. https://doi.org/10.1109/TFUZZ.2019.2935688
Hapfelmeier A, Pfahringer B, Kramer S (2014) Pruning incremental linear model trees with approximate lookahead. IEEE Trans Knowl Data Eng 26(8):2072–2076
Heeswijk M, Miche Y, Lindh-Knuutila T, Hilbers P, Honkela T, Oja E, Lendasse A (2009) Adaptive ensemble models of extreme learning machines for time series prediction. In: C. Alippi, P.C. Polycarpou M., G. Ellinas (eds.) Artificial Neural Networks - ICANN Lecture Notes in Computer Science. Springer - Berlin
Hisada M, Ozawa S, Zhang K, Kasabov N (2010) Incremental linear discriminant analysis for evolving feature spaces in multitask pattern recognition problems. Evol Syst 1:17–27
Iglesias JA, Ledezma A, Sanchis A (2014) An ensemble method based on evolving classifiers: estacking. In: Proceedings of the IEEE Symp on Evolving and Autonomous Learning Systems (EALS), pp. 1–8
Ikonomovska E, Gama J Learning model trees from data streams. In: J.F. Boulicaut, M. Berthold, T. Horváth (eds.) Discovery Science, Lecture Notes in Computer Science, vol. 5255, pp. 52–63. Springer Berlin, Heidelberg
Ikonomovska E, Gama J, Sebastião R, Gjorgjevik D (2009) Regression trees from data streams with drift detection. In: Proceedings of the 12th International Conference on Discovery Science, DS ’09, pp. 121–135
Janikow C (1998) Fuzzy decision trees: issues and methods. IEEE Trans Syst Man Cybern Part B-Cybern 28(1):1–14
Kangin D, Angelov P, Iglesias JA, Sanchis A (2015) Evolving classifier tedaclass for big data. Procedia Comput Sci 53:9–18
Kasabov N (2007) Evolving connectionist systems: the knowledge engineering approach. Springer-Verlag, New York Inc, Secaucus
Kasabov N (2014) Neucube: a spiking neural network architecture for mapping, learning and understanding of spatio-temporal brain data. Neural Netw 52:62–76
Kasabov N, Song Q (2002) DENFIS: dynamic evolving neural-fuzzy inference system and its application for time-series prediction. IEEE Trans Fuzzy Syst 10(2):144–154
Klančar G, Škrjanc I (2015) Evolving principal component clustering with a low run-time complexity for LRF data mapping. Appl Soft Comput 35:349–358
Kolter J, Maloof M (2007) Dynamic weighted majority: an ensemble method for drifting concepts. J Mach Learn Res 8:2755–2790
Kwok TY, Yeung DY (1997) Constructive algorithms for structure learning in feedforward neural networks for regression problems. IEEE Trans Neural Netw 8(3):630–645
Leite D, Ballini R, Costa P, Gomide F (2012) Evolving fuzzy granular modeling from nonstationary fuzzy data streams. Evol Syst 3:65–79
Leite D, Costa P, Gomide F (2013) Evolving granular neural networks from fuzzy data streams. Neural Netw 38:1–16
Leite D, Costa P, Gomide F (2010) Granular approach for evolving system modeling. In: Hullermeier E, Kruse R, Hoffmann F (eds) Computational intelligence for knowledge-based systems design, vol 6178. Springer, Berlin - Heidelberg, pp 340–349 Lecture Notes in Computer Science
Leite D, Costa P, Gomide F (2012) Interval approach for evolving granular system modeling. In: Sayed-Mouchaweh M, Lughofer E (eds) Learning in non-stationary environments. Springer, New York, pp 271–300
Leite D, Palhares R, Campos V, Gomide F (2015) Evolving granular fuzzy model-based control of nonlinear dynamic systems. IEEE Trans Fuzzy Syst 23:923–938
Leite D, Škrjanc I (2019) Ensemble of evolving optimal granular experts, owa aggregation, and time series prediction. Inf Sci 504:95–112
Leite D (2012) Evolving granular systems. Ph.D. thesis, University of Campinas, School of Electrical and Computer Engineering
Leite D (2019) Comparison of genetic and incremental learning methods for neural network-based electrical machine fault detection. In: E. Lughofer, M. Sayed-Mouchaweh (eds.) Predictive Maintenance in Dynamic Systems, pp. 231–268. Springer - Cham
Leite D, Andonovski G, Škrjanc I, Gomide F (2019) Optimal rule-based granular systems from data streams. IEEE Transactions on Fuzzy Systems, 14p., https://doi.org/10.1109/TFUZZ.2019.2911493
Leite D, Costa P, Gomide F (2010) Evolving granular neural network for semi-supervised data stream classification. In: International Joint Conference on Neural Networks (IJCNN), pp. 1–8
Leite D, Santana M, Borges A, Gomide F (2016) Fuzzy granular neural network for incremental modeling of nonlinear chaotic systems. In: Proceedings of the IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), pp. 64–71
Lemos A, Caminhas W, Gomide F (2010) Fuzzy multivariable gaussian evolving approach for fault detection and diagnosis. In: Hullermeier E, Kruse R, Hoffmann F (eds) Computational intelligence for knowledge-based systems design, vol 6178. Springer, Berlin / Heidelberg, pp 360–369 Lecture Notes in Computer Science
Lemos A, Caminhas W, Gomide F (2013) Evolving intelligent systems: methods, algorithms and applications. In: Ramanna S, Jain L, Howlett R (eds) Emerging paradigms in machine learning. Springer, New York
Lemos A, Gomide F, Caminhas W (2011) Fuzzy evolving linear regression trees. Evol Syst 2(1):1–14
Lemos A, Gomide F, Caminhas W (2011) Multivariable gaussian evolving fuzzy modeling system. IEEE Trans Fuzzy Syst 19(1):91–104
Leng G, Prasad G, McGinnty TM (2004) An online algorithm for creating self-organizing fuzzy neural networks. Neural Netw 17:1477–1493
Lima E, Hell M, Ballini R, Gomide F (2010) Evolving fuzzy modeling using participatory learning. In: P. Angelov, D. Filev, N. Kasabov (eds.) Evolving intelligent systems: methodology and Applications. Wiley-Interscience/IEEE Press
Ljung L (1999) System Identification. Prentice-Hall, Upper Saddle River
Lughofer E (2008) Extensions of vector quantization for incremental clustering. Pattern Recognit 41(3):995–1011
Lughofer ED (2008) FLEXFIS: a robust incremental learning approach for evolving Takagi-Sugeno fuzzy models. IEEE Trans Fuzzy Syst 16(6):1393–1410
Lughofer E, Bouchot JL, Shaker A (2011) Online elimination of local redundancies in evolving fuzzy systems. Evol Syst 2:165–187
Lughofer E, Buchtala O (2013) Reliable all-pairs evolving fuzzy classifiers. IEEE Trans Fuzzy Syst 21:625–641
Lughofer E, Cernuda C, Kindermann S, Pratama M (2015) Generalized smart evolving fuzzy systems. Evol Syst 6:269–292
Lughofer E, Pratama M, Škrjanc I (2018) Incremental rule splitting in generalized evolving fuzzy systems for autonomous drift compensation. IEEE Trans Fuzzy Syst 26(4):1854–1865
Lughofer E (2011) Evolving Fuzzy Systems: Methodologies, Advanced Concepts and Applications, vol. 266. Studies in Fuzziness and Soft Computing Series, J. Kacprzyk (Ed.), Springer-Verlag, Berlin Heidelberg
Maciel L, Ballini R, Gomide F (2017) An evolving possibilistic fuzzy modeling approach for value-at-risk estimation. Appl Soft Comput 60:820–830
Maciel L, Ballini R, Gomide F (2018) Evolving fuzzy modelling for yield curve forecasting. Int J Econ Bus Res 15:290–311
Malcangi M, Grew P (2017) Evolving connectionist method for adaptive audiovisual speech recognition. Evol Syst 8:85–94
Malcangi M, Quan H, Vaini E, Lombardi P, Rienzo M (2018) Evolving fuzzy-neural paradigm applied to the recognition and removal of artefactual beats in continuous seismocardiogram recordings. Evol Syst, 10p., https://doi.org/10.1007/s12530-018-9238-8
Pedrycz W, Gomide F (2007) Fuzzy systems engineering: toward human-centric computing. Wiley Interscience, NJ
Polikar R (2006) Ensemble based systems in decision making. IEEE Circuits Syst Mag 6:21–45
Potts D (2004) Incremental learning of linear model trees. In: ICML ’04: Proceedings of the twenty-first international conference on Machine learning, p. 84. ACM, New York, NY, USA
Prasad M, Za’in C, Pratama M, Lughofer E, Ferdaus M, Cai Q (2018) Big data analytics based on panfis mapreduce. Procedia Comput Sci 144:140–152
Pratama M, Anavatti S, Angelov P, Lughofer E (2014) Panfis: a novel incremental learning machine. IEEE Trans Neural Netw Learn Syst 25:55–67
Pratama M, Anavatti S, Lu J (2015) Recurrent classifier based on an incremental meta-cognitive scaffolding algorithm. IEEE Trans Fuzzy Syst 23:2048–2066
Pratama M, Lu J, Lughofer E, Zhang G, Er M (2016) An incremental learning of concept drifts using evolving type-2 recurrent fuzzy neural networks. IEEE Trans Fuzzy Syst 25:1175–1192
Pratama M, Pedrycz W, Lughofer E (2018) Evolving ensemble fuzzy classifier. IEEE Trans Fuzzy Syst 26:2552–2567
Pratama M, Wang D (2019) Deep stacked stochastic configuration networks for lifelong learning of non-stationary data streams. Inf Sci 495:150–174
Precup RE, Teban TA, Albu A, Szedlak-Stinean AI, Bojan-Dragos CA (2018) Experiments in incremental online identification of fuzzy models of finger dynamics. Rom J Inf Sci Technol 21:358–376
Rong HJ, Sundararajan N, Huang GB, Zhao GS (2011) Extended sequential adaptive fuzzy inference system for classification problems. Evol Syst 2:71–82
Rubio JJ (2009) Sofmls: online self-organizing fuzzy modified least square network. IEEE Trans Fuzzy Syst 17:1296–1309
Rubio JJ (2014) Evolving intelligent algorithms for the modelling of brain and eye signals. Appl Soft Comput 14:259–268
Rubio JJ (2017) Usnfis: uniform stable neuro fuzzy inference system. Neurocomputing 262:57–66
Rubio JJ, Bouchachia A (2017) Msafis: an evolving fuzzy inference system. Soft Comput 21:2357–2366
Silva A, Caminhas W, Lemos A, Gomide F (2013) A fast learning algorithm for evolving neo-fuzzy neuron. Appl Soft Comput 14:194–209
Silva A, Caminhas W, Lemos A, Gomide F (2015) Adaptive input selection and evolving neural fuzzy networks modeling. Int J Comput Intell Syst 8:3–14
Silva S, Costa P, Gouvea M, Lacerda A, Alves F, Leite D (2018) High impedance fault detection in power distribution systems using wavelet transform and evolving neural network. Electr Power Syst Res 154:474–483
Silva S, Costa P, Santana M, Leite D (2018) Evolving neuro-fuzzy network for real-time high impedance fault detection and classification. Neural Comput & Applic, 14p., https://doi.org/10.1007/s00521-018-3789-2
Soares E, Costa P, Costa B, Leite D (2018) Ensemble of evolving data clouds and fuzzy models for weather time series prediction. Appl Soft Comput 64:445–453
Soares E, Camargo H, Camargo S, Leite D (2018) Incremental gaussian granular fuzzy modeling applied to hurricane track forecasting. In: IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), pp. 1–8
Soleimani-B H, Lucas C, Araabi BN (2010) Recursive gath-geva clustering as a basis for evolving neuro-fuzzy modeling. Evol Syst 1:59–71
Takagi T, Sugeno M (1985) Fuzzy identification of systems and its applications to modeling and control. IEEE Trans Syst Man Cybern 15(1):116–132
Tung S, Quek C, Guan C (2013) et2fis: an evolving type-2 neural fuzzy inference system. Inf Sci 220:124–148
Tzafestas SG, Zikidis KC (2001) Neurofast: on-line neuro-fuzzy art-based structure and parameter learning tsk model. IEEE Trans Syst Man Cybern— Part B 31:797–802
Wang W, Vrbanek J (2008) An evolving fuzzy predictor for industrial applications. IEEE Trans Fuzzy Syst 16(6):1439–1449
Williamson JR (1996) Gaussian ARTMAP: a neural network for past incremental learning of noisy multidimensional maps. Neural Netw 9(5):881–897
Wu S, Er MJ (2000) Dynamic fuzzy neural networks—a novel approach to function approximation. IEEE Trans Syst Man Cybern—Part B 30:358–364
Wu S, Er MJ, Gao Y (2001) A fast approach for automatic generation of fuzzy rules by generalized dynamicc fuzzy neural networks. IEEE Trans Fuzzy Syst 9:578–594
Yager R (1990) A model of participatory learning. IEEE Trans Syst Man Cybern 20(5):1229–1234
Young P (1984) Recursive estimation and time-series analysis: an introduction. Springer-Verlag, New York Inc, New York
Yourdshahi ES, Angelov P, Marcolino L, Tsianakas G (2018) Towards evolving cooperative mapping for large-scale uav teams. In: Proceedings of the IEEE Symposium Series on Computational Intelligence (SSCI), pp. 2262–2269
Za’in C, Pratama M, Lughofer E, Anavatti S (2017) Evolving type-2 web news mining. Appl Soft Comput 54:200–220
Zdešar A, Dovžan D, Škrjanc I (2014) Self-tuning of 2 DOF control based on evolving fuzzy model. Appl Soft Comput 19:403–418
Škrjanc I (2009) Confidence interval of fuzzy models: an example using a waste-water treatment plant. Chemom Intell Lab Syst 96:182–187
Škrjanc I (2015) Evolving fuzzy-model-based design of experiments with supervised hierarchical clustering. IEEE Trans Fuzzy Syst 23(4):861–871
Škrjanc I, Andonovski G, Ledezma A, Sipele O, Iglesias JA, Sanchis A (2018) Evolving cloud-based system for the recognition of drivers’ actions. Expert Syst Appl 99:231–238
Škrjanc I, Blažič S, Lughofer E, Dovžan D (2019) Inner matrix norms in evolving Cauchy possibilistic clustering for classification and regression from data streams. Inf Sci 478:540–563
Škrjanc I, Dovžan D (2015) Evolving Gustafson-Kessel possibilistic c-means clustering. Procedia Comput Sci 53:191–198
Škrjanc I, Iglesias JA, Sanchis A, Leite D, Lughofer E, Gomide F (2019) Evolving fuzzy and neuro-fuzzy approaches in clustering, regression, identification, and classification: a survey. Inf Sci 490:344–368
Škrjanc I, Ozawa S, Ban T, Dovžan D (2018) Large-scale cyber attacks monitoring using evolving cauchy possibilistic clustering. Appl Soft Comput 62:592–601
Škrjanc I (2019) Cluster-volume-based merging approach for incrementally evolving fuzzy Gaussian clustering - eGAUSS+. IEEE Trans Fuzzy Syst pp. 1–11. https://doi.org/10.1109/TFUZZ.2019.2931874
Funding
This work was supported by Instituto Serrapilheira (Grant No. Serra-1812-26777), Javna Agencija za Raziskovalno Dejavnost RS (Grant No. P2-0219) and Conselho Nacional de Desenvolvimento Científico e Tecnológico (Grant No. 305906/2014-3).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Leite, D., Škrjanc, I. & Gomide, F. An overview on evolving systems and learning from stream data. Evolving Systems 11, 181–198 (2020). https://doi.org/10.1007/s12530-020-09334-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12530-020-09334-5