Abstract
In this paper, we propose a new methodology for learning evolving fuzzy systems (EFS) from data streams in terms of on-line regression/system identification problems. It comes with enhanced dynamic complexity reduction steps, acting on model components and on the input structure and by employing generalized fuzzy rules in arbitrarily rotated position. It is thus termed as Gen-Smart-EFS (GS-EFS), short for generalized smart evolving fuzzy systems. Equipped with a new projection concept for high-dimensional kernels onto one-dimensional fuzzy sets, our approach is able to provide equivalent conventional TS fuzzy systems with axis-parallel rules, thus maintaining interpretability when inferring new query samples. The on-line complexity reduction on rule level integrates a new merging concept based on a combined adjacency–homogeneity relation between two clusters (rules). On input structure level, complexity reduction is motivated by a combined statistical-geometric concept and acts in a smooth and soft manner by incrementally adapting feature weights: features may get smoothly out-weighted over time (\(\rightarrow\) soft on-line dimension reduction) but also may become reactivated at a later stage. Out-weighted features will contribute little to the rule evolution criterion, which prevents the generation of unnecessary rules and reduces over-fitting due to curse of dimensionality. The criterion relies on a newly developed re-scaled Mahalanobis distance measure for assuring monotonicity between feature weights and distance values. Gen-Smart-EFS will be evaluated based on high-dimensional real-world data (streaming) sets and compared with other well-known (evolving) fuzzy systems approaches. The results show improved accuracy with lower rule base complexity as well as smaller rule length when using Gen-Smart-EFS.
Similar content being viewed by others
References
Abonyi J, Babuska R, Szeifert F (2002) Modified Gath-Geva fuzzy clustering for identification of Takagi–Sugeno fuzzy models. IEEE Trans Syst Man Cybern Part B 32(5):612–621
Angelov P (2010) Evolving Takagi–Sugeno fuzzy systems from streaming data, eTS+. In: Angelov P, Filev D, Kasabov N (eds) Evolving intelligent systems: methodology and applications. Wiley, New York, pp 21–50
Angelov P, Filev D (2004) An approach to online identification of Takagi-Sugeno fuzzy models. IEEE Trans Syst Man Cybern Part B Cybern 34(1):484–498
Angelov P, Filev D, Kasabov N (2010) Evolving intelligent systems—methodology and applications. Wiley, New York
Angelov P, Kordon A (2010) Evolving inferential sensors in the chemical process industry. In: Angelov P, Filev D, Kasabov N (eds) Evolving intelligent systems—methodology and applications. Wiley, New York, pp 313–336
Angelov P, Lughofer E, Zhou X (2008) Evolving fuzzy classifiers using different model architectures. Fuzzy Sets Syst 159(23):3160–3182
Angelov P, Sadeghi-Tehran P, Ramezani R (2011) An approach to automatic real-time novelty detection, object identification, and tracking in video streams based on recursive density estimation and evolving Takagi–Sugeno fuzzy systems. Int J Intell Syst 26(3):189–205
Angelov P, Zhou XW (2006) Evolving fuzzy systems from data streams in real-time. In: 2006 international symposium on evolving fuzzy systems (EFS’06), Ambleside, pp 29–35 (2006)
Babuska R (1998) Fuzzy modeling for control. Kluwer Academic Publishers, Norwell
Backer SD, Scheunders P (2001) Texture segmentation by frequency-sensitive elliptical competitive learning. Image Vis Comput 19(9–10):639–648
Bauer F, Lukas M (2011) Comparing parameter choice methods for regularization of ill-posed problems. Math Comput Simul 81(9):1795–1841
Bhattacharyya A (1943) On a measure of divergence between two statistical populations defined by their probability distributions. Bull Calcutta Math Soc 35:99–109
Bifet A, Kirkby R (2011) Data stream mining—a practical approach. Tech. rep., Department of Computer Sciences, University of Waikato, Japan
Bouchachia A, Mittermeir R (2006) Towards incremental fuzzy classifiers. Soft Comput 11(2):193–207
Casillas J, Cordon O, Herrera F, Magdalena L (2003) Interpretability issues in fuzzy modeling. Springer, Berlin
Castro J, Delgado M (1996) Fuzzy systems with defuzzification are universal approximators. IEEE Trans Syst Man Cybern Part B Cybern 26(1):149–152
Chen H, Tino P, Yao X, Rodan A (2014) Learning in the model space for fault diagnosis. IEEE Trans Neural Netw Learn Syst 25(1):124–136
Cheng W, Juang C (2011) An incremental support vector machine-trained TS-type fuzzy system for online classification problems. Fuzzy Sets Syst 163(1):24–44
Cohen L, Avrahami-Bakish G, Last M, Kandel A, Kipersztok O (2008) Real-time data mining of non-stationary data streams from sensor networks. Inf Fusion 9(3):344–353
Costa B, Angelov P, Guedes L (2015) Fully unsupervised fault detection and identification based on recursive density estimation and self-evolving cloud-based classifier. Neurocomputing 150(A):289–303
Diehl C, Cauwenberghs G (2003) SVM incremental learning, adaptation and optimization. In: Proceedings of the international joint conference on neural networks, vol 4, pp 2685–2690, Boston (2003)
Djouadi A, Snorrason O, Garber F (1990) The quality of training-sample estimates of the Bhattacharyya coefficient. IEEE Trans Pattern Anal Mach Intell 12(1):92–97
Efendic H, Re LD (2006) Automatic iterative fault diagnosis approach for complex systems. WSEAS Trans Syst 5(2):360–367
Eitzinger C, Heidl W, Lughofer E, Raiser S, Smith J, Tahir M, Sannen D, van Brussel H (2010) Assessment of the influence of adaptive components in trainable surface inspection systems. Mach Vis Appl 21(5):613–626
Gama J (2010) Knowledge discovery from data streams. Chapman & Hall/CRC, Boca Raton
Gray R (1984) Vector quantization. IEEE ASSP Mag 1(2):4–29
Hametner C, Jakubek S (2013) Local model network identification for online engine modelling. Inf Sci 220:210–225
Hastie T, Tibshirani R, Friedman J (2009) The elements of statistical learning: data mining, inference and prediction, 2nd edn. Springer, New York
Heidl W, Thumfart S, Lughofer E, Eitzinger C, Klement E (2013) Machine learning based analysis of gender differences in visual inspection decision making. Inf Sci 224:62–76
Hill T, Lewicki P (2007) Statistics: methods and applications. StatSoft, Tulsa
Holm S (1979) A simple sequentially rejective multiple test procedure. Scand J Stat 6:65–70
Huang G, Saratchandran P, Sundararajan N (2004) An efficient sequential learning algorithm for growing and pruning RBF (GAP-RBF) networks. IEEE Trans Syst Man Cybern Part B Cybern 34(6):2284–2292
Iglesias J, Angelov P, Ledezma A, Sanchis A (2010) Evolving classification of agent’s behaviors: a general approach. Evol Syst 1(3):161–172
Jang JS (1993) ANFIS: adaptive-network-based fuzzy inference systems. IEEE Trans Syst Man Cybern 23(3):665–685
Jimenez L, Landgrebe D (1998) Supervised classification in high-dimensional space: geometrical, statistical, and asymptotical properties of multivariate data. IEEE Trans Syst Man Cybern Part C Rev Appl 28(1):39–54
Jin Y (2000) Fuzzy modelling of high dimensional systems: complexity reduction and interpretability improvement. IEEE Trans Fuzzy Syst 8(2):212–221
Klement E, Mesiar R, Pap E (2000) Triangular norms. Kluwer Academic Publishers, Dordrecht
Kohonen T (1995) Self-organizing maps, 2nd edn. Springer, Berlin
Komijani M, Lucas C, Araabi B, Kalhor A (2012) Introducing evolving Takagi–Sugeno method based on local least squares support vector machine models. Evol Syst 3(2):81–93
Krishnamoorthy K, Mathew T (2009) Statistical tolerance regions: theory, applications, and computation. Wiley, Hoboken
Leite D, Ballini R, Costa P, Gomide F (2012) Evolving fuzzy granular modeling from nonstationary fuzzy data streams. Evol Syst 3(2):65–79
Leite D, Costa P, Gomide F (2012) Interval approach for evolving granular system modeling. In: Sayed-Mouchaweh M, Lughofer E (eds) Learning in non-stationary environments: methods and applications. Springer, New York, pp 271–300
Lemos A, Caminhas W, Gomide F (2011) Multivariable Gaussian evolving fuzzy modeling system. IEEE Trans Fuzzy Syst 19(1):91–104
Lemos A, Caminhas W, Gomide F (2013) Adaptive fault detection and diagnosis using an evolving fuzzy classifier. Inf Sci 220:64–85
Liang N, Huang G, Saratchandran P, Sundararajan N (2006) A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans Neural Netw 17:1411–1423
Lima E, Hell M, Ballini R, Gomide F (2010) Evolving fuzzy modeling using participatory learning. In: Angelov P, Filev D, Kasabov N (eds) Evolving intelligent systems: methodology and applications. Wiley, New York, pp 67–86
Lin Y, Chang JY, Lin CT (2013) Identification and prediction of dynamic systems using an interactively recurrent self-evolving fuzzy neural network. IEEE Trans Neural Netw Learn Syst 24(2):310–321
Lippmann R (1991) A critical overview of neural network pattern classifiers. In: Proceedings of the IEEE workshop neural networks and signal processing, pp 266–275
Lughofer E (2008) Extensions of vector quantization for incremental clustering. Pattern Recognit 41(3):995–1011
Lughofer E (2008) FLEXFIS: a robust incremental learning approach for evolving TS fuzzy models. IEEE Trans Fuzzy Syst 16(6):1393–1410
Lughofer E (2011) Evolving fuzzy systems—methodologies, advanced concepts and applications. Springer, Berlin
Lughofer E (2011) On-line incremental feature weighting in evolving fuzzy classifiers. Fuzzy Sets Syst 163(1):1–23
Lughofer E (2013) On-line assurance of interpretability criteria in evolving fuzzy systems—achievements, new concepts and open issues. Inf Sci 251:22–46
Lughofer E, Angelov P (2011) Handling drifts and shifts in on-line data streams with evolving fuzzy systems. Appl Soft Comput 11(2):2057–2068
Lughofer E, Bouchot JL, Shaker A (2011) On-line elimination of local redundancies in evolving fuzzy systems. Evol Syst 2(3):165–187
Lughofer E, Kindermann S (2010) SparseFIS: data-driven learning of fuzzy systems with sparsity constraints. IEEE Trans Fuzzy Syst 18(2):396–411
Lughofer E, Smith JE, Caleb-Solly P, Tahir M, Eitzinger C, Sannen D, Nuttin M (2009) Human–machine interaction issues in quality control based on on-line image classification. IEEE Trans Syst Man Cybern Part A Syst Hum 39(5):960–971
Lughofer E, Trawinski B, Trawinski K, Kempa O, Lasota T (2011) On employing fuzzy modeling algorithms for the valuation of residential premises. Inf Sci 181(23):5123–5142
Macias-Hernandez J, Angelov P (2010) Applications of evolving intelligent systems to the oil and gas industry. In: Angelov P, Filev D, Kasabov N (eds) Evolving intelligent systems: methodology and applications. Wiley, New York, pp 401–421
Maciel L, Lemos A, Gomide F, Ballini R (2012) Evolving fuzzy systems for pricing fixed income options. Evol Syst 3(1):5–18
Mahalanobis PC (1936) On the generalised distance in statistics. Proc Natl Inst Sci India 2(1):49–55
Pang S, Ozawa S, Kasabov N (2005) Incremental linear discriminant analysis for classification of data streams. IEEE Trans Syst Men Cybern Part B Cybern 35(5):905–914
Pedrycz W, Gomide F (2007) Fuzzy systems engineering: toward human-centric computing. Wiley, Hoboken
Pratama M, Anavatti S, Lughofer E (2014) GENEFIS: towards an effective localist network. IEEE Trans Fuzzy Syst 22(3):547–562
Qin S, Li W, Yue H (2000) Recursive PCA for adaptive process monitoring. J Process Control 10(5):471–486
Rao R (2013) A novel weighted euclidean distance-based approach. In: Decision making in manufacturing environment using graph theory and fuzzy multiple attribute decision making methods. Springer Series in Manufacturing, New York, pp 159–191
Rong HJ, Sundararajan N, Huang GB, Saratchandran P (2006) Sequential adaptive fuzzy inference system (SAFIS) for nonlinear system identification and prediction. Fuzzy Sets Syst 157(9):1260–1275
Rubio J (2010) Stability analysis for an on-line evolving neuro-fuzzy recurrent network. In: Angelov P, Filev D, Kasabov N (eds) Evolving intelligent systems: methodology and applications. Wiley, New York, pp 173–199
Sannen D, Nuttin M, Smith J, Tahir M, Lughofer E, Eitzinger C (2008) An interactive self-adaptive on-line image classification framework. In: Gasteratos A, Vincze M, Tsotsos J (eds) Proceedings of ICVS 2008, LNCS, vol 5008. Springer, Santorini Island, pp 173–180
Sayed-Mouchaweh M, Lughofer E (2012) Learning in non-stationary environments: methods and applications. Springer, New York
Shilton A, Palaniswami M, Ralph D, Tsoi A (2005) Incremental training of support vector machines. IEEE Trans Neural Netw 16(1):114–131
Soleimani H, Lucas K, Araabi B (2010) Recursive gathgeva clustering as a basis for evolving neuro-fuzzy modeling. Evol Syst 1(1):59–71
Sun H, Wang S (2011) Measuring the component overlapping in the Gaussian mixture model. Data Min Knowl Discov 23:479–502
Tabata K, Kudo MSM (2010) Data compression by volume prototypes for streaming data. Pattern Recognit 43(9):3162–3176
Takagi T, Sugeno M (1985) Fuzzy identification of systems and its applications to modeling and control. IEEE Trans Syst Man Cybern 15(1):116–132
Tung S, Quek C, Guan C (2013) eT2FIS: an evolving type-2 neural fuzzy inference system. Inf Sci 220:124–148
Wang N, Er M, Meng X (2009) A fast and accurate online self-organizing scheme for parsimonious fuzzy neural networks. Neurocomputing 72(16–18):3818–3829
Yager R, Filev D (1994) Approximate clustering via the mountain method. IEEE Trans Syst Man Cybern 24(8):1279–1284
Yager RR (1990) A model of participatory learning. IEEE Trans Syst Man Cybern 20(5):1229–1234
Zdsar A, Dovzan D, Skrjanc I (2014) Self-tuning of 2 DOF control based on evolving fuzzy model. Appl Soft Comput 19:403–418
Acknowledgments
The first author acknowledges the support of the Austrian COMET-K2 programme of the Linz Center of Mechatronics (LCM), funded by the Austrian federal government and the federal state of Upper Austria. This publication reflects only the authors’ views.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Lughofer, E., Cernuda, C., Kindermann, S. et al. Generalized smart evolving fuzzy systems. Evolving Systems 6, 269–292 (2015). https://doi.org/10.1007/s12530-015-9132-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12530-015-9132-6