Skip to main content
SpringerLink
Log in

Evolving Fuzzy Min–Max Neural Network Based Decision Trees for Data Stream Classification

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Learning from data streams is a challenging task which demands a learning algorithm with several high quality features. In addition to space complexity and speed requirements needed for processing the huge volume of data which arrives at high speed, the learning algorithm must have a good balance between stability and plasticity. This paper presents a new approach to induce incremental decision trees on streaming data. In this approach, the internal nodes contain trainable split tests. In contrast with traditional decision trees in which a single attribute is selected as the split test, each internal node of the proposed approach contains a trainable function based on multiple attributes, which not only provides the flexibility needed in the stream context, but also improves stability. Based on this approach, we propose evolving fuzzy min–max decision tree (EFMMDT) learning algorithm in which each internal node of the decision tree contains an evolving fuzzy min–max neural network. EFMMDT splits the instance space non-linearly based on multiple attributes which results in much smaller and shallower decision trees. The extensive experiments reveal that the proposed algorithm achieves much better precision in comparison with the state-of-the-art decision tree learning algorithms on the benchmark data streams, especially in the presence of concept drift.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Bifet A, Gavaldá R (2009) Adaptive learning from evolving data streams, vol 5772., Advances in intelligent data analysis VIII, Lecture notes in computer scienceSpringer, Berlin, pp 249–260

    Google Scholar 

  2. Bifet A, Holmes G, Kirkby R, Pfahringer B (2010) Moa: massive online analysis. J Mach Learn Res 11:1601–1604

    Google Scholar 

  3. Breiman L (1996) Heuristics of instability and stabilization in model selection. Ann Stat 24(6):2350–2383

    Article  MathSciNet  MATH  Google Scholar 

  4. Davtalab R, Dezfoulian MH, Mansoorizadeh M (2014) Multi-level fuzzy min-max neural network classifier. IEEE Trans Neural Netw Learn Syst 25(3):470–482

    Article  Google Scholar 

  5. Demsar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30

    MathSciNet  MATH  Google Scholar 

  6. Domingos P, Hulten G (2000) Mining high-speed data streams. In: Proceedings of the sixth ACM SIGKDD international conference on knowledge discovery and data mining, KDD, pp 71–80. doi:10.1145/347090.347107

  7. Friedman M (1937) The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J Am Stat Assoc 32(200):675–701

    Article  MATH  Google Scholar 

  8. Friedman M (1940) A comparison of alternative tests of significance for the problem of m rankings. Ann Math Stat 11(1):86–92

    Article  MathSciNet  MATH  Google Scholar 

  9. Gabrys B, Bargiela A (2000) General fuzzy min-max neural network for clustering and classification. Trans Neural Netw 11(3):769–783

    Article  Google Scholar 

  10. Gama J, Rocha R, Medas P (2003) Accurate decision trees for mining high-speed data streams. In: Proceedings of the ninth ACM SIGKDD international conference on knowledge discovery and data mining, KDD’03, New York, USA, pp 523–528

  11. Gama J, Rodrigues P, Sebastião R (2009) Evaluating algorithms that learn from data streams. In: Proceedings of the 2009 ACM symposium on applied computing, SAC’09, pp 1496–1500

  12. Gama J (2004) Functional trees. Mach Learn 55(3):219–250

    Article  MATH  Google Scholar 

  13. Gama J, Sebastião R, Rodrigues P (2013) On evaluating stream learning algorithms. Mach Learn 90(3):317–346

    Article  MathSciNet  MATH  Google Scholar 

  14. Hashemi S, Yang Y, Mirzamomen Z, Kangavari M (2009) Adapted one-versus-all decision trees for data stream classification. IEEE Trans Knowl Data Eng 21(5):624–637

    Article  Google Scholar 

  15. Hashemi S, Yang Y (2009) Flexible decision tree for data stream classification in the presence of concept change, noise and missing values. Data Mining Knowl Discov 19:95–131

    Article  MathSciNet  Google Scholar 

  16. Hulten G, Spencer L, Domingos P (2001) Mining time-changing data streams. In: Proceedings of the 2001 ACM SIGKDD international conference on knowledge discovery and data mining, pp 97–106

  17. Ikonomovska E, Gama J, Dz̆eroski S (2011) Learning model trees from evolving data streams. Data Mining Knowl Discov 23(1):128–168

    Article  MathSciNet  MATH  Google Scholar 

  18. Kirkby R (2007) Improving hoeffding trees. Dissertation, University of Waikato

  19. Kohavi R (1996) Scaling up the accuracy of naive-bayes classifiers: a decision-tree hybrid. In: Proceedings of the second international conference on knowledge discovery and data mining, AAAI Press, pp 202–207

  20. Last M, Maimon O, Minkov E (2002) Improving stability of decision trees. Int J Pattern Recogn Artif Intell 16(02):145–159

    Article  Google Scholar 

  21. Nandedkar AV, Biswas PK (2007) A fuzzy min-max neural network classifier with compensatory neuron architecture. IEEE Trans Neural Netw 18(1):42–54

    Article  Google Scholar 

  22. Blake C, Keogh E, Merz C (1998) UCI repository of machine learning databases. http://archive.ics.uci.edu/ml. Accessed 20 Sep 2014

  23. Pfahringer B, Holmes G, Kirkby R (2007) New options for hoeffding trees. In: Orgun MA, Thornton J (eds) AI 2007: advances in artificial intelligence, lecture notes in computer science, vol 4830, Springer, Berlin, pp 90–99

  24. Simpson PK (1992) Fuzzy min-max neural networks, i, classification. IEEE Trans Neural Netw 3(5):776–786

    Article  Google Scholar 

  25. Street WN, Kim Y (2001) A streaming ensemble algorithm (sea) for large-scale classification. In: Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining, KDD’01, pp 377–382

  26. Wang X, Fan W, Yu PS, Han J (2003) Mining concept-drifting data streams using ensemble classifiers. In: Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining, KDD’03, pp 226–235

  27. Yang H, Fong S (2012) Incrementally optimized decision tree for noisy big data. In: Proceedings of the 1st international workshop on big data, streams and heterogeneous source mining: algorithms, systems, programming models and applications, BigMine’12, pp 36–44

  28. Yang Y, Wu X, Zhu X. (2005) Combining proactive and reactive predictions for data streams. In: Grossman R, Bayardo R, Bennett KP (eds), KDD, pp 710–715

  29. Zhang H, Liu J, Ma D, Wang Z (2011) Data-core-based fuzzy min-max neural network for pattern classification. IEEE Trans Neural Netw 22(12):2339–2352

    Article  Google Scholar 

  30. Zhao Q (2005) Learning with data streams: an nntree based approach. In: Enokido T, Yan L, Xiao B, Kim D, Dai Y, Yang L (eds) Embedded and ubiquitous computing EUC 2005 workshops, vol 3823., Lecture notes in computer scienceSpringer, Berlin, pp 519–528

    Chapter  Google Scholar 

  31. Zimmermann A (2008) Ensemble-trees: leveraging ensemble power inside decision trees. Discovery science, vol 5255., Lecture notes in computer scienceSpringer, Berlin, pp 76–87

    Chapter  Google Scholar 

  32. Mirzamomen Z, Kangavari M (2016) Fuzzy min-max neural network based decision trees. Intell Data Anal 20(4)

  33. Mohammed MF, Lim CP (2015) An enhanced fuzzy min-max neural network for pattern classification. IEEE Trans Neural Netw Learn Syst 26(3):417–429

    Article  MathSciNet  Google Scholar 

  34. Shinde SV, Kulkarni UV (2016) Extracting classification rules from modified fuzzy min max neural network for data with mixed attributes. Appl Soft Comput 40:364–378

    Article  Google Scholar 

  35. Shinde SV, Kulkarni UV, Chaudhary AN (2015) Extracting the classification rules from general fuzzy min-max neural network. Int J Comput Appl 121(23):1–7

  36. Kulkarni SU, Shetty BS (2015) Data mining using modified GFMM neural network. Int J Comput Appl 116(15):18–22

  37. Forghani Y, Yazdi HS (2015) Fuzzy min-max neural network for learning a classifier with symmetric margin. Neural Process Lett 42(2):317–353

    Article  Google Scholar 

  38. Seera M, Lim CP, Loo CK, Jain LC (2015) Data clustering using a modified fuzzy min-max neural network, soft computing applications. In: Proceedings of the 6th international workshop soft computing applications (SOFA 2014), Vol 1, Springer, pp 413–422

  39. Quteishat A, Lim CP (2008) A modified fuzzy min-max neural network with rule extraction and its application to fault detection and classification. Appl Soft Comput 8(2):985–995

    Article  Google Scholar 

  40. Ma D, Liu J, Wang Z (2012) The pattern classification based on fuzzy min-max neural network with new algorithm. In: Wang J, Yen GG, Polycarpou MM (eds) Proceedings of the 9th international symposium on advances in neural networks, Springer, Berlin, pp 1–9

  41. Rizzi A, Panella M, Massimo F, Mascioli F (2002) Adaptive resolution min-max classifiers. IEEE Trans Neural Netw 13(2):402–414

    Article  Google Scholar 

  42. Leite D, Costa P, Gomide F (2013) Evolving granular neural networks from fuzzy data streams. Neural Netw 38:1–16

    Article  MATH  Google Scholar 

  43. Pfahringer B, Holmes G, Kirkby R (2007) New options for hoeffding trees. In: Orgun MA, Thornton J (eds) Proceedings of the 20th Australian joint conference on advances in artificial intelligence, AI07, Springer, Berlin, pp 90–99

Download references

Author information

Authors and Affiliations

  1. School of Computer Engineering, Iran University of Science and Technology, Tehran, Iran

    Zahra Mirzamomen & Mohammad Reza Kangavari

Authors
  1. Zahra Mirzamomen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammad Reza Kangavari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zahra Mirzamomen.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mirzamomen, Z., Kangavari, M.R. Evolving Fuzzy Min–Max Neural Network Based Decision Trees for Data Stream Classification. Neural Process Lett 45, 341–363 (2017). https://doi.org/10.1007/s11063-016-9528-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-016-9528-8

Keywords

Search

Navigation