Abstract
The aim of this work is to examine the possibility of using the output of base learners as antecedents for fuzzy rule-based hybrid ensembles. We select a flexible, grammar-driven framework for generating ensembles that combines multilayer perceptrons and support vector machines by means of genetic programming. We assess the proposed model in three real-world regression problems and we test it against multi-level, hierarchical ensembles. Our first results show that for a given large size of the base learners pool, the outputs of some of them can be useful in the antecedent parts to produce accurate ensembles, while at the same time other more accurate members of the same pool contribute in the consequent part.
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Alba, E., Cotta, C., Troya, J.: Evolutionary design of fuzzy logic controllers using strongly-typed GP. In: Proc. 1996 IEEE Int’l Symposium on Intelligent Control, New York, NY (1996)
Brown, G., Wyatt, J., Harris, R., Yao, X.: Diversity creation methods: a survey and categorisation. Inf. Fusion 6(1), 5–20 (2005)
Chandra, A., Yao, X.: Divace: Diverse and accurate ensemble learning algorithm. In: Yang, Z.R., Yin, H., Everson, R.M. (eds.) IDEAL 2004. LNCS, vol. 3177, pp. 619–625. Springer, Heidelberg (2004)
Chandra, A., Yao, X.: Evolving hybrid ensembles of learning machines for better generalisation. Neurocomputing 69, 686–700 (2006)
Clemen, R.: Combining forecasts: A review and annotated bibliography. International Journal of Forecasting 5(4), 559–583 (1989)
Coelho, A., Fernandes, E., Faceli, K.: Multi-objective design of hierarchical consensus functions for clustering ensembles via genetic programming. Decision Support Systems 51(4), 794–809 (2011)
Dietterich, T.G.: Ensemble methods in machine learning. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857, pp. 1–15. Springer, Heidelberg (2000)
Duin, R.: The combining classifier: to train or not to train? In: Proc. of the 16th Int’l Conf. on Pattern Recognition, pp. 765–770 (2002)
Ein-Dor, P., Feldmesser, J.: Attributes of the performance of central processing units: A relative performance prediction model. Commun. ACM 30(30), 308–317 (1984)
Evangelista, P., Bonissone, P., Embrechts, M., Szymanski, B.: Unsupervised fuzzy ensembles and their use in intrusion detection. In: European Symposium on Artificial Neural Networks (ESANN 2005), Bruges, Belgium (2005)
Fernandez, F., Tommassini, M., Vanneschi, L.: An empirical study of multipopulation genetic programming. Genetic Programming and Evolvable Machines 4(1) (2003)
Folino, G., Pizzuti, C., Spezzano, G.: Ensemble techniques for parallel genetic programming based classifiers. In: Ryan, C., Soule, T., Keijzer, M., Tsang, E.P.K., Poli, R., Costa, E. (eds.) EuroGP 2003. LNCS, vol. 2610, pp. 59–69. Springer, Heidelberg (2003)
Frank, A., Asuncion, A.: UCI Machine Learning Repository. University of California, School of Information and Computer Science, Irvine, CA (2010), http://archive.ics.uci.edu/ml
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.: The weka data mining software: An update. SIGKDD Explorations 11(1) (2009)
Hong, J., Cho, S.: The classification of cancer based on dna microarray data that uses diverse ensemble genetic programming. Artif. Intell. in Med. 36(1), 43–58 (2006)
Ishibuchi, H.: Multiobjective genetic fuzzy systems: Review and future research directions. In: IEEE Int’l Conf. on Fuzzy Systems (FUZZ-IEEE 2007), pp. 59–69. Imperial College (2007)
Jacobs, R.: Bias-variance analyses of mixture-of-experts architectures. Neural Computation, 369–383 (1997)
Jensen, R., Shen, Q.: New approaches to fuzzy-rough feature selection. IEEE Trans. on Fuzzy Systems 17(4), 824–838 (2009)
Kadlec, P., Gabrys, B.: Local learning-based adaptive soft sensor for catalyst activation prediction. AIChE Journal 57(5), 1288–1301 (2011)
Koza, J.: Genetic programming - On the programming of computers by means of natural selection. The MIT Press, Cambridge (1992)
Kuncheva, L.: Fuzzy versus nonfuzzy in combining classifiers designed by boosting. IEEE Trans. on Fuzzy Systems 11(6), 729–741 (2003)
Kuncheva, L.: Combining pattern classifiers: methods and algorithms. John Wiley and Sons, New York (2004)
Liaw, A., Wiener, M.: Classification and regression by randomforest. Expert Systems with Applications (2002) (under review)
Medina-Chico, V., Suárez, A., Lutsko, J.F.: Backpropagation in decision trees for regression. In: Flach, P.A., De Raedt, L. (eds.) ECML 2001. LNCS (LNAI), vol. 2167, pp. 348–359. Springer, Heidelberg (2001)
Quinlan, J.R.: Learning with continuous classes. In: AI 1992. World Scientific, Singapore (1992)
Ruta, D., Gabrys, B.: An overview of classifier fusion methods. Computing and Information Systems 7(2), 1–10 (2000)
Scholkopf, B., Smola, A.: Learning with Kernels - Support Vector Machines, Regularization, Optimization and Beyond. The MIT Press, Cambridge (2002)
Sharkey, A., Sharkey, N., Gerecke, U., Chandroth, G.: The test and select approach to ensemble combination. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857, pp. 30–44. Springer, Heidelberg (2000)
Tsakonas, A.: A comparison of classification accuracy of four genetic programming evolved intelligent structures. Information Sciences 17(1), 691–724 (2006)
Tsakonas, A., Gabrys, B.: Evolving takagi-sugeno-kang fuzzy systems using multi-population grammar guided genetic programmings. In: Int’l Conf. Evol. Comp. Theory and Appl. (ECTA 2011), Paris, France (2011)
Tsakonas, A., Gabrys, B.: Gradient: Grammar-driven genetic programming framework for building multi-component, hierarchical predictive systems. Expert Systems with Applications 39(18), 13253–13266 (2012)
Tsakonas, A., Gabrys, B.: A fuzzy evolutionary framework for combining ensembles. Applied Soft Computing (2013), doi:dx.doi.org/10.1016/j.asoc.2012.12.027
Yeh, I.-C.: Modeling slump of concrete with fly ash and superplasticizer. Computers and Concrete 5(6), 559–572 (2008)
Zhang, Y., Bhattacharyya, S.: Genetic programming in classifying large-scale data: an ensemble method. Information Sciences 163(1), 85–101 (2004)
Zhou, Z., Wu, J., Jiang, Y., Chen, R.: Genetic algorithm based selective neural network ensemble. In: 17th Int’l Joint Conf. Artif. Intell., pp. 797–802. Morgan Kaufmann, USA (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Tsakonas, A., Gabrys, B. (2015). Application of Base Learners as Conditional Input for Fuzzy Rule-Based Combined System. In: Madani, K., Correia, A., Rosa, A., Filipe, J. (eds) Computational Intelligence. IJCCI 2012. Studies in Computational Intelligence, vol 577. Springer, Cham. https://doi.org/10.1007/978-3-319-11271-8_2
Download citation
DOI: https://doi.org/10.1007/978-3-319-11271-8_2
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-11270-1
Online ISBN: 978-3-319-11271-8
eBook Packages: EngineeringEngineering (R0)