skip to main content
10.1145/3205455.3205642acmconferencesArticle/Chapter ViewAbstractPublication PagesgeccoConference Proceedingsconference-collections
research-article

Evolving bagging ensembles using a spatially-structured niching method

Published: 02 July 2018 Publication History

Abstract

This paper presents a novel approach to constructing ensembles for prediction using a bootstrap aggregation (bagging) model. The proposed method uses analogies from ecological modelling to view bootstrap samples as a local adaptation resource in a spatially structured population. Through local competition and breeding, adaptation towards specific bootstrap samples takes place and the resulting ensemble emerges from a single global population in a single run. This makes better use of available computational resources, and negates the need for multiple runs typically required by a bagging approach. We examine the robustness of the method with respect to the number of bootstrap samples in the ensemble, and demonstrate that the resulting method also has a positive effect on bloat control. Finally, the effectiveness of the method relative to existing bagging approaches such as random forests is explored and encouraging performance is demonstrated on a range of benchmark problems.

References

[1]
Alexandros Agapitos, Anthony Brabazon, and Michael ONeill. 2012. Controlling overfitting in symbolic regression based on a bias/variance error decomposition. In International Conference on Parallel Problem Solving from Nature. Springer, 438--447.
[2]
Enrique Alba and Marco Tomassini. 2002. Parallelism and evolutionary algorithms. IEEE transactions on evolutionary computation 6, 5 (2002), 443--462.
[3]
Urvesh Bhowan, Mark Johnston, Mengjie Zhang, and Xin Yao. 2013. Evolving diverse ensembles using genetic programming for classification with unbalanced data. IEEE Transactions on Evolutionary Computation 17, 3 (2013), 368--386.
[4]
Leo Breiman. 1996. Bagging Predictors. Machine learning 24, 1 (1996), 123--140.
[5]
Leo Breiman. 2001. Random Forests. Machine learning 45, 1 (2001), 5--32.
[6]
Leo Breiman, Jerome Friedman, Charles J Stone, and Richard A Olshen. 1984. Classification and regression trees. CRC press.
[7]
Mauro Castelli, Ivo Gonçalves, Luca Manzoni, and Leonardo Vanneschi. 2018. Pruning Techniques for Mixed Ensembles of Genetic Programming Models. (2018). arXiv:arXiv:1801.07668
[8]
P. J. Darwen and Xin Yao. 1997. Speciation as automatic categorical modularization. IEEE Transactions on Evolutionary Computation 1, 2 (Jul 1997), 101--108.
[9]
Grant Dick. 2010. Automatic identification of the niche radius using spatially-structured clearing methods. In Evolutionary Computation (CEC), 2010 IEEE Congress on. IEEE, 1--8.
[10]
Grant Dick. 2012. Niche allocation in spatially-structured evolutionary algorithms with gradients. In 2012 IEEE Congress on Evolutionary Computation. 1--8.
[11]
Grant Dick. 2015. Improving Geometric Semantic Genetic Programming with Safe Tree Initialisation. In European Conference on Genetic Programming. Springer International Publishing, 28--40.
[12]
Grant Dick and Peter A. Whigham. 2006. Multimodal Optimisation with Structured Populations and Local Environments. In Simulated Evolution and Learning, Tzai-Der Wang, Xiaodong Li, Shu-Heng Chen, Xufa Wang, Hussein Abbass, Hitoshi Iba, Guo-Liang Chen, and Xin Yao (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 505--512.
[13]
Grant Dick and Peter A Whigham. 2006. Spatially-structured evolutionary algorithms and sharing: Do they mix?. In Asia-Pacific Conference on Simulated Evolution and Learning. Springer, 457--464.
[14]
Jeannie Fitzgerald, R Azad, and Conor Ryan. 2013. A bootstrapping approach to reduce over-fitting in genetic programming. In Proceedings of the 15th annual conference companion on Genetic and evolutionary computation. ACM, 1113--1120.
[15]
Gianluigi Folino, Clara Pizzuti, and Giandomenico Spezzano. 2003. Ensemble techniques for parallel genetic programming based classifiers. In European Conference on Genetic Programming. Springer, 59--69.
[16]
Gianluigi Folino, Clara Pizzuti, and Giandomenico Spezzano. 2006. GP ensembles for large-scale data classification. IEEE Transactions on Evolutionary Computation 10, 5 (2006), 604--616.
[17]
Christian Gagné, Michele Sebag, Marc Schoenauer, and Marco Tomassini. 2007. Ensemble learning for free with evolutionary algorithms?. In Proceedings of the 9th annual conference on Genetic and evolutionary computation. ACM, 1782--1789.
[18]
David Harrison and Daniel L Rubinfeld. 1978. Hedonic housing prices and the demand for clean air. Journal of environmental economics and management 5, 1 (1978), 81--102.
[19]
Hitoshi Iba. 1999. Bagging, boosting, and bloating in genetic programming. In Proceedings of the 1st Annual Conference on Genetic and Evolutionary Computation-Volume 2. Morgan Kaufmann Publishers Inc., 1053--1060.
[20]
Maarten Keijzer and Vladan Babovic. 2000. Genetic Programming, Ensemble Methods and the Bias/Variance Tradeoff - Introductory Investigations. In Genetic Programming, Riccardo Poli, Wolfgang Banzhaf, William B. Langdon, Julian Miller. Peter Nordin, and Terence C. Fogarty (Eds.). Springer Berlin Heidelberg, Berlin. Heidelberg, 76--90.
[21]
Michael Kirley. 2001. MEA: A metapopulation evolutionary algorithm for multi-objective optimisation problems. In Evolutionary Computation, 2001. Proceedings of the 2001 Congress on, Vol. 2. IEEE, 949--956.
[22]
Xiaodong Li. 2010. Niching Without Niching Parameters: Particle Swarm Optimization Using a Ring Topology. IEEE Transactions on Evolutionary Computation 14, 1 (2010), 150--169.
[23]
Yong Liu, Xin Yao, and Tetsuya Higuchi. 2000. Evolutionary ensembles with negative correlation learning. IEEE Transactions on Evolutionary Computation 4, 4 (2000), 380--387.
[24]
Ernst Mayr. 1970. Populations, species and evolution; an abridgment of Animal species and evolution. Harvard University Press, xv + 453 pages.
[25]
John Park, Su Nguyen, Mengjie Zhang, and Mark Johnston. 2015. Evolving ensembles of dispatching rules using genetic programming for job shop scheduling. In European Conference on Genetic Programming. Springer, 92--104.
[26]
J Ross Quinlan. 1993. Combining instance-based and model-based learning. In Proceedings of the Tenth International Conference on Machine Learning. 236--243.
[27]
Jared Sylvester and Nitesh V Chawla. 2005. Evolutionary ensembles: Combining learning agents using genetic algorithms. In AAAI Workshop on Multiagent Learning. 46--51.
[28]
Marco Tomassini. 2005. Spatially structured evolutionary algorithms. Springer.
[29]
Kalyan Veeramachaneni, Owen Derby, Dylan Sherry, and Una-May O'Reilly. 2013. Learning regression ensembles with genetic programming at scale. In Proceedings of the 15th annual conference on Genetic and evolutionary computation. ACM, 1117--1124.
[30]
Peter A Whigham and Grant Dick. 2010. Implicitly controlling bloat in genetic programming. IEEE Transactions on Evolutionary Computation 14, 2 (2010), 173--190.
[31]
I-C Yeh. 1998. Modeling of strength of high-performance concrete using artificial neural networks. Cement and Concrete research 28, 12 (1998), 1797--1808.

Cited By

View all
  • (2024)Using Decomposed Error for Reproducing Implicit Understanding of AlgorithmsEvolutionary Computation10.1162/evco_a_0032132:1(49-68)Online publication date: 1-Mar-2024
  • (2024)SR-Forest: A Genetic Programming-Based Heterogeneous Ensemble Learning MethodIEEE Transactions on Evolutionary Computation10.1109/TEVC.2023.324317228:5(1484-1498)Online publication date: Oct-2024
  • (2024)Revisiting Bagging for Stochastic AlgorithmsAI 2024: Advances in Artificial Intelligence10.1007/978-981-96-0351-0_12(162-173)Online publication date: 20-Nov-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
GECCO '18: Proceedings of the Genetic and Evolutionary Computation Conference
July 2018
1578 pages
ISBN:9781450356183
DOI:10.1145/3205455
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 July 2018

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. bagging
  2. ensemble learning
  3. genetic programming
  4. random forests
  5. symbolic regression

Qualifiers

  • Research-article

Conference

GECCO '18
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)6
  • Downloads (Last 6 weeks)0
Reflects downloads up to 27 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Using Decomposed Error for Reproducing Implicit Understanding of AlgorithmsEvolutionary Computation10.1162/evco_a_0032132:1(49-68)Online publication date: 1-Mar-2024
  • (2024)SR-Forest: A Genetic Programming-Based Heterogeneous Ensemble Learning MethodIEEE Transactions on Evolutionary Computation10.1109/TEVC.2023.324317228:5(1484-1498)Online publication date: Oct-2024
  • (2024)Revisiting Bagging for Stochastic AlgorithmsAI 2024: Advances in Artificial Intelligence10.1007/978-981-96-0351-0_12(162-173)Online publication date: 20-Nov-2024
  • (2023)Automatic design of machine learning via evolutionary computationApplied Soft Computing10.1016/j.asoc.2023.110412143:COnline publication date: 1-Aug-2023
  • (2023)Evolutionary Ensemble LearningHandbook of Evolutionary Machine Learning10.1007/978-981-99-3814-8_8(205-243)Online publication date: 2-Nov-2023
  • (2023)MAP-Elites with Cosine-Similarity for Evolutionary Ensemble LearningGenetic Programming10.1007/978-3-031-29573-7_6(84-100)Online publication date: 12-Apr-2023
  • (2022)Standardization and Data Augmentation in Genetic ProgrammingIEEE Transactions on Evolutionary Computation10.1109/TEVC.2022.316041426:6(1596-1608)Online publication date: Dec-2022
  • (2022)An Evolutionary Forest for RegressionIEEE Transactions on Evolutionary Computation10.1109/TEVC.2021.313666726:4(735-749)Online publication date: Aug-2022
  • (2022)Towards Explainable AutoML Using Error DecompositionAI 2022: Advances in Artificial Intelligence10.1007/978-3-031-22695-3_13(177-190)Online publication date: 5-Dec-2022
  • (2022)Genetic Programming for Ensemble Learning in Face RecognitionAdvances in Swarm Intelligence10.1007/978-3-031-09726-3_19(209-218)Online publication date: 15-Jul-2022
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media