Skip to main content
Springer Nature Link
Log in

Feature Encapsulation by Stages in the Regression Domain Using Grammatical Evolution

  • Conference paper
  • First Online:
Parallel Problem Solving from Nature – PPSN XVIII (PPSN 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 15149))

Included in the following conference series:

  • 302 Accesses

Abstract

Feature Encapsulation by Stages (FES) is a recently proposed mechanism that can be implemented in any Evolutionary Computation (EC) metaheuristic. Encapsulation occurs via input space expansion in several stages by adding the best individual so far as an additional input. FES has been shown to perform well in training Boolean problems. This paper extends FES to the regression domain. Grammatical Evolution (GE), a branch of Genetic Programming (GP), supports the implementation of the FES approach by enabling the investigation of performance across various search guides expressed in the grammar. We conduct experiments on both synthetic and real-world symbolic regression problems, including multi-target issues. Additionally, we study several FES-based approaches utilising the best selection process for each problem, choosing between tournament, \(\epsilon \)-Lexicase, and \(\epsilon \hbox {-}\textrm{Lexi}^2\). Statistical tests on unseen subsets’ results show that FES outperforms the standard baseline in all problems. Furthermore, we analyse individual complexity across generations, showing that populations utilising FES consist of simpler individuals, thereby reducing computational costs.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Aha, D.: UCI Machine Learning Repository (1987). https://archive.ics.uci.edu/ml/datasets.php

  2. Batista, J.E., Silva, S.: Comparative study of classifier performance using automatic feature construction by M3GP. In: 2022 IEEE Congress on Evolutionary Computation (CEC), Padua, Italy, pp. 1–8. IEEE (2022). https://doi.org/10.1109/CEC55065.2022.9870343

  3. Godbole, S., Sarawagi, S.: Discriminative methods for multi-labeled classification. In: Dai, H., Srikant, R., Zhang, C. (eds.) PAKDD 2004. LNCS (LNAI), vol. 3056, pp. 22–30. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24775-3_5

    Chapter  Google Scholar 

  4. Goldberg, D.E., Deb, K.: A comparative analysis of selection schemes used in genetic algorithms. In: RAWLINS, G.J. (ed.) Foundations of Genetic Algorithms, Foundations of Genetic Algorithms, San Francisco, vol. 1, pp. 69–93. Elsevier (1991). https://doi.org/10.1016/B978-0-08-050684-5.50008-2. https://www.sciencedirect.com/science/article/pii/B9780080506845500082

  5. Gupt, K.K., Kshirsagar, M., Dias, D.M., Sullivan, J.P., Ryan, C.: A novel ml-driven test case selection approach for enhancing the performance of grammatical evolution (2023)

    Google Scholar 

  6. Herbold, S.: Autorank: a python package for automated ranking of classifiers. J. Open Source Softw. 5(48), 2173 (2020). https://doi.org/10.21105/joss.02173

  7. Keijzer, M., Ryan, C., Cattolico, M.: Run transferable libraries — learning functional bias in problem domains. In: Deb, K. (ed.) GECCO 2004. LNCS, vol. 3103, pp. 531–542. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-24855-2_63

    Chapter  Google Scholar 

  8. Koza, J.R.: Genetic Programming, 1st edn. MIT Press, Cambridge (1992)

    Google Scholar 

  9. Koza, J.R., Bennett, F.H., Andre, D., Keane, M.A.: Reuse, parameterized reuse, and hierarchical reuse of substructures in evolving electrical circuits using genetic programming. In: Higuchi, T., Iwata, M., Liu, W. (eds.) ICES 1996. LNCS, vol. 1259, pp. 312–326. Springer, Heidelberg (1997). https://doi.org/10.1007/3-540-63173-9_56

    Chapter  Google Scholar 

  10. La Cava, W., Spector, L., Danai, K.: Epsilon-lexicase selection for regression. In: Proceedings of the Genetic and Evolutionary Computation Conference 2016, GECCO 2016, pp. 741–748. Association for Computing Machinery, New York (2016). https://doi.org/10.1145/2908812.2908898

  11. de Lima, A., Carvalho, S., Dias, D.M., Naredo, E., Sullivan, J.P., Ryan, C.: Grape: grammatical algorithms in python for evolution. Signals 3(3), 642–663 (2022). https://doi.org/10.3390/signals3030039

    Article  Google Scholar 

  12. de Lima, A., Carvalho, S., Dias, D.M., Naredo, E., Sullivan, J.P., Ryan, C.: \({\rm Lexi}^2\): lexicase selection with lexicographic parsimony pressure. In: Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2022, pp. 929–937. Association for Computing Machinery, New York (2022). https://doi.org/10.1145/3512290.3528803

  13. Liu, H., Cai, J., Ong, Y.S.: Remarks on multi-output gaussian process regression. Knowl.-Based Syst. 144, 102–121 (2018). https://doi.org/10.1016/j.knosys.2017.12.034. https://www.sciencedirect.com/science/article/pii/S0950705117306123

  14. Mastelini, S.M., Santana, E.J., Cerri, R., Barbon, S.: Dstars: a multi-target deep structure for tracking asynchronous regressor stacking. Appl. Soft Comput. 91, 106215 (2020). https://doi.org/10.1016/j.asoc.2020.106215. https://www.sciencedirect.com/science/article/pii/S1568494620301551

  15. McDermott, J., et al.: Genetic programming needs better benchmarks. In: Proceedings of the 14th Annual Conference on Genetic and Evolutionary Computation, GECCO 2012, pp. 791–798. Association for Computing Machinery, New York (2012). https://doi.org/10.1145/2330163.2330273

  16. Medernach, D., Fitzgerald, J., Azad, R.M.A., Ryan, C.: Wave: a genetic programming approach to divide and conquer. In: Proceedings of the Companion Publication of the 2015 Annual Conference on Genetic and Evolutionary Computation, GECCO Companion 2015, pp. 1435–1436. Association for Computing Machinery, New York (2015). https://doi.org/10.1145/2739482.2764659

  17. Murphy, A., Ryan, C.: Improving module identification and use in grammatical evolution. In: 2020 IEEE Congress on Evolutionary Computation (CEC), Glasgow, UK, pp. 1–7. IEEE (2020). https://doi.org/10.1109/CEC48606.2020.9185571

  18. Reyes, D.: BDS Group repository (2023). https://github.com/bdsul/fes

  19. Reyes, D., de Lima, A., Murphy, A., Dias, D.M., Ryan, C.: Feature encapsulation by stages using grammatical evolution. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion. GECCO 2024 (2024). https://doi.org/10.1145/3638530.3654097

  20. Ryan, C., Azad, R.M.A.: Sensible initialisation in grammatical evolution. In: Barry, A.M. (ed.) GECCO 2003: Proceedings of the Bird of a Feather Workshops, Genetic and Evolutionary Computation Conference, Chigaco, pp. 142–145. AAAI (2003)

    Google Scholar 

  21. Ryan, C., Collins, J.J., Neill, M.O.: Grammatical evolution: evolving programs for an arbitrary language. In: Banzhaf, W., Poli, R., Schoenauer, M., Fogarty, T.C. (eds.) EuroGP 1998. LNCS, vol. 1391, pp. 83–96. Springer, Heidelberg (1998). https://doi.org/10.1007/BFb0055930

    Chapter  Google Scholar 

  22. Spector, L.: Assessment of problem modality by differential performance of lexicase selection in genetic programming: a preliminary report. In: Proceedings of the 14th Annual Conference Companion on Genetic and Evolutionary Computation, GECCO 2012, pp. 401–408. Association for Computing Machinery, New York (2012). https://doi.org/10.1145/2330784.2330846

  23. Spyromitros-Xioufis, E., Tsoumakas, G., Groves, W., Vlahavas, I.: Multi-target regression via input space expansion: treating targets as inputs. Mach. Learn. 104(1), 55–98 (2016). https://doi.org/10.1007/s10994-016-5546-z

    Article  MathSciNet  Google Scholar 

  24. Vlachos, P.: StatLib-Datasets Archive (1987). https://lib.stat.cmu.edu/datasets/

  25. Xia, Y., Chen, K., Yang, Y.: Multi-label classification with weighted classifier selection and stacked ensemble. Inf. Sci. 557, 421–442 (2021). https://doi.org/10.1016/j.ins.2020.06.017. https://www.sciencedirect.com/science/article/pii/S0020025520306058

Download references

Acknowledgments

This publication has emanated from research conducted with the financial support of Science Foundation Ireland under Grant number 16/IA/4605.

Author information

Authors and Affiliations

  1. University of Limerick, Limerick, Ireland

    Darian Reyes Fernández de Bulnes, Allan de Lima & Conor Ryan

  2. Maynooth University, Maynooth, Ireland

    Edgar Galván

Authors
  1. Darian Reyes Fernández de Bulnes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Allan de Lima
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Edgar Galván
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Conor Ryan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Darian Reyes Fernández de Bulnes .

Editor information

Editors and Affiliations

  1. University of Applied Sciences Upper Austria, Wels, Austria

    Michael Affenzeller

  2. University of Applied Sciences Upper Austria, Hagenberg, Austria

    Stephan M. Winkler

  3. Leiden University, Leiden, The Netherlands

    Anna V. Kononova

  4. University of Paderborn, Paderborn, Germany

    Heike Trautmann

  5. Jožef Stefan Institute, Ljubljana, Slovenia

    Tea Tušar

  6. University of Coimbra, Coimbra, Portugal

    Penousal Machado

  7. Leiden University, LEIDEN, The Netherlands

    Thomas Bäck

Ethics declarations

Disclosure of Interest

The authors have seen and agree with the contents of the manuscript and they declare that they have no conflict of interest.

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Reyes Fernández de Bulnes, D., de Lima, A., Galván, E., Ryan, C. (2024). Feature Encapsulation by Stages in the Regression Domain Using Grammatical Evolution. In: Affenzeller, M., et al. Parallel Problem Solving from Nature – PPSN XVIII. PPSN 2024. Lecture Notes in Computer Science, vol 15149. Springer, Cham. https://doi.org/10.1007/978-3-031-70068-2_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-70068-2_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-70067-5

  • Online ISBN: 978-3-031-70068-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation