Abstract
Geometric semantic genetic programming (GSGP) represents one of the most promising developments in the area of evolutionary computation (EC) in the last decade. The results achieved by incorporating semantic awareness in the evolutionary process demonstrate the impact that geometric semantic operators have brought to the field of EC. An improvement to the geometric semantic mutation (GSM) operator is proposed, inspired by the results achieved by batch normalization in deep learning. While, in one of its most used versions, GSM relies on the use of the sigmoid function to constrain the semantics of two random programs responsible for perturbing the parent’s semantics, here a different approach is followed, which allows reducing the size of the resulting programs and overcoming the issues associated with the use of the sigmoid function, as commonly done in deep learning. The idea is to consider a single random program and use it to perturb the parent’s semantics only after standardization or normalization. The experimental results demonstrate the suitability of the proposed approach: despite its simplicity, the presented GSM variants outperform standard GSGP on the studied benchmarks, with a difference in terms of performance that is statistically significant. Furthermore, the individuals generated by the new GSM variants are easier to simplify, allowing us to create accurate but significantly smaller solutions.
Similar content being viewed by others
Notes
Only the definitions of GSOs for symbolic regression problems are presented in this paper as they are the only ones used. For the definitions of GSOs in other domains, the reader is referred to [28].
Given the nature of GSM and GSC, the ideas discussed here generalize to any type of program representation (trees, linear representations, graphs, etc.). The experimental work will use a linear genome representation for programs.
Version 1.10.1: https://github.com/sympy/sympy.
Boxplots are used instead of violin plots for easier depiction.
References
J. Albinati, G.l. Pappa, F.E. Otero et al., The effect of distinct geometric semantic crossover operators in regression problems, in Genetic Programming: 18th European Conference, EuroGP 2015, Copenhagen, Denmark, April 8–10, 2015, Proceedings 18 (Springer, 2015), pp. 3–15
I. Bakurov, L. Vanneschi, M. Castelli et al., Edda-v2—An improvement of the evolutionary demes despeciation algorithm, in Parallel Problem Solving from Nature—PPSN XV. ed. by A. Auger, C.M. Fonseca, N. Lourenço et al. (Springer International Publishing, Cham, 2018), pp. 185–196
I. Bakurov, M. Buzzelli, M. Castelli et al., General purpose optimization library (GPOL): a flexible and efficient multi-purpose optimization library in python. Appl. Sci. (2021). https://doi.org/10.3390/app11114774
I. Bakurov, M. Castelli, O. Gau et al., Genetic programming for stacked generalization. Swarm Evol. Comput. 65, 100913 (2021). https://doi.org/10.1016/j.swevo.2021.100913
I. Bakurov, M. Castelli, F. Fontanella et al., A novel binary classification approach based on geometric semantic genetic programming. Swarm Evol. Comput. 69, 101028 (2022). https://doi.org/10.1016/j.swevo.2021.101028
L. Beadle, C.G. Johnson, Semantically driven crossover in genetic programming, in 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence) (IEEE, 2008), pp. 111–116
L. Beadle, C.G. Johnson, Semantic analysis of program initialisation in genetic programming. Genet. Program Evolvable Mach. 10, 307–337 (2009)
L. Beadle, C.G. Johnson, Semantically driven mutation in genetic programming, in 2009 IEEE Congress on Evolutionary Computation (IEEE, 2009), pp. 1336–1342
M. Castelli, S. Silva, L. Vanneschi, A C++ framework for geometric semantic genetic programming. Genet. Program Evolvable Mach. 16(1), 73–81 (2015)
M. Castelli, L. Trujillo, L. Vanneschi et al., Geometric semantic genetic programming with local search, in Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation, GECCO ’15 (Association for Computing Machinery, New York, NY, USA, 2015), pp. 999–1006. https://doi.org/10.1145/2739480.2754795
M. Castelli, L. Manzoni, I. Gonçalves, et al., An analysis of geometric semantic crossover: a computational geometry approach, in International Joint Conference on Computational Intelligence (2016)
F. Chollet et al., Keras (2015). https://keras.io
J. Derrac, S. García, D. Molina et al., A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evol. Comput. 1(1), 3–18 (2011). https://doi.org/10.1016/j.swevo.2011.02.002
I. Gonçalves, S. Silva, C.M. Fonseca, On the generalization ability of geometric semantic genetic programming, in Genetic Programming. ed. by P. Machado, M.I. Heywood, J. McDermott et al. (Springer, Cham, 2015), pp.41–52
I. Gonçalves, S. Silva, C.M. Fonseca, Semantic learning machine: A feedforward neural network construction algorithm inspired by geometric semantic genetic programming, in Progress in Artificial Intelligence. ed. by F. Pereira, P. Machado, E. Costa et al. (Springer, Cham, 2015), pp.280–285
I. Gonçalves, S. Silva, C.M. Fonseca et al., Unsure when to stop?, in Proceedings of the Genetic and Evolutionary Computation Conference (ACM, 2017). https://doi.org/10.1145/3071178.3071328
I. Gonçalves, An exploration of generalization and overfitting in genetic programming: standard and geometric semantic approaches. Ph.D. Thesis, Department of Informatics Engineering, University of Coimbra, Portugal., Coimbra, Portugal (2017), available at https://www.cisuc.uc.pt/download-file/13946/sfxgEyeIRXv2dxxWgZS5
I.J. Goodfellow, Y. Bengio, A. Courville, Deep Learning (MIT Press, Cambridge, 2016)
K. He, Z. Zhang, S. Ren et al., Deep residual learning for image recognition, in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), pp. 770–778. https://doi.org/10.1109/CVPR.2016.90
G. Huang, Z. Liu, L.V.D. Maaten et al., Densely connected convolutional networks, in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE Computer Society, Los Alamitos, CA, USA, 2017), pp. 2261–2269. https://doi.org/10.1109/CVPR.2017.243, https://doi.ieeecomputersociety.org/10.1109/CVPR.2017.243
S. Ioffe, C. Szegedy, Batch normalization: Accelerating deep network training by reducing internal covariate shift, in Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37. JMLR.org, ICML’15 (2015) pp. 448–456
J.R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection, vol. 1 (MIT Press, Cambridge, 1992)
J.R. Koza, Human-competitive results produced by genetic programming. Genet. Program Evolvable Mach. 11, 251–284 (2010)
Y.A. LeCun, L. Bottou, G.B. Orr et al., Efficient BackProp (Springer, Berlin, 2012), pp.9–48. https://doi.org/10.1007/978-3-642-35289-8_3
J.F.B.S. Martins, L.O.V.B. Oliveira, L.F. Miranda et al., Solving the exponential growth of symbolic regression trees in geometric semantic genetic programming, in Proceedings of the Genetic and Evolutionary Computation Conference, GECCO ’18 (ACM, New York, NY, USA, 2018), pp. 1151–1158
J. McDermott, A. Agapitos, A. Brabazon et al., Geometric semantic genetic programming for financial data, in Applications of Evolutionary Computation: 17th European Conference, EvoApplications 2014, Granada, Spain, April 23–25, 2014, Revised Selected Papers 17, (Springer, 2014), pp. 215–226
NF. McPhee, B. Ohs, T. Hutchison, Semantic building blocks in genetic programming, in Genetic Programming: 11th European Conference, EuroGP 2008, Naples, Italy, March 26–28, 2008. Proceedings 11 (Springer, 2008), pp. 134–145
A. Moraglio, K. Krawiec, C. Johnson, Geometric semantic genetic programming, in Parallel Problem Solving from Nature—PPSN XII, ed. by C. Coello, V. Cutello, K. Deb, et al. Lecture Notes in Computer Science, vol. 7491 (Springer, Berlin, 2012), pp. 21–31
V. Nair, G.E. Hinton, Rectified linear units improve restricted Boltzmann machines, in Proceedings of the 27th International Conference on International Conference on Machine Learning, ICML’10 (Omnipress, Madison, WI, USA, 2010), pp. 807–814
M. Nicolau, J. McDermott, Genetic programming symbolic regression: What is the prior on the prediction?, in Genetic Programming Theory and Practice XVII (2020), pp. 201–225
L.O.V. Oliveira, F.E. Otero, G.L. Pappa, A dispersion operator for geometric semantic genetic programming, in Proceedings of the Genetic and Evolutionary Computation Conference, vol. 2016 (2016), pp. 773–780
I. Ortigosa, J.G.R. Lopez, A neural networks approach to residuary resistance of sailing yachts prediction, in Proceedings of the International Conference on Marine Engineering MARINE (2007), p. 250
A. Paszke, S. Gross, F. Massa et al., PyTorch: An Imperative Style, High-Performance Deep Learning Library (Curran Associates Inc., Red Hook, NY, USA, 2019)
J.R. Quinlan, Combining instance-based and model-based learning, in Machine Learning, Proceedings of the Tenth International Conference, University of Massachusetts, Amherst, MA, USA, June 27–29, 1993 (1993), pp. 236–243
L. Trujillo, J.M. Muñoz Contreras, D.E. Hernandez et al., GSGP-CUDA—a CUDA framework for geometric semantic genetic programming. SoftwareX 18, 101085 (2022). https://doi.org/10.1016/j.softx.2022.101085
A. Tsanas, A. Xifara, Accurate quantitative estimation of energy performance of residential buildings using statistical machine learning tools. Energy Build. 49, 560–567 (2012)
N.Q. Uy, N.X. Hoai, M. O’Neill et al., Semantically-based crossover in genetic programming: application to real-valued symbolic regression. Genet. Program Evolvable Mach. 12, 91–119 (2011)
L. Vanneschi, S. Silva, M. Castelli et al., Geometric semantic genetic programming for real life applications, in Genetic Programming Theory and Practice XI (2014), pp. 191–209
L. Vanneschi, S. Silva, M. Castelli et al., Geometric Semantic Genetic Programming for Real Life Applications (Springer, New York, 2014), pp.191–209
L. Vanneschi, I. Bakurov, M. Castelli, An initialization technique for geometric semantic GP based on demes evolution and despeciation, in 2017 IEEE Congress on Evolutionary Computation (CEC) (2017), pp. 113–120. https://doi.org/10.1109/CEC.2017.7969303
E.J. Vladislavleva, G.F. Smits, D. den Hertog, Order of nonlinearity as a complexity measure for models generated by symbolic regression via pareto genetic programming. IEEE Trans. Evol. Comput. 13(2), 333–349 (2009)
I.C. Yeh, Modeling of strength of high-performance concrete using artificial neural networks. Cem. Concr. Res. 28(12), 1797–1808 (1998)
Acknowledgements
This work was partially supported by FCT, Portugal, through funding of research units MagIC/NOVA IMS (UIDB/04152/2020) and LASIGE (UIDB/00408/2020 and UIDP/00408/2020). This work also was supported by CONACYT (Mexico) Project CF-2023-I-724, TecNM (Mexico) Project 16788.23-P and Project 17756.23-P. José Manuel Muñoz Contreras was supported by CONACYT scholarship 771416; Nuno Rodrigues was supported by FCT PhD Grant 2021/05322/BD.
Author information
Authors and Affiliations
Contributions
Conceptualization: IB, MC, NR, SS, LV; methodology: IB, JMMC, LT, LV; formal analysis and investigation: IB, JMMC, LT; writing—original draft preparation: IB, JMMC, MC, NR, LV; writing—review and editing: MC, SS, LT, LV; funding acquisition: LT, LV, MC, SS; supervision: LT, LV.
Corresponding author
Ethics declarations
Conflict of interest
The authors have no conflicts of interest to declare that are relevant to the content of this article. Several authors are board members of genetic programming and evolvable machines. Leonardo Trujillo, Sara Silva and Leonardo Vanneschi are Associate Editors, while Mauro Castelli is on the Editorial Board.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Bakurov, I., Muñoz Contreras, J.M., Castelli, M. et al. Geometric semantic genetic programming with normalized and standardized random programs. Genet Program Evolvable Mach 25, 6 (2024). https://doi.org/10.1007/s10710-024-09479-1
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10710-024-09479-1