Abstract
For all supervised learning problems, where the quality of solutions is measured by a distance between target and output values (error), geometric semantic operators of genetic programming induce an error surface characterized by the absence of locally suboptimal solutions (unimodal error surface). So, genetic programming that uses geometric semantic operators, called geometric semantic genetic programming, has a potential advantage in terms of evolvability compared to many existing computational methods. This fosters geometric semantic genetic programming as a possible new state-of-the-art machine learning methodology. Nevertheless, research in geometric semantic genetic programming is still much in demand. This chapter is oriented to researchers and students that are not familiar with geometric semantic genetic programming, and are willing to contribute to this exciting and promising field. The main objective of this chapter is explaining why the error surface induced by geometric semantic operators is unimodal, and why this fact is important. Furthermore, the chapter stimulates the reader by showing some promising applicative results that have been obtained so far. The reader will also discover that some properties of geometric semantic operators may help limiting overfitting, bestowing on genetic programming a very interesting generalization ability. Finally, the chapter suggests further reading and discusses open issues of geometric semantic genetic programming.
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
When the quality of a solution, or fitness, is equal to an error between calculated values and targets, like in the cases studied in this chapter, the terms error surface and fitness landscape are synonymous. The former term is generally used by the machine learning community, while the latter is more popular in the evolutionary computation terminology. In this chapter, these two terms will be used interchangeably.
- 2.
The example in Sect. 3.4, contrarily to the previous examples, is a case of continuous optimization. Thus, it is practically impossible to find exactly the global optimum. From now on, when continuous optimization is considered, the term “solving” the problem will be used to indicate that it is possible to approximate a globally optimal solution with any prefixed precision.
- 3.
In several references [49], the term ball mutation, instead of box mutation, can be found for indicating this operator. If the area of variation induced by this operator can geometrically be represented as a “box” or a “ball”, it depends on the particular metric used, as explained in [36]. In this simple example, we are considering the intuitive Euclidean distance and this is why we use the term box mutation.
- 4.
The word “cono” actually means cone in several languages of Latin origin, among which Italian and Spanish.
- 5.
How could it be otherwise? GP is working with a population of trees, so the genetic operators can only act on them!
- 6.
- 7.
The term “ancestors” here is a bit abused to designate not only the parents but also the random trees used to build an individual by crossover or mutation.
References
Aarts, E., Korst, J.: Simulated Annealing and Boltzmann Machines: A Stochastic Approach to Combinatorial Optimization and Neural Computing. Wiley, New York (1989)
Applegate, D.L., Bixby, R.E., Chvatal, V., Cook, W.J.: The Traveling Salesman Problem: A Computational Study (Princeton Series in Applied Mathematics). Princeton University Press, Princeton (2007)
Back, T., et al. (eds.): Handbook of Evolutionary Computation, 1st edn. IOP Publishing Ltd., Bristol (1997)
Castelli, M., Castaldi, D., Giordani, I., Silva, S., Vanneschi, L., Archetti, F., Maccagnola, D.: An efficient implementation of geometric semantic genetic programming for anticoagulation level prediction in pharmacogenetics. In: Correia, L., et al. (eds.) Progress in Artificial Intelligence. Lecture Notes in Computer Science, vol. 8154, pp. 78–89. Springer, Berlin (2013)
Castelli, M., Henriques, R., Vanneschi, L.: A geometric semantic genetic programming system for the electoral redistricting problem. Neurocomputing 154, 200–207 (2015)
Castelli, M., Silva, S., Vanneschi, L.: A C++ framework for geometric semantic genetic programming. Genet. Program. Evol. Mach. 1–9 (2014)
Castelli, M., Silva, S., Vanneschi, L., Cabral, A., Vasconcelos, M., Catarino, L., Carreiras, J.: Land cover/land use multiclass classification using gp with geometric semantic operators. In: Esparcia-Alczar, A. (ed.) Applications of Evolutionary Computation. Lecture Notes in Computer Science, vol. 7835, pp. 334–343. Springer, Berlin (2013)
Castelli, M., Trujillo, L., Vanneschi, L.: Energy consumption forecasting using semantic-based genetic programming with local search optimizer. Comput. Intell. Neurosci. Article ID 971908, 8 p. (2015). http://dx.doi.org/10.1155/2015/971908
Castelli, M., Trujillo, L., Vanneschi, L., Popovic, A.: Prediction of energy performance of residential buildings: A genetic programming approach. Energy Build. 102, 67–74 (2015)
Castelli, M., Trujillo, L., Vanneschi, L., Popovic, A.: Prediction of relative position of CT slices using a computational intelligence system. Appl. Soft Comput. (2015, in press)
Castelli, M., Trujillo, L., Vanneschi, L., Silva, S., Z-Flores, E., Legrand, P.: Geometric semantic genetic programming with local search. In: Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation, GECCO ’15, pp. 999–1006. ACM, New York, NY, USA (2015)
Castelli, M., Vanneschi, L., Felice, M.D.: Forecasting short-term electricity consumption using a semantics-based genetic programming framework: The south italy case. Energy Econ. 47, 37–41 (2015)
Castelli, M., Vanneschi, L., Manzoni, L., Popovic, A.: Semantic genetic programming for fast and accurate data knowledge discovery. Swarm Evol. Comput. (2015, in press)
Castelli, M., Vanneschi, L., Silva, S.: Prediction of high performance concrete strength using genetic programming with geometric semantic genetic operators. Expert Syst. Appl. 40(17), 6856–6862 (2013)
Castelli, M., Vanneschi, L., Silva, S.: Prediction of the unified parkinson’s disease rating scale assessment using a genetic programming system with geometric semantic genetic operators. Expert Syst. Appl. 41(10), 4608–4616 (2014)
Darwin, C.: On the Origin of Species by Means of Natural Selection. Murray, London (1859) or the Preservation of Favored Races in the Struggle for Life
Dick, G.: Improving geometric semantic genetic programming with safe tree initialisation. In: Machado, P., et al. (eds.) 18th European Conference on Genetic Programming. LNCS, vol. 9025, pp. 28–40. Springer, Copenhagen, 8–10 April 2015
Fan, W., Bifet, A.: Mining big data: current status, and forecast to the future. SIGKDD Explor. Newsl. 14(2), 1–5 (2013)
Garey, M.R., Johnson, D.S.: Computers and Intractability: A Guide to the Theory of NP-Completeness. W. H. Freeman & Co., New York (1979)
Haykin, S.: Neural Networks: A Comprehensive Foundation. Prentice Hall, Upper Saddle River (1999)
Hoffmann, L.: Multivariate Isotonic Regression and Its Algorithms. Wichita State University, College of Liberal Arts and Sciences, Department of Mathematics and Statistics (2009)
Keijzer, M.: Improving symbolic regression with interval arithmetic and linear scaling. In: Genetic Programming, Proceedings of EuroGP’2003. LNCS, vol. 2610, pp. 70–82. Springer (2003)
Kennedy, J., Eberhart, R.C.: Swarm Intelligence. Morgan Kaufmann Publishers Inc., San Francisco (2001)
Kirkpatrick, S., Gelatt Jr., C.D., Vecchi, M.P.: Optimization by simulated annealing. Science 220, 671–680 (1983)
Koza, J.R.: Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge (1992)
Krawiec, K.: Behavioral Program Synthesis with Genetic Programming. Studies in Computational Intelligence, vol. 618. Springer, Berlin (2016)
Langdon, W.B., Poli, R.: Foundations of Genetic Programming. Springer, Berlin (2002)
Mambrini, A., Manzoni, L., Moraglio, A.: Theory-laden design of mutation-based geometric semantic genetic programming for learning classification trees. In: 2013 IEEE Congress on Evolutionary Computation (CEC), pp. 416–423 (2013)
Martello, S., Toth, P.: Knapsack Problems: Algorithms and Computer Implementations. Wiley, New York (1990)
Moraglio, A.: Towards a Geometric Unification of Evolutionary Algorithms. Ph.D. thesis, Department of Computer Science, University of Essex, UK (2007)
Moraglio, A.: An efficient implementation of GSGP using higher-order functions and memoization. In: Johnson, C., et al. (eds.) Semantic Methods in Genetic Programming, Ljubljana, Slovenia, 13 Sept. 2014. Workshop at Parallel Problem Solving from Nature 2014 conference (2014)
Moraglio, A., Krawiec, K., Johnson, C.G.: Geometric semantic genetic programming. In: Parallel Problem Solving from Nature, PPSN XII (part 1). Lecture Notes in Computer Science, vol. 7491, pp. 21–31. Springer (2012)
Moraglio, A., Mambrini, A.: Runtime analysis of mutation-based geometric semantic genetic programming for basis functions regression. In: Blum, C., et al. (eds.) Proceedings of the 15th annual international conference on Genetic and Evolutionary Computation. GECCO ’13, pp. 989–996. ACM, New York, NY, USA (2013)
Moraglio, A., Mambrini, A., Manzoni, L.: Runtime analysis of mutation-based geometric semantic genetic programming on boolean functions. In: Neumann, F., De Jong, K. (eds.) Foundations of Genetic Algorithms, pp. 119–132. ACM, Adelaide, Australia, 16–20 January 2013
Nocedal, J., Wright, S.J.: Numerical Optimization, 2nd edn. World Scientific, Singapore (2006)
Pawlak, T.P., Krawiec, K.: Progress properties and fitness bounds for geometric semantic search operators. Genetic Programming and Evolvable Machines (Online first)
Pawlak, T.P., Krawiec, K.: Guarantees of progress for geometric semantic genetic programming. In: Johnson, C., et al. (eds.) Semantic Methods in Genetic Programming, Ljubljana, Slovenia, 13 Sept. 2014. Workshop at Parallel Problem Solving from Nature 2014 conference (2014)
Pawlak, T.P., Wieloch, B., Krawiec, K.: Review and comparative analysis of geometric semantic crossovers. Genet. Progr. Evol. Mach. 16(3), 351–386 (2015)
Pawlak, T.P., Wieloch, B., Krawiec, K.: Semantic backpropagation for designing search operators in genetic programming. IEEE Trans. Evol. Comput. 19(3), 326–340 (2015)
Poli, R., Langdon, W.B., Mcphee, N.F.: A field guide to genetic programming (2008)
Richter, H., Engelbrecht, A. (eds.): Recent Advances in the Theory and Application of Fitness Landscapes. Emergence. Complexity and Computation, vol. 6. Springer, Berlin (2014)
Schölkopf, B., Smola, A.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. Adaptive computation and machine learning. MIT Press (2002)
Seber, G., Wild, C.: Nonlinear Regression. Wiley Series in Probability and Statistics. Wiley (2003)
Silva, S., Ingalalli, V., Vinga, S., Carreiras, J., Melo, J., Castelli, M., Vanneschi, L., Gonalves, I., Caldas, J.: Prediction of forest aboveground biomass: An exercise on avoiding overfitting. In: Esparcia-Alczar, A. (ed.) Applications of Evolutionary Computation. Lecture Notes in Computer Science, vol. 7835, pp. 407–417. Springer, Berlin Heidelberg (2013)
Tomassini, M., Vanneschi, L., Collard, P., Clergue, M.: A study of fitness distance correlation as a difficulty measure in genetic programming. Evol. Comput. 13(2), 213–239 (2005)
Vanneschi, L.: Theory and Practice for Efficient Genetic Programming. Ph.D. thesis, Faculty of Sciences, University of Lausanne, Switzerland (2004)
Vanneschi, L.: Improving genetic programming for the prediction of pharmacokinetic parameters. Memet. Comput. 6(4), 255–262 (2014)
Vanneschi, L., Castelli, M., Costa, E., Re, A., Vaz, H., Lobo, V., Urbano, P.: Improving maritime awareness with semantic genetic programming and linear scaling: prediction of vessels position based on ais data. In: Mora, A.M., Squillero, G. (eds.) Applications of Evolutionary Computation. Lecture Notes in Computer Science, vol. 9028, pp. 732–744. Springer International Publishing (2015)
Vanneschi, L., Castelli, M. Manzoni, L., Silva, S.: A new implementation of geometric semantic GP and its application to problems in pharmacokinetics. In: Proceedings of the 16th European Conference on Genetic Programming, EuroGP 2013. LNCS, vol. 7831, pp. 205–216. Springer, Vienna, Austria, 3–5 April 2013
Vanneschi, L., Castelli, M., Silva, S.: A survey of semantic methods in genetic programming. Genet. Progr. Evol. Mach. 15(2), 195–214 (2014)
Vanneschi, L., Silva, S., Castelli, M., Manzoni, L.: Geometric semantic genetic programming for real life applications. In: Riolo, R., et al. (eds.) Genetic Programming Theory and Practice XI, Genetic and Evolutionary Computation. Springer US, Computer Science Collection, 2013. Invited article (2013, to appear)
Weisberg, S.: Applied Linear Regression. Wiley, Wiley Series in Prob. and Stat (2005)
Wright, S.: The roles of mutation, inbreeding, crossbreeding and selection in evolution. In: Jones, D.F. (ed.) Proceedings on the Sixth International Congress on Genetics, vol. 1, pp. 356–366 (1932)
Acknowledgments
My heartfelt acknowledgment goes to Sara Silva and Mauro Castelli, who shared with me the work on geometric semantic genetic programming and much much more! I also dearly thank Leonardo Trujillo and Oliver Schütze for inviting me to the NEO 2015 workshop. It has been an unforgettable experience.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Vanneschi, L. (2017). An Introduction to Geometric Semantic Genetic Programming. In: Schütze, O., Trujillo, L., Legrand, P., Maldonado, Y. (eds) NEO 2015. Studies in Computational Intelligence, vol 663. Springer, Cham. https://doi.org/10.1007/978-3-319-44003-3_1
Download citation
DOI: https://doi.org/10.1007/978-3-319-44003-3_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-44002-6
Online ISBN: 978-3-319-44003-3
eBook Packages: EngineeringEngineering (R0)