Abstract
There are two important limitations of standard tree-based genetic programming (GP). First, GP tends to evolve unnecessarily large programs, what is referred to as bloat. Second, GP uses inefficient search operators that focus on modifying program syntax. The first problem has been studied extensively, with many works proposing bloat control methods. Regarding the second problem, one approach is to use alternative search operators, for instance geometric semantic operators, to improve convergence. In this work, our goal is to experimentally show that both problems can be effectively addressed by incorporating a local search optimizer as an additional search operator. Using real-world problems, we show that this rather simple strategy can improve the convergence and performance of tree-based GP, while also reducing program size. Given these results, a question arises: Why are local search strategies so uncommon in GP? A small survey of popular GP libraries suggests to us that local search is underused in GP systems. We conclude by outlining plausible answers for this question and highlighting future work.
Keywords
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
We will use the terms “standard GP” and “tree-based GP” interchangeably in this work, referring to the basic GP algorithm that relies on a tree representation and subtree genetic operators.
- 2.
References
Azad, R.M.A., Ryan, C.: A simple approach to lifetime learning in genetic programming based symbolic regression. Evol. Comput. 22(2), 287–317 (2014). https://doi.org/10.1162/EVCO_a_00111. http://www.mitpressjournals.org/doi/abs/10.1162/EVCO_a_00111
Castelli, M., Silva, S., Vanneschi, L.: A C++ framework for geometric semantic genetic programming. Genet. Program. Evolvable Mach. 16(1), 73–81 (2015). https://doi.org/10.1007/s10710-014-9218-0
Castelli, M., Trujillo, L., Vanneschi, L., Popovic, A.: Prediction of energy performance of residential buildings: a genetic programming approach. Energy Build. 102(1), 67–74 (2015). https://doi.org/10.1016/j.enbuild.2015.05.013. http://www.sciencedirect.com/science/article/pii/S0378778815003849
Castelli, M., Trujillo, L., Vanneschi, L., Popovic, A.: Prediction of relative position of CT slices using a computational intelligence system. Appl. Soft Comput. (2015). https://10.1016/j.asoc.2015.09.021. http://www.sciencedirect.com/science/article/pii/S1568494615005931
Castelli, M., Trujillo, L., Vanneschi, L., Silva, S., Z-Flores, E., Legrand, P.: Geometric semantic genetic programming with local search. In: Silva, S., Esparcia-Alcazar, A.I., Lopez-Ibanez, M., Mostaghim, S., Timmis, J., Zarges, C., Correia, L., Soule, T., Giacobini, M., Urbanowicz, R., Akimoto, Y., Glasmachers, T., Fernandez de Vega, F., Hoover, A., Larranaga, P., Soto, M., Cotta, C., Pereira, F.B., Handl, J., Koutnik, J., Gaspar-Cunha, A., Trautmann, H., Mouret, J.B., Risi, S., Costa, E., Schuetze, O., Krawiec, K., Moraglio, A., Miller, J.F., Widera, P., Cagnoni, S., Merelo, J., Hart, E., Trujillo, L., Kessentini, M., Ochoa, G., Chicano, F., Doerr, C. (eds.) GECCO ’15: Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation, pp. 999–1006. ACM, Madrid (2015). http://doi.acm.org/10.1145/2739480.2754795
Castelli, M., Vanneschi, L., Trujillo, L.: Energy consumption forecasting using semantics based genetic programming with local search optimizer. Comput. Intell. Neurosci. 2015 (2015). https://doi.org/10.1155/2015/971908. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4464001/
Castelli, M., Sormani, R., Trujillo, L., Popovic, A.: Predicting per capita violent crimes in urban areas: an artificial intelligence approach. J. Ambient. Intell. Humaniz. Comput. 8(1), 29–36 (2017). https://doi.org/10.1007/s12652-015-0334-3
Chen, D.Y., Chuang, T.R., Tsai, S.C.: JGAP: a java-based graph algorithms platform. Softw. Pract. Exp. 31(7), 615–635 (2001)
Chen, X., Ong, Y.S., Lim, M.H., Tan, K.C.: A multi-facet survey on memetic computation. IEEE Trans. Evol. Comput. 15(5), 591–607 (2011)
De Rainville, F.M., Fortin, F.A., Gardner, M.A., Parizeau, M., Gagne, C.: Deap: a python framework for evolutionary algorithms. In: Wagner, S., Affenzeller, M. (eds.) GECCO 2012 Evolutionary Computation Software Systems (EvoSoft), pp. 85–92. ACM, Philadelphia, PA (2012). https://doi.org/10.1145/2330784.2330799
Dignum, S., Poli, R.: Operator equalisation and bloat free GP. In: O’Neill, M., Vanneschi, L., Gustafson, S., Esparcia Alcazar, A.I., De Falco, I., Della Cioppa, A., Tarantino, E. (eds.) Proceedings of the 11th European Conference on Genetic Programming, EuroGP 2008. Lecture Notes in Computer Science, vol. 4971, pp. 110–121. Springer, Naples (2008). https://doi.org/10.1007/978-3-540-78671-9_10
Emmerich, M., Grötzner, M., Schütz, M.: Design of graph-based evolutionary algorithms: a case study for chemical process networks. Evol. Comput. 9(3), 329–354 (2001)
Gagné, C., Parizeau, M.: Open BEAGLE: A new C++ evolutionary computation framework. In: Langdon, W.B., Cantú-Paz, E., Mathias, K., Roy, R., Davis, D., Poli, R., Balakrishnan, K., V. Honavar, G. Rudolph, J. Wegener, L. Bull, M.A. Potter, A.C. Schultz, J.F. Miller, E. Burke, N. Jonoska (eds.) GECCO 2002: Proceedings of the Genetic and Evolutionary Computation Conference, p. 888. Morgan Kaufmann Publishers, New York (2002). http://www.cs.bham.ac.uk/~wbl/biblio/gecco2002/GP272.pdf
Graff, M., Pena, R., Medina, A.: Wind speed forecasting using genetic programming. In: de la Fraga, L.G. (ed.) 2013 IEEE Conference on Evolutionary Computation, vol. 1, pp. 408–415. Cancun, Mexico (2013). https://doi.org/10.1109/CEC.2013.6557598
Keijzer, M., Merelo, J.J., Romero, G., Schoenauer, M.: Evolving objects: a general purpose evolutionary computation library. In: EA-01, Evolution Artificielle, 5th International Conference in Evolutionary Algorithms, pp. 231–244 (2001). http://www.lri.fr/~marc/EO/EO-EA01.ps.gz
Kommenda, M., Kronberger, G., Winkler, S., Affenzeller, M., Wagner, S.: Effects of constant optimization by nonlinear least squares minimization in symbolic regression. In: Proceedings of the 15th Annual Conference Companion on Genetic and Evolutionary Computation, pp. 1121–1128. ACM, New York (2013)
Koza, J.R.: Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge, MA (1992). http://mitpress.mit.edu/books/genetic-programming
Koza, J.R.: Human-competitive results produced by genetic programming. Genet. Program. Evolvable Mach. 11(3/4), 251–284 (2010). https://doi.org/10.1007/s10710-010-9112-3. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.297.6227. Tenth Anniversary Issue: Progress in Genetic Programming and Evolvable Machines
La Cava, W., Spector, L.: Inheritable epigenetics in genetic programming. In: Riolo, R., Worzel, W.P., Kotanchek, M. (eds.) Genetic Programming Theory and Practice XII, Genetic and Evolutionary Computation, pp. 37–51. Springer, Ann Arbor (2014). https://doi.org/10.1007/978-3-319-16030-6_3
Langdon, W.B., Poli, R.: Foundations of Genetic Programming. Springer, Berlin (2002). https://doi.org/10.1007/978-3-662-04726-2. http://www.cs.ucl.ac.uk/staff/W.Langdon/FOGP/
Lara, A., Sanchez, G., Coello, C.A.C., Schutze, O.: Hcs: A new local search strategy for memetic multiobjective evolutionary algorithms. IEEE Trans. Evol. Comput. 14(1), 112–132 (2010)
Lohmann, R.: Application of evolution strategy in parallel populations. In: International Conference on Parallel Problem Solving from Nature, pp. 198–208. Springer, Berlin (1990)
McConaghy, T.: Ffx: fast, scalable, deterministic symbolic regression technology. In: Riolo, R., Vladislavleva, E., Moore, J.H. (eds.) Genetic Programming Theory and Practice IX, Genetic and Evolutionary Computation, chap. 13, pp. 235–260. Springer, Ann Arbor (2011). https://doi.org/10.1007/978-1-4614-1770-5_13. http://trent.st/content/2011-GPTP-FFX-paper.pdf
Moraglio, A., Krawiec, K., Johnson, C.G.: Geometric semantic genetic programming. In: International Conference on Parallel Problem Solving from Nature, pp. 21–31. Springer, Berlin (2012)
Neri, F., Cotta, C., Moscato, P.: Handbook of Memetic Algorithms, vol. 379. Springer, Berlin (2012)
Olague, G., Trujillo, L.: Evolutionary-computer-assisted design of image operators that detect interest points using genetic programming. Image Vis. Comput. 29(7), 484–498 (2011). https://doi.org/10.1016/j.imavis.2011.03.004. http://www.sciencedirect.com/science/article/B6V09-52GXV83-1/2/1462102339b445428fa4f2702939a41e
Ortigosa, I., Lopez, R., Garcia, J.: A neural networks approach to residuary resistance of sailing yachts prediction. In: Proceedings of the International Conference on Marine Engineering MARINE, vol. 2007, p. 250 (2007)
Poli, R.: TinyGP. See TinyGP GECCO 2004 competition at http://cswww.essex.ac.uk/staff/sml/gecco/TinyGP.html (2004)
Poli, R., McPhee, N.F.: General schema theory for genetic programming with subtree-swapping crossover: Part I. Evol. Comput. 11(1), 53–66 (2003). https://doi.org/10.1162/106365603321829005. http://cswww.essex.ac.uk/staff/rpoli/papers/ecj2003partI.pdf
Poli, R., McPhee, N.F.: General schema theory for genetic programming with subtree-swapping crossover: Part II. Evol. Comput. 11(2), 169–206 (2003). https://doi.org/10.1162/106365603766646825. http://cswww.essex.ac.uk/staff/rpoli/papers/ecj2003partII.pdf
Poli, R., McPhee, N.: Parsimony pressure made easy. In: Keijzer, M., Antoniol, G., Congdon, C.B., Deb, K., Doerr, B., Hansen, N., Holmes, J.H., Hornby, G.S., Howard, D., Kennedy, J., Kumar, S., Lobo, F.G., Miller, J.F., Moore, J., Neumann, F., Pelikan, M., Pollack, J., Sastry, K., Stanley, K., Stoica, A., Talbi, E.G., Wegener, I. (eds.) GECCO ’08: Proceedings of the 10th Annual Conference on Genetic and Evolutionary Computation, pp. 1267–1274. ACM, Atlanta (2008). https://doi.org/10.1145/1389095.1389340. http://www.cs.bham.ac.uk/~wbl/biblio/gecco2008/docs/p1267.pdf
Punch, B., Zongker, D.: LiL-GP 1.1. a genetic programming system (1998)
Quinlan, J.R.: Combining instance-based and model-based learning. In: Proceedings of the Tenth International Conference on Machine Learning, pp. 236–243 (1993)
Silva, S.: Reassembling operator equalisation: a secret revealed. In: Krasnogor, N., Lanzi, P.L., Engelbrecht, A., Pelta, D., Gershenson, C., Squillero, G., Freitas, A., Ritchie, M., Preuss, M., Gagne, C., Ong, Y.S., Raidl, G., Gallager, M., Lozano, J., Coello-Coello, C., Silva, D.L., Hansen, N., Meyer-Nieberg, S., Smith, J., Eiben, G., Bernado-Mansilla, E., Browne, W., Spector, L., Yu, T., Clune, J., Hornby, G., Wong, M.L., Collet, P., Gustafson, S., Watson, J.P., Sipper, M., Poulding, S., Ochoa, G., Schoenauer, M., Witt, C., Auger A. (eds.) GECCO ’11: Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation, pp. 1395–1402. ACM, Dublin (2011) https://doi.org/10.1145/2001576.2001764
Silva, S., Almeida, J.: Gplab-a genetic programming toolbox for matlab. In: Proceedings of the Nordic MATLAB Conference, pp. 273–278 (2003)
Silva, S., Costa, E.: Dynamic limits for bloat control in genetic programming and a review of past and current bloat theories. Genetic Program. Evolvable Mach. 10(2), 141–179 (2009). https://doi.org/10.1007/s10710-008-9075-9
Silva, S., Vanneschi, L.: The importance of being flat-studying the program length distributions of operator equalisation. In: Riolo, R., Vladislavleva, E., Moore J.H. (eds.) Genetic Programming Theory and Practice IX, Genetic and Evolutionary Computation, chap. 12, pp. 211–233. Springer, Ann Arbor (2011). https://doi.org/10.1007/978-1-4614-1770-5_12
Silva, S., Dignum, S., Vanneschi, L.: Operator equalisation for bloat free genetic programming and a survey of bloat control methods. Genetic Programming and Evolvable Machines 13(2), 197–238 (2012). https://doi.org/10.1007/s10710-011-9150-5
Stanley, K.O., Miikkulainen, R.: Evolving neural networks through augmenting topologies. Evol. Comput. 10(2), 99–127 (2002)
Topchy, A., Punch, W.F.: Faster genetic programming based on local gradient search of numeric leaf values. In: Spector, L., Goodman, E.D., Wu, A., Langdon, W.B., Voigt, H.M., Gen, M., Sen, S., Dorigo, M., Pezeshk, S., Garzon, M.H., Burke E. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2001), pp. 155–162. Morgan Kaufmann, San Francisco, CA (2001). http://garage.cse.msu.edu/papers/GARAGe01-07-01.pdf
Trujillo, L., Legrand, P., Olague, G., Levy-Vehel, J.: Evolving estimators of the pointwise hoelder exponent with genetic programming. Inf. Sci. 209, 61–79 (2012). https://doi.org/10.1016/j.ins.2012.04.043. http://www.sciencedirect.com/science/article/pii/S0020025512003386
Trujillo, L., Munoz, L., Naredo, E., Martinez, Y.: Neat, there’s no bloat. In: Nicolau, M., Krawiec, K., Heywood, M.I., Castelli, M., Garcia-Sanchez, P., Merelo, J.J., Rivas Santos, V.M., Sim K. (eds.) 17th European Conference on Genetic Programming. Lecture Notes in Computer Science, vol. 8599, pp. 174–185. Springer, Granada (2014). https://doi.org/10.1007/978-3-662-44303-3_15
Trujillo, L., Munoz, L., Galvan-Lopez, E., Silva, S.: Neat genetic programming: controlling bloat naturally. Inf. Sci. 333, 21–43 (2016). https://doi.org/10.1016/j.ins.2015.11.010. http://www.sciencedirect.com/science/article/pii/S0020025515008038
Tsanas, A., Xifara, A.: Accurate quantitative estimation of energy performance of residential buildings using statistical machine learning tools. Energy Build. 49, 560–567 (2012)
Vanneschi, L., Castelli, M., Silva, S.: A survey of semantic methods in genetic programming. Genetic Program. Evolvable Mach. 15(2), 195–214 (2014). https://doi.org/10.1007/s10710-013-9210-0. http://link.springer.com/article/10.1007/s10710-013-9210-0
Wagner, S., Affenzeller, M.: Heuristiclab: A generic and extensible optimization environment. In: Adaptive and Natural Computing Algorithms, pp. 538–541. Springer, Berlin, (2005)
White, D.R.: Software review: the ECJ toolkit. Genet. Program. Evolvable Mach. 13(1), 65–67 (2012). https://doi.org/10.1007/s10710-011-9148-z
Worm, T., Chiu, K.: Prioritized grammar enumeration: symbolic regression by dynamic programming. In: Blum, C., Alba, E., Auger, A., Bacardit, J., Bongard, J., Branke, J., Bredeche, N., Brockhoff, D., Chicano, F., Dorin, A., Doursat, R., Ekart, A., Friedrich, T., Giacobini, M., Harman, M., Iba, H., Igel, C., Jansen, T., Kovacs, T., Kowaliw, T., Lopez-Ibanez, M., Lozano, J.A., Luque, G., McCall, J., Moraglio, A., Motsinger-Reif, A., Neumann, F., Ochoa, G., Olague, G., Ong, Y.S., Palmer, M.E., Pappa, G.L., Parsopoulos, K.E., Schmickl, T., Smith, S.L., Solnon, C., Stuetzle, T., Talbi, E.G., Tauritz, D., Vanneschi L. (eds.) GECCO ’13: Proceeding of the Fifteenth Annual Conference on Genetic and Evolutionary Computation Conference, pp. 1021–1028. ACM, Amsterdam (2013). https://doi.org/10.1145/2463372.2463486
Z-Flores, E., Trujillo, L., Schuetze, O., Legrand, P.: Evaluating the effects of local search in genetic programming. In: Tantar, A.A., Tantar, E., Sun, J.Q., Zhang, W., Ding, Q., Schuetze, O., Emmerich, M., Legrand, P., Del Moral, P., Coello Coello, C.A. (eds.) EVOLVE - A Bridge between Probability, Set Oriented Numerics, and Evolutionary Computation V. Advances in Intelligent Systems and Computing, vol. 288, pp. 213–228. Springer, Peking (2014). https://doi.org/10.1007/978-3-319-07494-8_15. https://hal.inria.fr/hal-01060315
Z-Flores, E., Trujillo, L., Schuetze, O., Legrand, P.: A local search approach to genetic programming for binary classification. In: Silva, S., Esparcia-Alcazar, A.I., Lopez-Ibanez, M., Mostaghim, S., Timmis, J., Zarges, C., Correia, L., Soule, T., Giacobini, M., Urbanowicz, R., Akimoto, Y., Glasmachers, T., Fernandez de Vega, F., Hoover, A., Larranaga, P., Soto, M., Cotta, C., Pereira, F.B., Handl, J., Koutnik, J., Gaspar-Cunha, A., Trautmann, H., Mouret, J.B., Risi, S., Costa, E., Schuetze, O., Krawiec, K., Moraglio, A., Miller, J.F., Widera, P., Cagnoni, S., Merelo, J., Hart, E., Trujillo, L., Kessentini, M., Ochoa, G., Chicano, F., Doerr, C. (eds.) GECCO ’15: Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation, pp. 1151–1158. ACM, Madrid (2015). http://doi.acm.org/10.1145/2739480.2754797
Zhang, M., Smart, W.: Genetic programming with gradient descent search for multiclass object classification. In: Keijzer, M., O’Reilly, U.M., Lucas, S.M., Costa, E., Soule T. (eds.) Genetic Programming 7th European Conference, EuroGP 2004, Proceedings, Lecture Notes in Computer Science, vol. 3003, pp. 399–408. Springer, Coimbra (2004). https://doi.org/10.1007/978-3-540-24650-3_38. http://www.springerlink.com/openurl.asp?genre=article&issn=0302-9743&volume=3003&spage=399
Acknowledgements
Funding for this work was provided by CONACYT Basic Science Research Project No. 178323 and Fronteras de la Ciencia FC-2015-2/944, TecNM (México) Research Projects 5414.14-P and 5621.15-P, and by the FP7 Marie Curie-IRSES 2013 European Commission program through project ACoBSEC with contract 439 No. 612689
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Trujillo, L. et al. (2018). Local Search is Underused in Genetic Programming. In: Riolo, R., Worzel, B., Goldman, B., Tozier, B. (eds) Genetic Programming Theory and Practice XIV. Genetic and Evolutionary Computation. Springer, Cham. https://doi.org/10.1007/978-3-319-97088-2_8
Download citation
DOI: https://doi.org/10.1007/978-3-319-97088-2_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-97087-5
Online ISBN: 978-3-319-97088-2
eBook Packages: Computer ScienceComputer Science (R0)