Abstract
Symbolic regression is to search the space of mathematical expressions to find a model that best fits a given dataset. As genetic programming (GP) with the tree representation can represent solutions as expression trees, it is popularly-used for regression. However, GP tends to evolve unnecessarily large programs (known as bloat), causing excessive use of CPU time/memory and evolving solutions with poor generalization ability. Moreover, even though the importance of local search has been proved in augmenting the search ability of GP (termed as memetic algorithms), local search is underused in GP-based methods. This work aims to handle the above problems simultaneously. To control bloat, a multi-objective (MO) technique (NSGA-II, Non-dominant Sorting Genetic Algorithm) is selected to incorporate with GP, forming a multi-objective GP (MOGP). Moreover, three mutation-based local search operators are designed and incorporated with MOGP respectively to form three multi-objective memetic algorithms (MOMA), i.e. MOMA_MR (MOMA with Mutation-based Random search), MOMA_MF (MOMA with Mutation-based Function search) and MOMA_MC (MOMA with Mutation-based Constant search). The proposed methods are tested on both benchmark functions and real-world applications, and are compared with both GP-based (i.e. GP and MOGP) and nonGP-based symbolic regression methods. Compared with GP-based methods, the proposed methods can reduce the risk of bloat with the evolved solutions significantly smaller than GP solutions, and the local search strategies introduced in the proposed methods can improve their search ability with the evolved solutions dominating MOGP solutions. In addition, among the three proposed methods, MOMA_MR performs best in RMSE for testing, yet it consumes more training time than others. Moreover, compared with six reference nonGP-based symbolic regression methods, MOMA_MR generally performs better than or similar to them consistently.
Similar content being viewed by others
Notes
Weka is a set of machine learning algorithms for solving real-world data mining tasks [7].
References
Alexandropoulos SAN, Aridas CK, Kotsiantis SB, Vrahatis MN (2019) Multi-objective evolutionary optimization algorithms for machine learning: a recent survey. In: Approximation and optimization, vol 145. Springer, Cham
Azzouz R, Bechikh S, Said LB (2017) Dynamic multi-objective optimization using evolutionary algorithms: a survey. In: Recent advances in evolutionary multi-objective optimization, vol 20. Springer, Cham
Boryczka U, Szwarc K (2019) Selected variants of a memetic algorithm for jsp - a comparative study. Int J Prod Res 44:1–16
Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans Evolut Comput 6(2):182–197
Doerr B, Kötzing T, Lagodzinski JAG, Lengler J (2018) Bounding bloat in genetic programming. In: genetic and evolutionary computation conference
Dua D, Graff C (2017) UCI machine learning repository . http://archive.ics.uci.edu/ml
Frank E, Hall MA, Witten IH (2016) Data Mining: practical machine learning tools and techniques (Fourth Edition)
Izadi Rad H, Feng J, Iba H (2018) GP-RVM: Genetic programing-based symbolic regression using relevance vector machine. In: International conference on systems, man, and cybernetics (SMC)
Jie L, Jia H, Verma N, Jha NK (2018) Genetic programming for energy-efficient and energy-scalable approximate feature computation in embedded inference systems. IEEE Trans Comput 67:222–236
Kordestani JK, Firouzjaee HA, Meybodi MR (2018) An adaptive bi-flight cuckoo search with variable nests for continuous dynamic optimization problems. Appl Intell 48(1):97–117
Koza JR (1992) Genetic programming: on the programming of computers by means of natural selection. MIT press, Cambridge
Koza JR (2014) Genetic programming ii: automatic discovery of reusable programs. Artif Life 1(4):439–441
Kronberger G, Kammerer L, Burlacu B, Winkler SM, Kommenda M, Affenzeller M (2019) Cluster analysis of a symbolic regression search space. In: Genetic programming theory and practice XVI. Springer, Cham
Li B, Li J, Ke T, Xin Y (2015) Many-objective evolutionary algorithms:a survey. ACM Comput Surv 48(1):1–35
Li B, Yang X (2019) Xuan H (2019) A hybrid simulated annealing heuristic for multistage heterogeneous fleet scheduling with fleet sizing decisions. J Adv Trans 10:1–19
Liang Y, Zhang M, Browne WN (2016) Figure-ground image segmentation using genetic programming and feature selection. In: IEEE congress on evolutionary computation (CEC), pp. 3839–3846. IEEE
Liang Y, Zhang M, Browne WN (2016) Multi-objective genetic programming for figure-ground image segmentation. In: Australasian conference on artificial life and computational intelligence, pp. 134–146. Springer
Liang Y, Zhang M, Browne WN (2017) Image feature selection using genetic programming for figure-ground segmentation. Eng Appl Artif Intell 62:96–108
Liao CC, Ting CK (2018) A novel integer-coded memetic algorithm for the set\(k\)-cover problem in wireless sensor networks. IEEE Trans Cybern 48(8):1–14
Luke S, Panait L (2006) A comparison of bloat control methods for genetic programming. Evolut Comput 14(3):309–344
Luo J, Yun Y, Liu Q, Xia L, Gao K (2018) A new hybrid memetic multi-objective optimization algorithm for multi-objective optimization. Inform Sci 448:164–186
de Melo VV (2014) Kaizen programming. In: GECCO ’14: Proceedings of the 2014 conference on Genetic and evolutionary computation, pp. 895–902
Murrietamendoza A, Ternisien L, Beuze B, Botez RM (2018) Aircraft vertical route optimization by beam search and initial search space reduction. J Aerosp Inf Syst 15(5):1–15
Neri F, Cotta C et al (2012) Handbook of memetic algorithms [M]. Springer Berlin Heidelberg
Pawlak TP, Krawiec K (2018) Competent geometric semantic genetic programming for symbolic regression and boolean function synthesis. Evolut Comput 26(2):1–36
Poli R, Langdon WB, Mcphee NF (2008) A field guide to genetic programming. lulu.com. [S.L.] : Lulu Press (lulu.com), 2008. 10(2), 229–230
Poli R, Mcphee NF (2014) Parsimony pressure made easy: Solving the problem of bloat in gp. Theory and Principled Methods for the Design of Metaheuristics pp. 181–204
Quade M, Gout J (2018) Abel M (2019) Glyph: Symbolic regression tools. Journal of Open Research Software 7(5923)
Shao L, Liu L, Li X (2014) Feature learning for image classification via multiobjective genetic programming. IEEE Trans Neural Netw Learn Syst 25(7):1359–1371
Sonccoálvarez JL, Muñoz DM, Ayalarincón M (2018) Opposition-based memetic algorithm and hybrid approach for sorting permutations by reversals. Evolut Comput 27:229–265
Trujillo L, Z-Flores E, Juarez P et al (2018) Local search is underused in genetic programming. In: Genetic and evolutionary computation XIV. Springer, Cham, pp 119–137
Wang B, Singh HK, Ray T (2015) A multi-objective genetic programming approach to uncover explicit and implicit equations from data. In: IEEE congress on evolutionary computation (CEC), pp. 1129–1136. IEEE
Wang P, Tang K, Weise T, Tsang EPK, Yao X (2014) Multiobjective genetic programming for maximizing ROC performance. Neurocomputing 125(3):102–118
Xu G, Ying L, Lohse N, Pessemier TD, Joseph W (2018) Energy and labor aware production scheduling for industrial demand response using adaptive multi-objective memetic algorithm. IEEE Trans Ind Inf 15:942–953
Xu J, Liu SC, Zhao C, Wu J, Lin WC, Yu PW (2019) An iterated local search and tabu search for two-parallel machine scheduling problem to minimize the maximum total completion time. J Inf Optim Sci 40(3):751–766
Z-Flores E, Trujillo L, Schütze O, Legrand P (2014) Evaluating the effects of local search in genetic programming. Springer International Publishing, Berlin
Zhang C Genetic programming for symbolic regression. Available at https://pdfs.semanticscholar.org/e5ee/ddd04b8344fd4f39a5836be686886c80df13.pdf
Zhang H, Dong Y, Xie X, Hu S, Weng S (2018) Pareto-dominance based adaptive multi-objective optimization for hydrothermal coordinated scheduling with environmental emission. Appl Soft Comput 69:270–287
Zhang K, Cai Y, Fu S, Zhang H (2019) Multiobjective memetic algorithm based on adaptive local search chains for vehicle routing problem with time windows. Evolut Intell 3:1–12
Zhao Z, He X, Feng L (2017) An improved multi-objective memetic algorithm for bi-objective permutation flow shop scheduling. In: International conference on service systems and service management
Zhong J, Liang F, Cai W, Ong YS (2018) Multifactorial genetic programming for symbolic regression problems. IEEE Trans Syst Man Cybern Syst 50:4492–4505
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
Authors declares that she has no conflict of interest.
Funding
This study was funded by National Natural Science Foundation of China (Grant Number 61902281 and Grant Number 61876089).
Human or animal participants
This article does not contain any studies with human participants or animals performed by any of the authors.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Liang, J., Xue, Y. Multi-Objective Memetic Algorithms with Tree-Based Genetic Programming and Local Search for Symbolic Regression. Neural Process Lett 53, 2197–2219 (2021). https://doi.org/10.1007/s11063-021-10497-8
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11063-021-10497-8