Abstract
We propose a framework of memetic optimization using variable global and local surrogate-models for optimization of expensive functions. The framework employs the trust-region approach but replaces the quadratic models with the more general RBF ones. It makes an extensive use of accuracy assessment to select the models used and to improve them if necessary. It also employs several efficient and stable numerical methods to improve its performance. Rigorous performance analysis shows the proposed framework significantly outperforms several existing surrogate-assisted evolutionary algorithms.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Alexandrov NM, Dennis JE, Lewis RM, Torczon V (1998) A trust region framework for managing use of approximation models in optimization. Struct Optim 15(1): 16–23
Barthelemy JFM, Haftka RT (1993) Approximation concepts for optimum structural design—a review. Struct Optim 5: 129–144
Bellman RE (1961) Adaptive control processes:a guided tour. Princeton University Press, Princeton
Booker AJ, Dennis JE, Frank PD, Serafini DB, Torczon V, Trosset MW (1999) A rigorous framework for optimization of expensive functions by surrogates. Struct Optim 17(1): 1–13
Box GEP, Draper NR (1987) Empirical model building and response surface. Wiley, New York
Chen VP, Burton RR, Meckesheimer M, Tsui K-T (2006) A review on design modeling and applications of computer experiments. IEE Trans 38(4): 273–291
Chiba K, Obayashi S, Nakahashi K (2005) High-fidelity multidisciplinary design optimization of aerostructural wing shape for regional jet. In: Proceedings of the 23rd AIAA applied aerodynamics conference, number AIAA-2005-5080, American Institute for Aeronautics and Astronautics (AIAA), Reston
Chipperfield A, Fleming P, Pohlheim H, Fonseca C (1997) Genetic algorithm TOOLBOX for use with MATLAB, version 1.2. Department of Automatic Control and Systems Engineering, University of Sheffield
Conn AR, Scheinberg K, Toint PL (1997) Recent progress in unconstrained nonlinear optimization without derivatives. Math Program 79: 397–414
Conover WJ (1980) Practical nonparametric statistics, 2nd edn. Wiley, New York
deSouza PS, Talukdar SN (1991) Genetic algorithms in asynchronous teams. In: Belew RK, Booker LB (eds) Proceedings of the fourth international conference on genetic algorithms, San Mateo, pp 392–397
Demmel JW (1987) The geometry of ill-conditioning. Comput J 3(2): 201–229
Dennis JE Jr, Torczon V (1997) Managing approximation models in optimization. In: Alexandrov NM, Hussaini MY (eds) Multidisciplinary design optimization: state of the art. SIAM, Philadelphia, pp 330–347
Dixon LCW, Szegö GP (1978) The global optimization problem: An introduction. In: Dixon LCW, Szegö GP (eds) Towards global optimisation 2. North-Holland Publishing Company, Amsterdam, pp 1–15
Drela M, Youngren H (2001) XFOIL 6.9 User Primer. Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge
Eby D, Averill RC, Punch III, WF, Goodman ED (1998) Evaluation of injection island GA performance on flywheel design optimization. In: Proceedings of the third conference on adaptive computing in design and manufacturing, Springer, Plymouth, pp 121–136
El-Betalgy MA, Keane AJ (2001) Evolutionary optimization for computationally expensive problems using Gaussian processes. In: Hamid Arbania (ed) Proceedings of the international conference on artificial intelligence–IC-AI’2001, CSREA Press, pp 708–714
Filippone A (2006) Flight performance of fixed and rotary wing aircraft, 1st edn. Elsevier, Amsterdam
Frank PD (1995) Global modeling for optimization. SIAM SIAG/OPT Views-and-News 7: 9–12
Franke R (1982) Scattered data interpolation: Tests of some methods. Math Comput 38(157): 181–200
Gaspar-Cunha A, Vieira A (2005) A multi-objective evolutionary algorithm using neural networks to approximate fitness evaluations. Int J Comput Syst Signals 6(1): 18–36
Giannakoglou KC (2002) Design of optimal aerodynamic shapes using stochastic optimization methods and computational intelligence. Int Rev J Prog Aerospace Sci 38(1): 43–76
Gutmann HM (2001) A radial basis function method for global optimization. J Glob Optim 19: 201–227
Hardy RL (1971) Multiquadric equations of topography and other irregular surfaces. J Geophys Res 76: 1905–1915
Hart WE, Belew RK (1995) Optimization with genetic algorithm hybrids that use local search. In: Belew RK, Mitchell M (eds) Adaptive Individuals in Evolving Populations: Models and Algorithms, Santa Fe Institute Studies in the Sciences of Complexity, chapter 27. Addison-Wesley, Reading, pp 483–496
Holst TL, Pulliam TH (2001) Aerodynamic shape optimization using a real-number-encoded genetic algorithm. Technical Report 2001-2473, AIAA
Jin Y (2005) A comprehensive survey of fitness approximation in evolutionary computation. J Soft Comput 9(1): 3–12
Jin R, Chen W, Simpson TW (2001) Comparative studies of metamodeling techniques under multiple modeling criteria. Struct Optim 23(1): 1–13
Jin Y, Olhofer M, Sendhoff B (2002) A framework for evolutionary optimization with approximate fitness functions. IEEE Trans Evol Comput 6(5): 481–494
Johnson ME, Moore LM, Ylvisaker D (1990) Minimax and maximin distance designs. J Stat Plan Inference 26(2): 131–148
Kansa EJ, Hon Y-C (2000) Circumventing the ill-conditioning problem with multiquadric radial basis functions: applications to elliptic partial differential equations. Comput Math Appl 39(7): 123–137
Karakasis MK, Giannakoglou KC (2004) On the use of surrogate evaluation models in multi-objective evolutionary algorithms. In: Proceedings of the European conference on computational methods in applied sciences and engineering–ECCOMAS 2004
Kim H-S, Cho S-B (2001) An efficient genetic algorithm with less fitness evaluation by clustering. In: Proceedings of 2001 IEEE conference on evolutionary computation, IEEE, pp 887–894
Kleijnen JPC, Sargent RG (2000) A methodology for fitting and validating metamodels in simulation. Eur J Oper Res 120(1): 14–29
Kruskal WH, Wallis WA (1952) Use of ranks on one-criterion variance analysis. J Am Stat Assoc 47: 583–621
Liang K-H, Yao X, Newton C (2000) Evolutionary search of approximated N-dimensional landscapes. Int J Knowl Based Intell Eng Syst 4(3): 172–183
Linhart H, Zucchini W (1986) Model selection. Wiley series in probability and mathematical statistics. Wiley-Interscience Publication, New York
Mann HB, Whitney DR (1947) On a test whether one of two variables is stochastically larger than the other. Ann Math Stat 18: 50–60
De Marchi S (2003) On optimal center locations for radial basis function interpolation: computational aspects. Rendiconti del Semin Matematico. 61(3): 343–358
Martin JD, Simpson TW (2005) Use of kriging models to approximate deterministic computer models. AIAA J 43(4): 853–863
McKay MD, Beckman RJ, Conover WJ (1979) A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21(2): 239–245
Meckesheimer M, Booker AJ, Burton RR, Simpson TW (1979) Computationally inexpensive metamodel assessment strategies. AIAA J 40(10): 2053–2060
Micceri T (1989) The unicorn, the normal curve and other improbable creatures. Psychol Bull 105(1): 156–166
Micchelli CA (1986) Interpolation of scattered data: distance matrices and conditionally positive definite functions:. Constructive Approximation 2: 11–22
Mitchell TJ, Morris MD (1992) Bayesian design and analysis of computer experiments: two examples. Stat Sin 2: 359–379
Mühlenbein H, Schlierkamp-Voosen D (1993) Predictive models for the breeder genetic algorithm I: continuous parameter optimization. Evol Comput 1(1): 25–49
Myers RH, Montgomery DC (1995) Response surface methodology: process and product optimization using designed experiments. Wiley, New York
Obayashi S (2000) Airfoil shape optimization for evolutionary computation. In: Genetic algorithms for optimization in aeronautics and turbomachinery, VKI Lecture Series 2000-07. Rhode Saint Genese, Belgium and Von Karman Institute for Fluid Dynamics
Ong Y-S, Keane AJ (2004) Meta-Lamarckian learning in memetic algorithm. IEEE Trans Evol Comput 8(2): 99–110
Ong Y-S, Nair PB, Keane AJ (2003) Evolutionary optimization of computationally expensive problems via surrogate modeling. Am Inst Aeron Astron J 41(4): 687–696
Ong Y-S, Nair PB, Lum KY (2006a) Max-min surrogate-assisted evolutionary algorithm for robust aerodynamic design. IEEE Trans Evol Comput 10(4): 392–404
Ong Y-S, Zong Z, Lim D (2006b) Curse and blessing of uncertainty in evolutionary algorithm using approximation. In: Proceedings of the IEEE congress on evolutionary computation–WCCI 2006, IEEE, pp 2928–2935
Oyama A, Obayashi S, Nakahashi K (2000) Real-coded adaptive range genetic algorithm and its application to aerodynamic design. Int J Jpn Soc Mech Eng 43(2): 124–129
Oyama A, Obayashi S, Nakahashi T (2002) Real-coded adaptive range genetic algorithm applied to transonic wing optimization. In: Marc Schoenauer (ed) The 6th international conference on parallel problem solving from nature–PPSN VI, Springer, Berlin, pp 712–721
Poggio T, Girosi F(1989) A theory of networks for approximation and learning. A.I.Memo 1140, Massachusetts Institute of Technology, Artificial Intelligence Laboratory and Center for Biological Information Processing
Poloni C, Giurgevich A, Onseti L, Pediroda V (2000) Hybridization of a multi-objective genetic algorithm, a neural network and a classical optimizer for a complex design problem in fluid dynamics. Comput Methods Appl Mech Eng 186: 403–420
Quagliarella D, Vicini A (1997) Coupling genetic algorithms and gradient based optimization techniques. In: Quagliarella D, Periaux J, Poloni C, Winter G (eds) Genetic algorithms in engineering and computer science, Chap 14. Wiley, New York, pp 288–309
Ratle A (1998) Accelerating the convergence of evolutionary algorithms by fitness landscape approximations. In: Eiben AE, Bäck T, Schoenauer M, Schwefel HP (eds) Proceedings of the 5th international conference on parallel problem solving from nature–PPSN V. Lecture notes in computer science, vol 1498, Springer, Berlin, pp 87–96
Ratle A (1999) Optimal sampling strategies for learning a fitness model. In: The 1999 IEEE congress on evolutionary computation–CEC 1999, IEEE, New York, pp 2078–2085
Renderes J-M, Bersini H (1994) Hybridizing genetic algorithms with hill-climbing methods for global optimization: two possible ways. In: Sebald AV, Fogel LJ (eds) Proceedings of the third annual conference on evolutionary programming, World Scientific, pp 312–317
Renderes J-M, Flasse SP (1996) Hybrid methods using genetic algorithms for global optimization. IEEE Trans Syst Man Cybern B 26(2): 243–258
Rippa S (1999) An algorithm for selecting a good value for the parameter c in radial basis function interpolation. Adv Comput Math 11(2–3): 193–210
Rodríguez JF, Renaud JE, Watson LT (1998) Convergence of trust region augmented Lagrangian methods using variable fidelity approximation data. Struct Optim 15(3/4): 141–156
Schaback R (1993) Comparison of radial basis function interpolants. In: Jetter K, Utreras FI (eds) Multivariate approximations: from CAGD to wavelets approximations and decompositions. World Scientific, , pp 293–305
Schaback R (1996) Multivariate interpolation and approximation by translates of a basis function. In: Chui CK, Schumaker LL (eds) Approximation theory VIII. Vanderbilt University Press, , pp 491–514
Sefrioui M, Périaux J (2000) A hierarchical genetic algorithm using multiple models for optimization. In: Marc Schoenauer, Kalyanmoy Deb, Günter Rudolph, Xin Yao, Evelyne Lutton, Juan J. Merelo Guervós, and Hans-Paul Schwefel (eds) Proceedings of the 6th international conference on parallel problem solving from nature–PPSN VI, number 1917 in Lecture notes in computer science, Springer, pp 879–888
Shapiro SS, Wilk MB (1965) An analysis of variance test for normality (complete samples). Biometrika 52: 591–611
Shapiro SS, Wilk MB (1968) Approximation for the null distribution of the W statistic. Technometrics 10: 861–866
Sheskin DJ (2007) Handbook of parametric and nonparametric statistical procedures, 4th edn. Chapman and Hall, Boca Raton
Simpson TW, Lin DKJ, Chen W (2001a) Sampling strategies for computer experiments: design and analysis. Int J Reliab Appl 2(3): 209–240
Simpson TW, Poplinski JD, Koch PN, Allen JK (2001b) Metamodels for computer-based engineering design: survey and recommendations. Eng Comput 17: 129–150
Simpson TW, Booker AJ, Ghosh D, Giunta AA, Koch PN, Yang R-J (2004) Approximation methods in multidisciplinary analysis and optimization: a panel discussion. Struct Multidiscip Optim 27: 302–313
Smith RE, Dike BA, Stegmann SA (1995) Fitness inheritance in genetic algorithms. In: George KM (ed), Proceedings of the 1995 ACM symposium on applied computing–SAC 95, ACM Press, New York, pp 345–350
Sobieckzy H (1999) Parametric airfoils and wings. In: Kozo Fujii, Dulikravich GS, Susumu Takanashi (eds) Recent development of aerodynamic design methodologies : inverse design and optimization. Notes on numerical fluid mechanics, vol 68, Vieweg, Braunschweig; Wiesbaden, pp 71–88
Sobieszczansk-Sobieski J, Haftka RT (1997) Multidisciplinary aerospace design optimization: Survey of recent developments. Struct Optim 14(1): 1–23
Stone CJ (1982) Optimal global rates of convergence for nonparametric regression. Ann Stat 10(4): 1040–1053
Stone M (1974) Cross-validatory choice and assessment of statistical predictors. J R Stat Soc Ser B 36(2): 111–147
Tenne Y, Armfield SW (2007) A memetic algorithm using a trust-region derivative-free optimization with quadratic modelling for optimization of expensive and noisy black-box functions. In: Yang S, Ong Y-S, Jin Y (eds) Evolutionary computation in dynamic and uncertain environments, vol 51 of studies in computational intelligence. Springer, Berlin, pp 389–415
Tenne Y, Armfield SW (2008a) A versatile surrogate-assisted memetic algorithm for optimization of computationally expensive functions and its engineering applications. In: Yang A, Shan Y, Bui LT (eds) Success in evolutionary computation, vol 92/2008. Springer, Berlin, pp 43–72
Tenne Y, Armfield SW (2008b) A memetic algorithm assisted by an adaptive topology artificial neural network and variable local models for expensive optimization problems. In: Advances in evolutionary algorithms (accepted)
Tenne Y, Obayashi S, Armfield SW (2007) Airfoil shape optimization using an algorithm for minimization of expensive and discontinuous black-box functions. In: Proceedings of the AIAA InfoTec 2007, vol 2, American Institute for Aeronautics and Astronautics (AIAA), AIAA-2007-2874, pp 1691–1710
Toint PL (1978) Some numerical results using a sparse matrix updating formula in unconstrained optimization. Math Comput 32(143): 839–851
Torczon V, Trosset MW (1998) Using approximations to accelerate engineering design optimization. In: Proceedings of the 7th AIAA/USAF/NASA/ISSMO symposium on multidisciplinary analysis and optimization, American Institute for Aeronautics and Astronautics (AIAA), Reston, pp 738–748
Törn A (1973) Global optimization as a combination of global and local search. In: Proceedings of computer simulation versus analytical solutions for business and economic models, number 17 in BAS Business Administration Studies, School of Business Administration, Gothenburg, pp 191–206
Törn A, Žilinskas A (1989) Global optimization. Number 350 in lecture notes in computer science. Springer, Berlin
Törn A, Ali MM, Viitanen S (1999) Stochastic global optimization: problems classes and solution techniques. J Glob Optim 14: 437–447
Trosset MW, Alexandrov NM, Watson LT (2003) New methods for robust design using computer simulation. In: Proceedings of the section on physical and engineering sciences, American Statistical Association, pp 191–206
Yao X, Lin G, Liu Y (1997) An analysis of evolutionary algorithms based on neighbourhood and step sizes. In: Angeline PJ, Reynolds RG, McDonnel JR, Eberhart R (eds) Evolutionary programming VI: Proceedings of the sixth annual conference on evolutionary programming, Lecture notes on computer science, vol 1213, Springer, Berlin, pp 297–307
Yen J, Liao JC, Lee B, Randolph D (1998) A hybrid approach to modeling metabolic systems using a genetic algorithm and simplex method. IEEE Trans Systems Man Cybern B 28(2): 173–191
Yun Y, Gen M, Seo S (2003) Various hybrid methods based on genetic algorithm with fuzzy logic controller. J Intell Manuf 14: 401–419
Zhou Z, Ong Y-S, Lim M, Lee B (2007a) Memetic algorithms using multi-surrogates for computationally expensive optimization problems. J Soft Comput 11(10): 957–971
Zhou Z, Ong Y-S, Nair PB, Keane AJ, Lum KY (2007b) Combining global and local surrogate models to accelerate evolutionary optimization. IEEE Trans Syst Man Cybern C 37(1): 66–76
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Tenne, Y., Armfield, S.W. A framework for memetic optimization using variable global and local surrogate models. Soft Comput 13, 781–793 (2009). https://doi.org/10.1007/s00500-008-0348-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-008-0348-2