Abstract
Surrogate-based global optimization (SBGO) methods are widely used to deal with the computationally expensive black-box optimization problems. In order to reduce the computational source, multiple popular individual surrogates containing polynomial response surface (PRS), radial basis functions (RBF), kriging (KRG) and multiple derived ensemble models are constructed to replace the computationally expensive black-box functions. Moreover, a new multi-points infill strategy is presented to accelerate the optimization. New promising points are located by alternately using a hybrid and adaptive promising sampling (HAPS) method and a multi-start sequential quadratic programming (MSSQP) method. The proposed multi-surrogates and multi-points infill strategy-based global optimization (MSMPIGO) method is examined using eighteen unconstrained optimization problems, six nonlinear constrained engineering problems, and one airfoil design optimization problem. Three basic surrogate PRS, RBF, KRG-based global optimization methods using the similar multi-points infill strategy, PRSMPIGO, RBFMPIGO and KRGMPIGO are both considered as the comparative methods. In comparison with PRSMPIGO, RBFMPIGO, KRGMPIGO and three recently introduced SBGO methods, MSMPIGO shows superior search efficiency and strong robustness in locating the global optima.






Similar content being viewed by others
References
Bhosekar A, Ierapetritou M (2018) Advances in surrogate based modeling, feasibility analysis, and optimization: a review. Comput Chem Eng 108:250–267
Chatterjee T, Chakraborty S, Chowdhury R (2019) A critical review of surrogate assisted robust design optimization. Arch Comput Methods Eng 26(1):245–274
Forrester AIJ, Keane AJ (2009) Recent advances in surrogate-based optimization. Prog Aerosp Sci 45(1–3):50–79
Haftka RT, Villanueva D, Chaudhuri A (2016) Parallel surrogate-assisted global optimization with expensive functions—a survey. Struct Multidiscip Optim 54(1):3–13
Jin Y (2011) Surrogate-assisted evolutionary computation: recent advances and future challenges. Swarm Evol Comput 1(2):61–70
Wang LQ, Shan SQ, Wang GG (2004) Mode-pursuing sampling method for global optimization on expensive black-box functions. Eng Optim 36(4):419–438
Holmstrom K (2008) An adaptive radial basis algorithm (ARBF) for expensive black-box global optimization. J Global Optim 41(3):447–464
Dong H, Song B, Dong Z et al (2016) Multi-start space reduction (MSSR) surrogate-based global optimization method. Struct Multidiscip Optim 54(4):907–926
Xiang H, Li Y, Liao H et al (2017) An adaptive surrogate model based on support vector regression and its application to the optimization of railway wind barriers. Struct Multidiscip Optim 55(2):701–713
Zhou Q, Jiang P, Huang X et al (2018) A multi-objective robust optimization approach based on Gaussian process model. Struct Multidiscip Optim 57(1):213–233
Acar E, Rais-Rohani M (2009) Ensemble of metamodels with optimized weight factors. Struct Multidiscip Optim 37(3):279–294
Goel T, Haftka RT, Shyy W et al (2007) Ensemble of surrogates. Struct Multidiscip Optim 33(3):199–216
Lee Y, Choi DH (2014) Pointwise ensemble of meta-models using v nearest points cross-validation. Struct Multidiscip Optim 50(3):383–394
Gu J, Li GY, Dong Z (2012) Hybrid and adaptive meta-model-based global optimization. Eng Optim 44(1):87–104
Viana FAC, Haftka RT, Watson LT (2013) Efficient global optimization algorithm assisted by multiple surrogate techniques. J Global Optim 56(2):669–689
Ye P, Pan G (2017) Global optimization method using ensemble of metamodels based on fuzzy clustering for design space reduction. Eng Comput 33(3):573–585
Ye P, Pan G, Dong Z (2018) Ensemble of surrogate based global optimization methods using hierarchical design space reduction. Struct Multidiscip Optim 58(2):537–554
Habib A, Singh HK, Ray T (2018) A multiple surrogate assisted evolutionary algorithm for optimization involving iterative solvers. Eng Optim 50(9):1625–1644
Zhang N, Wang P, Dong H et al (2020) Shape optimization for blended-wing-body underwater glider using an advanced multi-surrogate-based high-dimensional model representation method. Eng Optim. https://doi.org/10.1080/0305215X.2019.1694674
Box GEP, Hunter WG, Hunter JS (1978) Statistics for experimenters. Wiley-Interscience Press, New York
Wang GG, Dong Z, Aitchison P (2001) Adaptive response surface method—a global optimization scheme for approximation-based design problems. Eng Optim 33(6):707–733
Ye P, Pan G (2017) Global optimization method using adaptive and parallel ensemble of surrogates for engineering design optimization. Optimization 66(7):1135–1155
Dyn N, Levin D, Rippa S (1986) Numerical procedures for surface fitting of scattered data by radial basis functions. SIAM J Sci Stat Comput 7(2):639–659
Hussain MF, Barton RR, Joshi SB (2002) Metamodeling: radial basis functions, versus polynomials. Eur J Oper Res 138(1):142–154
Lophaven SN, Nielsen HB, Søndergaard J (2002) DACE—a MATLAB kriging toolbox. http://www2.imm.dtu.dk/~hbn/dace/
Martin JD, Simpson TW (2005) Use of Kriging models to approximate deterministic computer models. AIAA J 43(4):853–863
Kitayama S, Arakawa M, Yamazaki K (2011) Sequential approximate optimization using radial basis function network for engineering optimization. Optim Eng 12(4):535–557
Sóbester A, Leary SJ, Keane AJ (2005) On the design of optimization strategies based on global response surface approximation models. J Global Optim 33(1):31–59
Ao YY, Chi HQ (2010) An adaptive differential evolution algorithm to solve constrained optimization problems in engineering design. Engineering 2:65–77
Coello CAC (2002) Theoretical and numerical constraint-handling techniques used with evolutionary algorithms: a survey of the state of the art. Comput Methods Appl Mech Eng 191(11):1245–1287
Garg H (2014) Solving structural engineering design optimization problems using an artificial bee colony algorithm. J Ind Manag Optim 10(3):777–794
Himmelblau DM (1972) Applied nonlinear programming. McGraw-Hill Book Company, New York
Kulfan BM (2008) Universal parametric geometry representation method. J Aircr 45(1):142–158
Funding
This research is supported by National Natural Science Foundation of China (Grant No. 61803306), China Postdoctoral Science Foundation (Grant No. 2019M660264), Fundamental Research Funds for the Central Universities (Grant No. 3102021bzb003).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix: List of test optimization problems
Appendix: List of test optimization problems
-
(1)
Six-hump camel-back function (SC) with n = 2
$$f\left( {\varvec{x}} \right) = 4x_{1}^{2} - 2.1x_{1}^{4} + {{x_{1}^{6} } \mathord{\left/ {\vphantom {{x_{1}^{6} } 3}} \right. \kern-\nulldelimiterspace} 3} + x_{1} x_{2} - 4x_{2}^{2} + 4x_{2}^{4} {\kern 1pt} {\kern 1pt} .$$(19) -
(2)
Branin function (BR) with n = 2
$$\begin{aligned} f\left( {\varvec{x}} \right) =\,& \left[ {x_{2} - 5.1\left( {{{x_{1} } \mathord{\left/ {\vphantom {{x_{1} } {2\pi }}} \right. \kern-\nulldelimiterspace} {2\pi }}} \right)^{2} + \left( {{5 \mathord{\left/ {\vphantom {5 \pi }} \right. \kern-\nulldelimiterspace} \pi }} \right)x_{1} - 6} \right]{\kern 1pt}^{2}\\ & + {10}\left[ {1 - \left( {{1 \mathord{\left/ {\vphantom {1 {8\pi }}} \right. \kern-\nulldelimiterspace} {8\pi }}} \right)} \right]{\text{cos}}x_{1} + 10. \end{aligned}$$(20) -
(3)
Generalized polynomial function (GF) with n = 2
$$\begin{aligned} f\left( {\varvec{x}} \right) = \,& \left( {1.5 - x_{1} \left( {1 - x_{2} } \right)} \right)^{2} + \left( {2.25 - x_{1} \left( {1 - x_{2}^{2} } \right)} \right)^{2}\\ & + \left( {2.625 - x_{1} \left( {1 - x_{2}^{3} } \right)} \right)^{2} . \end{aligned}$$(21) -
(4)
Goldstein and price function (GP) with n = 2
$$\eqalign{ \left( x \right) & = \left[ {1 + {{\left( {{x_1} + {x_2} + 1} \right)}^2}\left( {19 - 14{x_1} + 3x_1^2 - 14{x_2} + 6{x_1}{x_2} + 3x_2^2} \right)} \right] \cr & \quad \times \left[ {30 + {{\left( {2{x_1} - 3{x_2}} \right)}^2}\left( {18 - 32{x_1} + 12x_1^2 + 48{x_2} - 36{x_1}{x_2} + 27x_2^2} \right)} \right] \cr}$$(22) -
(5)
Shubert function (SE) with n = 2
$$f\left( {\varvec{x}} \right) = {\kern 1pt} \left( {\sum\limits_{i = 1}^{5} {i{\text{cos}}\left( {\left( {i + 1} \right)x_{1} + i} \right)} } \right)\left( {\sum\limits_{i = 1}^{5} {i{\text{cos}}\left( {\left( {i + 1} \right)x_{2} + i} \right)} } \right).$$(23) -
(6)
Banana function (BA) with n = 2
$$f\left( {\varvec{x}} \right) = 100\left( {x_{2} - x_{1}^{2} } \right){\kern 1pt} {\kern 1pt}^{2} + \left( {1 - x_{1} } \right){\kern 1pt}^{2} .$$(24) -
(7)
Himmelblau function (HM) with n = 2
$$f\left( {\varvec{x}} \right) = \left( {x_{1}^{2} + x_{2} - 11} \right)^{2} + \left( {x_{1} + x_{2}^{2} - 7} \right)^{2} .$$(25) -
(8)
Cross-IN-TRAY function (CT) with n = 2
$$f\left( {\varvec{x}} \right) = - 0.0001\left( {\left| {\sin \left( {x_{1} } \right)\sin \left( {x_{2} } \right)\exp \left( {\left| {100 - \frac{{\sqrt {x_{1}^{2} + x_{2}^{2} } }}{\pi }} \right|} \right)} \right| + 1} \right)^{0.1} .$$(26) -
(9)
Zakharov function (ZK) with n = 2
$$f\left( {\varvec{x}} \right) = {\kern 1pt} \sum\limits_{i = 1}^{2} {x_{i}^{2} } + \left( {\sum\limits_{i = 1}^{2} {0.5ix_{i} } } \right)^{2} + \left( {\sum\limits_{i = 1}^{2} {0.5ix_{i} } } \right)^{4} .$$(27) -
(10)
Trid function (TR6 and TR10) with n = 6, 10
$$f\left( {\varvec{x}} \right) = {\kern 1pt} \sum\limits_{i = 1}^{n} {\left( {x_{i} - 1} \right)} {\kern 1pt}^{2} - \sum\limits_{i = 2}^{n} {x_{i} x_{i - 1} } .$$(28) -
(11)
Paviani function (PF) with n = 10
$$f\left( {\varvec{x}} \right) = \sum\limits_{i = 1}^{n} {\left[ {\ln^{2} \left( {x_{i} - 2} \right) + \ln^{2} \left( {10 - x_{i} } \right)} \right]} - \left( {\prod\limits_{i = 1}^{n} {x_{i} } } \right)^{0.2} .$$(29) -
(12)
Sum squares function (SF12 and SF15) with n = 12, 15
$$f\left( {\varvec{x}} \right) = {\kern 1pt} \sum\limits_{i = 1}^{n} {ix_{i}^{2} } {\kern 1pt} .$$(30) -
(13)
Ellipsoid function (ED12 and ED) with n = 12, 15
$$f\left( {\varvec{x}} \right) = {\kern 1pt} \sum\limits_{i = 1}^{n} {\sum\limits_{j = 1}^{i} {x_{j}^{2} } } .$$(31) -
(14)
A function of 16 variables (F16) with n = 16
$$f\left( {\varvec{x}} \right) = {\kern 1pt} {\kern 1pt} \sum\limits_{i = 1}^{n} {\sum\limits_{j = 1}^{n} {\alpha_{ij} \left( {x_{i}^{2} + x_{i} + 1} \right)} } \left( {x_{j}^{2} + x_{j} + 1} \right),$$(32)$${a_{ij}}_{\left( {row1 - 8} \right)} = \left[ \begin{array}{llllllllllllllll} 1 & 0 & 0 & 1 & 0 & 0 & 1 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 \\ 0 & 1 & 1 & 0 & 0 & 0 & 1 & 0 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 & 0 & 0 & 1 & 0 & 1 & 1 & 0 & 0 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 & 0 & 0 & 1 & 0 & 0 & 0 & 1 & 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 1 & 1 & 0 & 0 & 0 & 1 & 0 & 1 & 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 & 0 & 1 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 0 & 0 & 1 & 0 & 1 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 1 & 0 & 0 & 0 & 0 & 1 & 0 \\ \end{array} \right]$$$${a_{ij}}_{\left( {row 9 - 16} \right)} = \left[ \begin{array}{llllllllllllllll} 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 0 & 1 & 0 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 0 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 1 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 1 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 \\ \end{array} \right]$$ -
(15)
Sphere function (SP) with n = 20
$$f\left( {\varvec{x}} \right) = {\kern 1pt} \sum\limits_{i = 1}^{n} {x_{i}^{2} } {\kern 1pt} .$$(33) -
(16)
Tension/compression spring design (TSD) with n = 3
This problem aims to minimize the weight of a tension/compression subject to constraints on minimum deflection, shear stress and surge frequency, limits on outside diameter and side constraints.
-
(17)
I-beam design (IBD) with n = 4
This problem aims to minimize the vertical deflection of an I-beam and meanwhile satisfies the cross-section area and stress constraints under given loads.
-
(18)
Welded beam design (WBD) with n = 4
This problem aims to minimize the cost and meanwhile meets the constraints on shear stress, bending stress in the beam, buckling load on the bar, end deflection of the beam and side constraints.
-
(19)
Pressure vessel design (PVD) with n = 4
This problem is designed to minimize the fabrication cost covering the cost of materials, forming, and welding of the pressure vessel. Four design variables are thickness of the pressure vessel, thickness of the head, inner radius of the pressure vessel, and length of the vessel.
-
(20)
Himmelblau’ s nonlinear optimization problem (HIM) with n = 5
This problem has five design variables, six nonlinear inequality constraints and ten boundary conditions.
-
(21)
Speed reducer design (SRD) with n = 7
This problem is designed to minimize the total weight of the speed reducer. It has eleven constraints involving the limits on the bending stress of the gear teeth, surface stress and transverse deflections of shafts.
Rights and permissions
About this article
Cite this article
Ye, P., Pan, G. Multi-surrogates and multi-points infill strategy-based global optimization method. Engineering with Computers 39, 1617–1636 (2023). https://doi.org/10.1007/s00366-021-01557-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00366-021-01557-7