Skip to main content
Springer Nature Link
Log in

Variable-fidelity hypervolume-based expected improvement criteria for multi-objective efficient global optimization of expensive functions

  • Original Article
  • Published:
Engineering with Computers Aims and scope Submit manuscript

Abstract

Variable-fidelity surrogate-based efficient global optimization (EGO) method with the ability to adaptively select low-fidelity (LF) and high-fidelity (HF) infill point has emerged as an alternative to further save the computational cost of the single-fidelity EGO method. However, in terms of the variable-fidelity surrogate-assisted multi-objective optimization methods, existing methods rely on empirical parameters or are unable to adaptively select LF/HF sample in the optimal search process. In this paper, two variable-fidelity hypervolume-based expected improvement criteria with analytic expressions for variable-fidelity multi-objective EGO method are proposed. The first criterion relies on the concept of variable-fidelity expected improvement matrix (VFEIM) and is obtained by aggregating the VFEIM using a simplified hypervolume-based aggregation scheme. The second criterion termed as VFEMHVI is derived analytically based on a modified hypervolume definition. Both criteria can adaptively select new LF/HF samples in the iterative optimal search process to update the variable-fidelity models towards the HF Pareto front, distinguishing the proposed methods to the rests in the open literature. The constrained versions of the two criteria are also derived for problems with constraints. The effectiveness and efficiency of the proposed methods are verified and validated over analytic problems and demonstrated by two engineering problems including aerodynamic shape optimizations of the NACA0012 and RAE2822 airfoils. The results show that the VFEMHVI combined with the normalization-based strategy to define the reference point is the most efficient one over the compared methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

Availability of data and material

The data of the numerical experiments will be provided after the paper being published.

Code availability

The Appendix provides the MATLAB codes to calculate the proposed criteria. The executable code of the proposed method will be available online after the paper being published.

References

  1. Sacks J, Welch WJ, Mitchell TJ, Wynn HP (1989) Design and analysis of computer experiments. Stat Sci 4:409–423. https://doi.org/10.1214/ss/1177012413

    Article  MathSciNet  MATH  Google Scholar 

  2. Rasmussen CE (2004) Gaussian processes in machine learning. Lect Notes Comput Sci (including Subser Lect Notes Artif Intell Lect Notes Bioinformatics). https://doi.org/10.1007/978-3-540-28650-9_4

    Article  MATH  Google Scholar 

  3. Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black-box functions. J Glob Optim 13:455–492. https://doi.org/10.1023/A:1008306431147

    Article  MathSciNet  MATH  Google Scholar 

  4. Song P, Sun J, Wang K (2014) Axial flow compressor blade optimization through flexible shape tuning by means of cooperative co-evolution algorithm and adaptive surrogate model. Proc Inst Mech Eng Part A J Power Energy 228:782–798. https://doi.org/10.1177/0957650914541647

    Article  Google Scholar 

  5. Venturelli G, Benini E (2016) Kriging-assisted design optimization of S-shape supersonic compressor cascades. Aerosp Sci Technol 58:275–297. https://doi.org/10.1016/j.ast.2016.08.021

    Article  Google Scholar 

  6. Song P, Sun J, Wang K, He Z (2011) Development of an optimization design method for turbomachinery by incorporating the Cooperative Coevolution Genetic Algorithm and adaptive approximate model. In: Proceedings of the ASME Turbo Expo. pp 1139–1153 https://doi.org/10.1115/GT2011-45411

  7. Song L, Guo Z, Li J, Feng Z (2016) Research on metamodel-based global design optimization and data mining methods. J Eng Gas Turbines Power 138:92604–92614. https://doi.org/10.1115/1.4032653

    Article  Google Scholar 

  8. Forrester AIJ, Keane AJ (2009) Recent advances in surrogate-based optimization. Prog Aerosp Sci 45:50–79. https://doi.org/10.1016/j.paerosci.2008.11.001

    Article  Google Scholar 

  9. Bhosekar A, Ierapetritou M (2018) Advances in surrogate-based modeling, feasibility analysis, and optimization: a review. Comput Chem Eng 108:250–267. https://doi.org/10.1016/j.compchemeng.2017.09.017

    Article  Google Scholar 

  10. Park C, Haftka RT, Kim NH (2017) Remarks on multi-fidelity surrogates. Struct Multidiscip Optim 55:1029–1050. https://doi.org/10.1007/s00158-016-1550-y

    Article  MathSciNet  Google Scholar 

  11. Giselle Fernández-Godino M, Park C, Kim NH, Haftka RT (2019) Issues in deciding whether to use multifidelity surrogates. AIAA J. https://doi.org/10.2514/1.J057750

    Article  Google Scholar 

  12. Knowles J (2006) ParEGO: A hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems. IEEE Trans Evol Comput 10:50–66. https://doi.org/10.1109/TEVC.2005.851274

    Article  Google Scholar 

  13. He Y, Sun J, Song P et al (2020) Preference-driven Kriging-based multiobjective optimization method with a novel multipoint infill criterion and application to airfoil shape design. Aerosp Sci Technol. https://doi.org/10.1016/j.ast.2019.105555

    Article  Google Scholar 

  14. Chen S, Jiang Z, Yang S, Chen W (2017) Multimodel fusion-based sequential optimization. AIAA J 55:241–254. https://doi.org/10.2514/1.J054729

    Article  Google Scholar 

  15. Kennedy MC, O’Hagan A (2000) Predicting the output from a complex computer code when fast approximations are available. Biometrika. https://doi.org/10.1093/biomet/87.1.1

    Article  MathSciNet  MATH  Google Scholar 

  16. Forrester AIJ, Sóbester A, Keane AJ (2007) Multi-fidelity optimization via surrogate modelling. Proc R Soc A Math Phys Eng Sci. https://doi.org/10.1098/rspa.2007.1900

    Article  MATH  Google Scholar 

  17. Huang D, Allen TT, Notz WI, Miller RA (2006) Sequential kriging optimization using multiple-fidelity evaluations. Struct Multidiscip Optim 32:369–382. https://doi.org/10.1007/s00158-005-0587-0

    Article  Google Scholar 

  18. Xiong Y, Chen W, Tsui KL (2008) A new variable-fidelity optimization framework based on model fusion and objective-oriented sequential sampling. J Mech Des Trans ASME. https://doi.org/10.1115/1.2976449

    Article  Google Scholar 

  19. Zhang Y, Han Z-H, Zhang K-S (2018) Variable-fidelity expected improvement method for efficient global optimization of expensive functions. Struct Multidiscip Optim 58:1431–1451. https://doi.org/10.1007/s00158-018-1971-x

    Article  MathSciNet  Google Scholar 

  20. Liu Y, Chen S, Wang F, Xiong F (2018) Sequential optimization using multi-level cokriging and extended expected improvement criterion. Struct Multidiscip Optim 58:1155–1173. https://doi.org/10.1007/s00158-018-1959-6

    Article  MathSciNet  Google Scholar 

  21. Jiang P, Cheng J, Zhou Q et al (2019) Variable-fidelity lower confidence bounding approach for engineering optimization problems with expensive simulations. AIAA J. https://doi.org/10.2514/1.j058283

    Article  Google Scholar 

  22. Hao P, Feng S, Li Y et al (2020) Adaptive infill sampling criterion for multi-fidelity gradient-enhanced kriging model. Struct Multidiscip Optim. https://doi.org/10.1007/s00158-020-02493-8

    Article  MathSciNet  Google Scholar 

  23. Forrester A, Sobester A, Keane A (2008) Engineering design via surrogate modelling: a practical guide. Wiley, Hoboken. https://doi.org/10.2514/4.479557

    Book  Google Scholar 

  24. Emmerich MTM, Deutz AH, Klinkenberg JW (2011) Hypervolume-based expected improvement: Monotonicity properties and exact computation. In: 2011 IEEE Congress of evolutionary computation, CEC 2011. pp 2147–2154 /https://doi.org/10.1109/CEC.2011.5949880

  25. Couckuyt I, Deschrijver D, Dhaene T (2014) Fast calculation of multiobjective probability of improvement and expected improvement criteria for Pareto optimization. J Glob Optim 60:575–594. https://doi.org/10.1007/s10898-013-0118-2

    Article  MathSciNet  MATH  Google Scholar 

  26. Bautista DC (2009) A Sequential design for approximating the pareto front using the expected pareto improvement function. Ph.D. dissertation, Ohio State University, Ohio, USA

  27. Svenson J, Santner T (2016) Multiobjective optimization of expensive-to-evaluate deterministic computer simulator models. Comput Stat Data Anal 94:250–264. https://doi.org/10.1016/j.csda.2015.08.011

    Article  MathSciNet  MATH  Google Scholar 

  28. Namura N, Shimoyama K, Obayashi S (2017) Expected improvement of penalty-based boundary intersection for expensive multiobjective optimization. IEEE Trans Evol Comput 21:898–913. https://doi.org/10.1109/TEVC.2017.2693320

    Article  Google Scholar 

  29. Zhan D, Cheng Y, Liu J (2017) Expected improvement matrix-based infill criteria for expensive multiobjective optimization. IEEE Trans Evol Comput 21:956–975. https://doi.org/10.1109/TEVC.2017.2697503

    Article  Google Scholar 

  30. van der Herten J, Knudde N, Couckuyt I, Dhaene T (2020) Multi-objective Bayesian optimization for engineering simulation. In: Bartz-Beielstein T, Filipič B, Korošec P, Talbi E-G (eds) High-performance simulation-based optimization. Springer International Publishing, Cham, pp 47–68. https://doi.org/10.1007/978-3-030-18764-4_3

    Chapter  Google Scholar 

  31. Zuhal LR, Palar PS, Shimoyama K (2019) A comparative study of multi-objective expected improvement for aerodynamic design. Aerosp Sci Technol 91:548–560. https://doi.org/10.1016/j.ast.2019.05.044

    Article  Google Scholar 

  32. Li Z, Wang X, Ruan S et al (2018) A modified hypervolume-based expected improvement for multi-objective efficient global optimization method. Struct Multidiscip Optim 58:1961–1979. https://doi.org/10.1007/s00158-018-2006-3

    Article  MathSciNet  Google Scholar 

  33. Cheng S, Zhan H, Shu Z et al (2019) Effective optimization on Bump inlet using meta-model multi-objective particle swarm assisted by expected hyper-volume improvement. Aerosp Sci Technol 87:431–447. https://doi.org/10.1016/j.ast.2019.02.039

    Article  Google Scholar 

  34. Koziel S, Bekasiewicz A, Couckuyt I, Dhaene T (2014) Efficient multi-objective simulation-driven antenna design using co-kriging. IEEE Trans Antennas Propag 62:5900–5905. https://doi.org/10.1109/TAP.2014.2354673

    Article  MathSciNet  MATH  Google Scholar 

  35. Koziel S, Bekasiewicz A (2013) Multi-objective design of antennas using Variable-Fidelity Simulations and Surrogate surrogate models. IEEE Trans Antennas Propag 61:5931–5939. https://doi.org/10.1142/q0043

    Article  Google Scholar 

  36. Bekasiewicz A, Koziel A (2014) Rapid multi-objective optimization of a MIMO antenna for UWB applications. Loughbrgh Antennas Propag Conf LAPC 2014:500–503. https://doi.org/10.1109/LAPC.2014.6996434

    Article  Google Scholar 

  37. Leifsson L, Koziel S, Tesfahuneng YA, Hosder S (2015) Multi-objective aeroacoustic shape optimization by variable-fidelity models and response surface surrogates. In: 56th AIAA/ASCE/AHS/ASC Struct Struct Dyn Mater Conf, pp 1–12. https://doi.org/10.2514/6.2015-1800

  38. Koziel S, Leifsson L (2014) Multi-objective airfoil design using variable-fidelity CFD simulations and response surface surrogates. In: 10th AIAA Multidiscip Des Optim Spec Conf, pp 1–9. https://doi.org/10.2514/6.2014-0289

  39. Liu Y, Collette M (2014) Improving surrogate-assisted variable fidelity multi-objective optimization using a clustering algorithm. Appl Soft Comput J 24:482–493. https://doi.org/10.1016/j.asoc.2014.07.022

    Article  Google Scholar 

  40. Zhu J, Wang YJ, Collette M (2014) A multi-objective variable-fidelity optimization method for genetic algorithms. Eng Optim 46:521–542. https://doi.org/10.1080/0305215X.2013.786063

    Article  MathSciNet  Google Scholar 

  41. Shu L, Jiang P, Zhou Q et al (2018) An online variable fidelity metamodel-assisted multi-objective genetic algorithm for engineering design optimization. Appl Soft Comput J 66:438–448. https://doi.org/10.1016/j.asoc.2018.02.033

    Article  Google Scholar 

  42. Shu L, Jiang P, Zhou Q, Xie T (2019) An online variable-fidelity optimization approach for multi-objective design optimization. Struct Multidiscip Optim. https://doi.org/10.1007/s00158-019-02256-0

    Article  MathSciNet  Google Scholar 

  43. Zhou Q, Wu J, Xue T, Jin P (2019) A two-stage adaptive multi-fidelity surrogate model-assisted multi-objective genetic algorithm for computationally expensive problems. Eng Comput. https://doi.org/10.1007/s00366-019-00844-8

    Article  Google Scholar 

  44. Jiang P, Zhou Q, Liu J, Cheng Y (2019) A three-stage surrogate model-assisted multi-objective genetic algorithm for computationally expensive problems. In: 2019 IEEE Congress on evolutionary computation, CEC 2019–Proceedings. pp 1680–1687 https://doi.org/10.1109/CEC.2019.8790241

  45. Yi J, Gao L, Li X et al (2019) An on-line variable-fidelity surrogate-assisted harmony search algorithm with multi-level screening strategy for expensive engineering design optimization. Knowl-Based Syst 170:1–19. https://doi.org/10.1016/j.knosys.2019.01.004

    Article  Google Scholar 

  46. Habib A, Singh KH, Ray HT (2019) A multiple surrogate-assisted multi/many-objective multi-fidelity evolutionary algorithm. Inf Sci (Ny) 502:537–557. https://doi.org/10.1016/j.ins.2019.06.016

    Article  MathSciNet  Google Scholar 

  47. Belakaria S, Deshwal A, Doppa JR (2020) Multi-fidelity multi-objective bayesian optimization: an output space entropy search approach. In: Proceedings of the AAAI Conference on artificial intelligence, pp 10035–10043 https://doi.org/10.1609/aaai.v34i06.6560

  48. Han ZH, Görtz S (2012) Hierarchical kriging model for variable-fidelity surrogate modeling. AIAA J 50:1885–1896. https://doi.org/10.2514/1.J051354

    Article  Google Scholar 

  49. Xu J, Han Z, Song W, Li K (2020) Efficient aerodynamic optimization of propeller using hierarchical kriging models. J Phys Conf Ser 1519:12019. https://doi.org/10.1088/1742-6596/1519/1/012019

    Article  Google Scholar 

  50. Bu Y, Song W, Han Z et al (2020) Aerodynamic/aeroacoustic variable-fidelity optimization of helicopter rotor based on hierarchical Kriging model. Chin J Aeronaut 33:476–492. https://doi.org/10.1016/j.cja.2019.09.019

    Article  Google Scholar 

  51. Han Z, Xu C, Zhang L et al (2020) Efficient aerodynamic shape optimization using variable-fidelity surrogate models and multilevel computational grids. Chin J Aeronaut 33:31–47. https://doi.org/10.1016/j.cja.2019.05.001

    Article  Google Scholar 

  52. Zhang Y, Neelakantan A, Park C et al (2019) Adaptive sampling with varying sampling cost for design space exploration. AIAA J. https://doi.org/10.2514/1.J057470

    Article  Google Scholar 

  53. Klamroth K, Lacour R, Vanderpooten D (2015) On the representation of the search region in multi-objective optimization. Eur J Oper Res 245:767–778. https://doi.org/10.1016/j.ejor.2015.03.031

    Article  MathSciNet  MATH  Google Scholar 

  54. Tian Y, Cheng R, Zhang X, Jin Y (2017) PlatEMO: A MATLAB platform for evolutionary multi-objective optimization [Educational Forum]. IEEE Comput Intell Mag. https://doi.org/10.1109/MCI.2017.2742868

    Article  Google Scholar 

  55. Zitzler E, Thiele L (1999) Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach. IEEE Trans Evol Comput 3:257–271. https://doi.org/10.1109/4235.797969

    Article  Google Scholar 

  56. Coello CAC, Cortés NC (2005) Solving multiobjective optimization problems using an artificial immune system. Genet Program Evol Mach 6:163–190. https://doi.org/10.1007/s10710-005-6164-x

    Article  Google Scholar 

  57. Zhang Q, Liu W, Tsang E, Virginas B (2010) Expensive multiobjective optimization by MOEA/D with Gaussian process model. IEEE Trans Evol Comput 14:456–474. https://doi.org/10.1109/TEVC.2009.2033671

    Article  Google Scholar 

  58. Han Z, Liu F, Xu C, et al (2019) Efficient multi-objective evolutionary algorithm for constrained global optimization of expensive functions. In: 2019 IEEE Congress on Evolutionary Computation, CEC 2019–Proceedings. pp 2026–2033 https://doi.org/10.1109/CEC.2019.8789986

  59. Ishibuchi H, Imada R, Setoguchi Y, Nojima Y (2018) How to specify a reference point in hypervolume calculation for fair performance comparison. Evol Comput. https://doi.org/10.1162/EVCO_a_00226

    Article  Google Scholar 

  60. Yang K, Emmerich M, Deutz A, Bäck T (2019) Efficient computation of expected hypervolume improvement using box decomposition algorithms. J Glob Optim. https://doi.org/10.1007/s10898-019-00798-7

    Article  MathSciNet  MATH  Google Scholar 

  61. Yang K, Emmerich M, Deutz A, Bäck T (2018) Multi-objective bayesian global optimization using expected hypervolume improvement gradient. Swarm Evol Comput. https://doi.org/10.1016/j.swevo.2018.10.007

    Article  Google Scholar 

  62. Economon TD, Palacios F, Copeland SR et al (2016) SU2: an open-source suite for multiphysics simulation and design. AIAA J 54:828–846. https://doi.org/10.2514/1.J053813

    Article  Google Scholar 

  63. Hicks RM, Henne PA (1978) Wing design by numerical optimization. J Aircr doi 10(2514/3):58379

    Google Scholar 

  64. He Y, Sun J, Song P, Wang X (2020) Dual Kriging-assisted efficient global optimization of expensive problems with evaluation failures. Aerosp Sci Technol 105:106006. https://doi.org/10.1016/j.ast.2020.106006

    Article  Google Scholar 

Download references

Funding

The work is supported by National Natural Science Foundation of China under Grant No. 51606141, China Postdoctoral Science Foundation under Grant No. 2016M602817 and National Science and Technology Major Project under Grant No.2017-II-0007-0021.

Author information

Authors and Affiliations

  1. School of Energy and Power Engineering, Xi’an Jiaotong University, 28 West Xianning Road, Xi’an, 710049, China

    Youwei He, Jinju Sun, Peng Song & Xuesong Wang

  2. Collaborative Innovation Center for Advance Aero-Engine (CICAAE) of China, 37 Xueyuan Road, Beijing, 100191, China

    Jinju Sun

Authors
  1. Youwei He
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Jinju Sun
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Peng Song
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Xuesong Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Jinju Sun.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Appendix

Appendix

figure b
figure c

Gaussian_CDF and Gaussian_PDF are the Gaussian cumulative distribution function and probability density function respectively, and are defined as follows.

figure d

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

He, Y., Sun, J., Song, P. et al. Variable-fidelity hypervolume-based expected improvement criteria for multi-objective efficient global optimization of expensive functions. Engineering with Computers 38, 3663–3689 (2022). https://doi.org/10.1007/s00366-021-01404-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00366-021-01404-9

Keywords