Skip to main content
SpringerLink
Log in

Kriging-based infill sampling criterion for constraint handling in multi-objective optimization

  • Published:
Journal of Global Optimization Aims and scope Submit manuscript

Abstract

This paper proposes a novel infill sampling criterion for constraint handling in multi-objective optimization of computationally expensive black-box functions. To reduce the computational burden, Kriging models are used to emulate the objective and constraint functions. The challenge of this multi-objective optimization problem arises from the fact that the epistemic uncertainty of the Kriging models should be taken into account to find Pareto-optimal solutions in the feasible domain. This is done by the proposed sampling criterion combining the Expected HyperVolume Improvement of the front of nondominated solutions and the Probability of Feasibility of new candidates. The proposed criterion is non-intrusive and derivative-free, and it is oriented to: (1) problems in which the computational cost is mainly from the function evaluation rather than optimization, and (2) problems that use complex in-house or commercial software that cannot be modified. The results using the proposed sampling criterion are compared with the results using Multi-Objective Evolutionary Algorithms. These results show that the proposed sampling criterion permits to identify both the feasible domain and an approximation of the Pareto front using a reduced number of computationally expensive simulations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Arora, J.S.: Introduction to Optimum Design. McGraw-Hill, New York (1989)

    Google Scholar 

  2. Bautista, D.C.: A Sequential Design for Approximating the Pareto Front Using the Expected Pareto Improvement Function. Ohio State University, PhD thesis (2009)

  3. Beume, N., Naujoks, B., Emmerich, M.: SMS-EMOA: multiobjective selection based on dominated hypervolume. Eur. J. Oper. Res. 181(3), 1653–1669 (2007)

    Article  MATH  Google Scholar 

  4. Coello Coello, C.A.: Handling preferences in evolutionary multiobjective optimization: a survey. In: Proceedings of IEEE Evolutionary Computation, La Jolla, CA, pp. 30–37 (2000)

  5. Coello Coello, C.A., Lamont, G.B., Van-Veldhuizen, D.A.: Evolutionary algorithms for solving multi-objective problems. In: Goldberg, D.E., Koza, J.R. (eds.) Genetic Algorithms and Evolutionary Computation. Springer, New York (2007)

    Google Scholar 

  6. Collette, Y., Siarry, P.: Multiobjective Optimization—Principles and Case Studies. Springer, Berlin (2003)

    Google Scholar 

  7. Couckuyt, I., Deschrijver, D., Dhaene, T.: Fast calculation of multiobjective probability of improvement and expected improvement criteria for Pareto optimization. J. Glob. Optim. 60(3), 575–594 (2014)

    Article  MATH  MathSciNet  Google Scholar 

  8. Deb, K.: Multi-objective Optimization Using Evolutionary Algorithms. Wiley, New York (2001)

    MATH  Google Scholar 

  9. Deb, K., Pratap, A., Agarwal, S., Meyarivan, : A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 6(2), 182–197 (2002)

    Article  Google Scholar 

  10. Ehrgott, M.: Multiple Criteria Optimization—Classification and Methodology. Shaker Verlag, Aachen (1997)

    Google Scholar 

  11. Emmerich, M.T.M., Giannakoglou, K.C., Naujoks, B.: Single- and multiobjective evolutionary optimization assisted by Gaussian random field metamodels. IEEE Trans. Evol. Comput. 10(4), 421–439 (2006)

    Article  Google Scholar 

  12. Emmerich, M., Deutz, A.H., Klinkenberg, J.W.: The computation of the expected improvement in dominated hypervolume of Pareto front approximations, Technical report, Leiden University (2008)

  13. Feng, Z., Zhang, Q., Zhang, Q., Tang, Q., Yang, T., Ma, Y.: A multiobjective optimization based framework to balance the global exploration and local exploitation in expensive optimization. J. Glob. Optim. 61(4), 677–694 (2015)

    Article  MathSciNet  Google Scholar 

  14. Finkel, D.E.: DIRECT Optimization Algorithm User Guide. North Carolina State University. http://www4.ncsu.edu/definkel/research/index.html (2003). Accessed 28 Sep 2014

  15. Forrester, A.I.J., Sobester, A., Keane, A.J.: Engineering Design via Surrogate Modelling—A Practical Guide. Wiley, New York (2008)

    Book  Google Scholar 

  16. Forrester, A.I.J., Keane, A.J.: Recent advances in surrogate-based optimization. Prog. Aerosp. Sci. 45(1–3), 50–79 (2009)

    Article  Google Scholar 

  17. Gaspar-Cunha, A., Vieira, A.: A multi-objective evolutionary algorithm using neural networks to approximate fitness evaluations. Int. J. Comput. Syst. Signals 6(1), 18–36 (2005)

    Google Scholar 

  18. Hawe, G.I., Sykulski, J.K.: An enhanced probability of improvement utility function for locating Pareto-optimal solutions. In: Proceedings of the Conference on Computation of Electromagnetic Fields, Aachen, Germany, pp. 965–966 (2007)

  19. Hawe, G.I., Sykulski, J.K.: Scalarizing cost-effective multi-objective optimization algorithms made possible with kriging. COMPEL 27(4), 836–844 (2008)

    Article  MATH  Google Scholar 

  20. Henkenjohann, N., Kunert, J.: An efficient sequential optimization approach based on the multivariate expected improvement criterion. Qual. Eng. 19(4), 267–280 (2007)

    Article  Google Scholar 

  21. Jeong, S., Minemura, Y., Obayashi, S.: Optimization of combustion chamber for diesel engine using Kriging model. J. Fluid Sci. Technol. 1(2), 138–146 (2006)

    Article  Google Scholar 

  22. Jones, D.R., Perttunen, C.D., Stuckmann, B.E.: Lipschitzian optimization without the Lipschitz constant. J. Optim. Theory App. 79(1), 157–181 (1993)

    Article  MATH  Google Scholar 

  23. Jones, D.R., Schonlau, M., Welch, W.J.: Efficient global optimization of expensive black-box functions. J. Glob. Optim. 13(4), 455–492 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  24. Jones, D.R.: A taxonomy of global optimization methods based on response surfaces. J. Glob. Optim. 21(4), 345–383 (2001)

    Article  MATH  Google Scholar 

  25. Keane, A.J.: Statistical improvement criteria for use in multiobjective design optimization. AIAA J. 44(4), 879–891 (2006)

    Article  Google Scholar 

  26. Kleijnen, J.P.C.: Kriging metamodeling in simulation: a review. Eur. J. Oper. Res. 192(3), 707–716 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  27. Knowles, J.: ParEGO: a hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems. IEEE Trans. Evol. Comput. 10(1), 50–66 (2006)

    Article  Google Scholar 

  28. Krige, D.G.: A Statistical approach to some basic mine valuation problems on the Witwatersrand. J. South. Afr. Inst. Min. Metall. 52(6), 119–139 (1951)

    Google Scholar 

  29. Li, M., Li, G., Azarm, S.: A Kriging metamodel assisted multi-objective genetic algorithm for design optimization. J. Mech. Des. 130(3), 1–10 (2008)

    Article  Google Scholar 

  30. Li, G., Li, M., Azarm, S., Al Hashimi, S., Al Ameri, T., Al Qasas, N.: Improving multi-objective genetic algorithms with adaptive design of experiments and online metamodeling. Struct. Multidiscip. Optim. 37(5), 447–461 (2009)

    Article  Google Scholar 

  31. Liu, Q., Cheng, W.: A modified DIRECT algorithm with bilevel partition. J. Glob. Optim. 60(3), 483–499 (2014)

    Article  MATH  MathSciNet  Google Scholar 

  32. Marler, R.T., Arora, J.S.: Survey of multi-objective optimization methods for engineering. Struct. Multidiscip. Optim. 26(6), 369–395 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  33. Meckesheimer, M., Booker, A.J., Barton, R.R., Simpson, T.W.: Computationally inexpensive metamodel assessment strategies. AIAA J. 40(10), 2053–2060 (2002)

    Article  Google Scholar 

  34. Mockus, J., Tiesis, V., Zilinskas, A.: The application of Bayesian methods for seeking the extremum. In: Dixon, L.C.W., Szego, G.P. (eds.) Toward Global Optimisation 2, pp. 117–130. Elsevier, Amsterdam (1978)

    Google Scholar 

  35. Nielsen, H.B., Lophaven, S.N., Søndergaard, J.: DACE—A Matlab Kriging Toolbox. Department of Informatics and Mathematical Modeling, Technical University of Denmark (2002)

  36. Obayashi, S.: Multi-objective design exploration using efficient global optimization. In: Proceedings of the European Conference on Computational Fluid Dynamics (ECCOMAS CFD), TU Delft, pp. 1–8 (2006)

  37. Pareto, V.: Manuale di Economia Politica. Societa Editrice, Milan (1906)

    Google Scholar 

  38. Ponweiser, W., Wagner, T., Biermann, D., Vincze, M.: Multiobjective optimization on a limited budget of evaluations using model-assisted S-Metric selection. In: Rudolph, G., Jansen, T., Lucas, S., Poloni, C., Beume, N. (eds.) Parallel Problem Solving from Nature—PPSN X, pp. 784–794. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  39. Regis, R.G., Shoemaker, C.A.: Constrained global optimization of expensive black box functions using radial basis functions. J. Glob. Optim. 31(1), 153–171 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  40. Regis, R.G.: Constrained optimization by radial basis function interpolation for high-dimensional expensive black-box problems with infeasible initial points. Eng. Optim. 46(2), 218–243 (2014)

    Article  MathSciNet  Google Scholar 

  41. Regis, R.G.: Evolutionary programming for high-dimensional constrained expensive black-box optimization using radial basis functions. IEEE Trans. Evolut. Comput. 18(3), 326–347 (2014)

    Article  Google Scholar 

  42. Sacks, J., Welch, W.J., Mitchell, T.J., Wynn, H.P.: Design and analysis of computer experiments. Stat. Sci. 4(4), 409–435 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  43. Santner, T., Williams, B., Notz, W.: The Design and Analysis of Computer Experiments. Springer Series in Statistics. Springer, Berlin (2003)

    Book  Google Scholar 

  44. Sasena, M.J., Papalambros, P.Y., Goovaerts, P.: Exploration of metamodeling sampling criteria for constrained global optimization. Eng. Optim. 34(3), 263–278 (2002)

    Article  Google Scholar 

  45. Singh, P., Couckuyt, I., Ferranti, F., Dhaene, T.: A constrained multi-objective surrogate-based optimization algorithm. In: Proceedings of World Congress on Computational Intelligence, Beijing, China (2014)

  46. Shimoyama, K., Jeong, S., Obayashi, S.: Kriging-surrogate-based optimization considering expected hypervolume improvement in non-constrained many-objective test problems. In: Proceedings of IEEE Congress on Evolutionary Computation (CEC), Cancun, Mexico, pp. 658–665 (2013)

  47. Ulmer, H., Streichert, F., Zell, A.: Evolution strategies assisted by Gaussian processes with improved pre-selection criterion. In: Proceedings of IEEE Congress on Evolutionary Computation (CEC), Canberra, Australia, pp. 692–699 (2003)

  48. Voutchkov, I., Keane, A.: Multiobjective optimization using surrogates. In: Tenne, Y., Goh, C.-K. (eds.) Computational Intelligence in Optimization—Applications and Implementations, pp. 155–175. Springer, Berlin (2010)

    Chapter  Google Scholar 

  49. Wang, G., Shan, S.: An efficient Pareto set identification approach for multiobjective optimization on black-box functions. J. Mech. Des. 127(5), 866–874 (2005)

    Article  MathSciNet  Google Scholar 

  50. Wolfe, M.A.: An interval algorithm for bound constrained global optimization. Optim. Methods Softw. 6(2), 145–159 (1995)

    Article  Google Scholar 

  51. Zhang, Y., Leithead, W.: Exploiting Hessian matrix and trust-region algorithm in hyperparameters estimation of Gaussian process. Appl. Math. Comput. 171(2), 1264–1281 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  52. Zhang, Q., Liu, W., Tsang, E., Virginas, B.: Expensive multiobjective optimization by MOEA/D with Gaussian process model. IEEE Trans. Evol. Comp. 14(3), 456–474 (2010)

    Article  Google Scholar 

  53. Zilinskas, A.: One-step Bayesian method of the search for extremum of an one-dimensional function. Cybernetics 1, 139–144 (1975)

    MathSciNet  Google Scholar 

  54. Zitzler, E., Laumanns, M., Thiele, L.: SPEA2: improving the strength Pareto evolutionary algorithm for multiobjective optimization. In: Giannakoglou, K.C., et al. (eds.) Evolutionary Methods for Design, Optimisation and Control with Application to Industrial Problems, pp. 95–100. CIMNE, Barcelona (2002)

    Google Scholar 

  55. Zitzler, E., Thiele, L., Laumanns, M., Fonseca, C.M., da Fonseca, V.G.: Performance assessment of multiobjective optimizers: an analysis and review. IEEE Trans. Evol. Comp. 7(2), 117–132 (2003)

    Article  Google Scholar 

  56. Zuluaga, M., Krause, A., Sergent, G., Markus, P.: Active learning for multi-objective optimization. In: Proceedings of International Conference on Machine Learning, Atlanta, USA (2013)

Download references

Acknowledgments

This work was supported through the research support programme from “Fundación Séneca-Agencia de Ciencia y Tecnología de la Región de Murcia” under the contract 19274/PI/14.

Author information

Authors and Affiliations

  1. Department of Structures and Construction, Technical University of Cartagena, Campus Muralla del Mar, 30202, Cartagena, Murcia, Spain

    Jesús Martínez-Frutos & David Herrero-Pérez

Authors
  1. Jesús Martínez-Frutos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David Herrero-Pérez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jesús Martínez-Frutos.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Martínez-Frutos, J., Herrero-Pérez, D. Kriging-based infill sampling criterion for constraint handling in multi-objective optimization. J Glob Optim 64, 97–115 (2016). https://doi.org/10.1007/s10898-015-0370-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10898-015-0370-8

Keywords

Search

Navigation