Abstract
Multi-objective optimization requires many evaluations to identify a sufficiently dense approximation of the Pareto front. Especially for a higher number of objectives, extracting the Pareto front might not be easy nor cheap. On the other hand, the Decision-Maker is not always interested in the entire Pareto front, and might prefer a solution where there is a desirable trade-off between different objectives. An example of an attractive solution is the knee point of the Pareto front, although the current literature differs on the definition of a knee. In this work, we propose to detect knee solutions in a data-efficient manner (i.e., with a limited number of time-consuming evaluations), according to two definitions of knees. In particular, we propose several novel acquisition functions in the Bayesian Optimization framework for detecting these knees, which allows for scaling to many objectives. The suggested acquisition functions are evaluated on various benchmarks with promising results.
This work has been supported by the Flemish Government under the ‘Onderzoeksprogramma Artificiële Intelligentie (AI) Vlaanderen’ and the ‘Fonds Wetenschappelijk Onderzoek (FWO)’ programmes.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Berkeley, J., et al.: Trieste, February 2022. https://github.com/secondmind-labs/trieste
Bhattacharjee, K.S., Singh, H.K., Ryan, M., Ray, T.: Bridging the gap: many-objective optimization and informed decision-making. IEEE Trans. Evol. Comput. 21(5), 813–820 (2017). https://doi.org/10.1109/TEVC.2017.2687320
Blank, J., Deb, K.: pymoo: multi-objective optimization in python. IEEE Access 8, 89497–89509 (2020)
Branke, J., Deb, K., Dierolf, H., Osswald, M.: Finding knees in multi-objective optimization. In: Yao, X., et al. (eds.) PPSN 2004. LNCS, vol. 3242, pp. 722–731. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-30217-9_73
Chand, S., Wagner, M.: Evolutionary many-objective optimization: a quick-start guide. Surv. Oper. Res. Manage. Sci. 20(2), 35–42 (2015)
Couckuyt, I., Deschrijver, D., Dhaene, T.: Fast calculation of multiobjective probability of improvement and expected improvement criteria for Pareto optimization. J. Global Optim. 60(3), 575–594 (2013). https://doi.org/10.1007/s10898-013-0118-2
Deb, K., Thiele, L., Laumanns, M., Zitzler, E.: Scalable multi-objective optimization test problems. In: Proceedings of the 2002 Congress on Evolutionary Computation. CEC 2002 (Cat. No.02TH8600). vol. 1, pp. 825–830 (2002). https://doi.org/10.1109/CEC.2002.1007032
Emmerich, M., Giannakoglou, K., Naujoks, B.: Single- and multiobjective evolutionary optimization assisted by gaussian random field metamodels. IEEE Trans. Evol. Comput. 10(4), 421–439 (2006). https://doi.org/10.1109/TEVC.2005.859463
Hakanen, J., Knowles, J.D.: On using decision maker preferences with ParEGO. In: Trautmann, H., et al. (eds.) EMO 2017. LNCS, vol. 10173, pp. 282–297. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-54157-0_20
Hua, Y., Liu, Q., Hao, K., Jin, Y.: A survey of evolutionary algorithms for multi-objective optimization problems with irregular pareto fronts. IEEE/CAA J. Automatica Sinica 8(2), 303–318 (2021). https://doi.org/10.1109/JAS.2021.1003817
Indraneel, D.: On characterizing the “knee” of the Pareto curve based on normal-boundary intersection. Struct. Optim. 18(2), 107–115 (1999). https://doi.org/10.1007/BF01195985
Jones, D.: A taxonomy of global optimization methods based on response surfaces. J. Global Optim. 21, 345–383 (2001). https://doi.org/10.1023/A:1012771025575
Jones, D.R., Schonlau, M., Welch, W.J.: Efficient global optimization of expensive black-box functions. J. Global Optim. 13, 455–492 (1998)
Knowles, J.: ParEGO: a hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems. IEEE Trans. Evol. Comput. 10(1), 50–66 (2006)
Ma, X., Yu, Y., Li, X., Qi, Y., Zhu, Z.: A survey of weight vector adjustment methods for decomposition-based multiobjective evolutionary algorithms. IEEE Trans. Evol. Comput. 24(4), 634–649 (2020). https://doi.org/10.1109/TEVC.2020.2978158
Minasny, B., McBratney, A.B.: The matérn function as a general model for soil variograms. Geoderma 128(3), 192–207 (2005). https://doi.org/10.1016/j.geoderma.2005.04.003, pedometrics 2003
Mockus, J.: Bayesian Approach to global Optimization: Theory and Applications, vol. 37. Springer (1989). https://doi.org/10.1007/978-94-009-0909-0
Mockus, J., Tiesis, V., Zilinskas, A.: The application of Bayesian methods for seeking the extremum. Towards Global Optim. 2(117–129), 2 (1978)
Picheny, V., Wagner, T., Ginsbourger, D.: A benchmark of kriging-based infill criteria for noisy optimization. Struct. Multidiscip. Optim. 48, 607–626 (2013). https://doi.org/10.1007/s00158-013-0919-4
Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning, 1st edn. The MIT Press, Cambridge (2005)
Ray, T., Singh, H.K., Rahi, K.H., Rodemann, T., Olhofer, M.: Towards identification of solutions of interest for multi-objective problems considering both objective and variable space information. Appl. Soft Comput. 119, 108505 (2022). https://doi.org/10.1016/j.asoc.2022.108505
Rojas-Gonzalez, S., Van Nieuwenhuyse, I.: A survey on kriging-based infill algorithms for multiobjective simulation optimization. Comput. Oper. Res. 116, 104869 (2020)
Trivedi, A., Srinivasan, D., Sanyal, K., Ghosh, A.: A survey of multiobjective evolutionary algorithms based on decomposition. IEEE Trans. Evol. Comput. 21(3), 440–462 (2017). https://doi.org/10.1109/TEVC.2016.2608507
Yu, G., Jin, Y., Olhofer, M.: Benchmark problems and performance indicators for search of knee points in multiobjective optimization. IEEE Trans. Cybern. 50(8), 3531–3544 (2020). https://doi.org/10.1109/TCYB.2019.2894664
Yu, G., Ma, L., Jin, Y., Du, W., Liu, Q., Zhang, H.: A survey on knee-oriented multi-objective evolutionary optimization. IEEE Trans. Evol. Comput. 1 (2022). https://doi.org/10.1109/TEVC.2022.3144880
Zhang, K., Yen, G.G., He, Z.: Evolutionary algorithm for knee-based multiple criteria decision making. IEEE Trans. Cybern. 51(2), 722–735 (2021). https://doi.org/10.1109/TCYB.2019.2955573
Zitzler, E., Thiele, L., Laumanns, M., Fonseca, C., da Fonseca, V.: Performance assessment of multiobjective optimizers: an analysis and review. IEEE Trans. Evol. Comput. 7(2), 117–132 (2003). https://doi.org/10.1109/TEVC.2003.810758
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Heidari, A., Qing, J., Rojas Gonzalez, S., Branke, J., Dhaene, T., Couckuyt, I. (2022). Finding Knees in Bayesian Multi-objective Optimization. In: Rudolph, G., Kononova, A.V., Aguirre, H., Kerschke, P., Ochoa, G., Tušar, T. (eds) Parallel Problem Solving from Nature – PPSN XVII. PPSN 2022. Lecture Notes in Computer Science, vol 13398. Springer, Cham. https://doi.org/10.1007/978-3-031-14714-2_8
Download citation
DOI: https://doi.org/10.1007/978-3-031-14714-2_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-14713-5
Online ISBN: 978-3-031-14714-2
eBook Packages: Computer ScienceComputer Science (R0)