Abstract
In this research, we develop a Bayesian optimization algorithm to solve expensive, constrained problems. We consider the presence of heteroscedastic noise in the evaluations and thus propose a new acquisition function to account for this noise in the search for the optimal point. We use stochastic kriging to fit the metamodels, and we provide computational results to highlight the importance of accounting for the heteroscedastic noise in the search for the optimal solution. Finally, we propose some promising directions for further research.
This study is supported by the Special Research Fund (BOF) of Hasselt University (grant number: BOF19OWB01), and the Flanders Artificial Intelligence Research Program (FLAIR).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Amaran, S., Sahinidis, N.V., Sharda, B., Bury, S.J.: Simulation optimization: a review of algorithms and applications. Ann. Oper. Res. 240(1), 351–380 (2016)
Ankenman, B., Nelson, B.L., Staum, J.: Stochastic kriging for simulation metamodeling. Oper. Res. 58(2), 371–382 (2010). https://doi.org/10.1287/opre.1090.0754
Fieldsend, J.E., Everson, R.M.: The rolling tide evolutionary algorithm: a multiobjective optimizer for noisy optimization problems. IEEE Trans. Evol. Comput. 19(1), 103–117 (2014)
Frazier, P.I.: Bayesian optimization. In: Recent advances in optimization and modeling of contemporary problems, pp. 255–278. Informs (2018)
Frazier, P.I., Powell, W.B., Dayanik, S.: A knowledge-gradient policy for sequential information collection. SIAM J. Control. Optim. 47(5), 2410–2439 (2008)
Gelbart, M.A., Snoek, J., Adams, R.P.: Bayesian optimization with unknown constraints. arXiv preprint arXiv:1403.5607 (2014)
Gonzalez, S.R., Jalali, H., Van Nieuwenhuyse, I.: A multiobjective stochastic simulation optimization algorithm. Eur. J. Oper. Res. 284(1), 212–226 (2020)
Gramacy, R.B., et al.: Modeling an augmented Lagrangian for blackbox constrained optimization. Technometrics 58(1), 1–11 (2016)
Gramacy, R.B., Lee, H.K.: Optimization under unknown constraints. In: Proceeding of the ninth Bayesian Statistics International Meeting, pp. 229–256. Oxford University Press (2011)
Hennig, P., Schuler, C.J.: Entropy search for information-efficient global optimization. J. Mach. Learn. Res. 13(6), 1809–1837 (2012)
Hernández-Lobato, J.M., Gelbart, M.A., Adams, R.P., Hoffman, M.W., Ghahramani, Z.: A general framework for constrained Bayesian optimization using information-based search. J. Mach. Learn. Res. 17(1), 1–53 (2016)
Jalali, H., Van Nieuwenhuyse, I., Picheny, V.: Comparison of kriging-based algorithms for simulation optimization with heterogeneous noise. Eur. J. Oper. Res. 261(1), 279–301 (2017)
Jones, D.R., Schonlau, M., Welch, W.J.: Efficient global optimization of expensive black-box functions. J. Global Optim. 13(4), 455–492 (1998)
Kleijnen, J.P.: Kriging metamodeling in simulation: a review. Eur. J. Oper. Res. 192(3), 707–716 (2009)
Kleijnen, J.P., Van Nieuwenhuyse, I., van Beers, W.: Constrained optimization in simulation: efficient global optimization and karush-kuhn-tucker conditions (2021)
Letham, B., Karrer, B., Ottoni, G., Bakshy, E.: Constrained Bayesian optimization with noisy experiments. Bayesian Anal. 14(2), 495–519 (2019)
Loeppky, J., Sacks, J., Welch, W.: Choosing the sample size of a computer experiment: a practical guide. Technometrics 51(4), 366–376 (2009)
Močkus, J.: On Bayesian methods for seeking the extremum. In: Marchuk, G.I. (ed.) Optimization Techniques 1974. LNCS, vol. 27, pp. 400–404. Springer, Heidelberg (1975). https://doi.org/10.1007/3-540-07165-2_55
Pourmohamad, T., Lee, H.K.: Bayesian optimization via barrier functions. J. Comput. Graph. Stat. 31(1), 74–83 (2022)
Quan, N., Yin, J., Ng, S.H., Lee, L.H.: Simulation optimization via kriging: a sequential search using expected improvement with computing budget constraints. IIE Trans. 45(7), 763–780 (2013)
Rasmussen, C.E., Williams, C.: Gaussian Processes for Machine Learning. MIT press, Cambridge (2006)
Rojas-Gonzalez, S., Van Nieuwenhuyse, I.: A survey on kriging-based infill algorithms for multiobjective simulation optimization. Comput. Oper. Res. 116, 104869 (2020)
Sammut, C., Webb, G.I.: Encyclopedia of Machine Learning. Springer, New York (2011)
Ungredda, J., Branke, J.: Bayesian optimisation for constrained problems. arXiv preprint arXiv:2105.13245 (2021)
Zeng, Y., Cheng, Y., Liu, J.: An efficient global optimization algorithm for expensive constrained black-box problems by reducing candidate infilling region. Inf. Sci. 609, 1641–1669 (2022)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Amini, S., Van Nieuwenhuyse, I. (2023). A Bayesian Optimization Algorithm for Constrained Simulation Optimization Problems with Heteroscedastic Noise. In: Sellmann, M., Tierney, K. (eds) Learning and Intelligent Optimization. LION 2023. Lecture Notes in Computer Science, vol 14286. Springer, Cham. https://doi.org/10.1007/978-3-031-44505-7_6
Download citation
DOI: https://doi.org/10.1007/978-3-031-44505-7_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44504-0
Online ISBN: 978-3-031-44505-7
eBook Packages: Computer ScienceComputer Science (R0)