Abstract
Many real-world problems have complicated objective functions whose optimization requires sophisticated sequential decision-making strategies. Modelling human function learning has been the subject of intense research in cognitive sciences. The topic is relevant in black-box optimization where information about the objective and/or constraints is not available and must be learned through function evaluations. The Gaussian Process based Bayesian learning paradigm is central in the development of active learning approaches balancing exploration/exploitation in uncertain conditions towards effective generalization in large decision spaces. In this paper we focus on Bayesian Optimization and analyse experimentally how it compares to humans while searching for the maximum of an unknown 2D function. A set of controlled experiments with 53 subjects confirm that Gaussian Processes provide a general model to explain different patterns of learning enabled search and optimization in humans.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Adam, S.P., Alexandropoulos, S.A.N., Pardalos, P.M., Vrahatis, M.N.: No free lunch theorem: a review. In: Demetriou, I.C., Pardalos, P.M. (eds.) Approximation and Optimization. SOIA, vol. 145, pp. 57–82. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-12767-1_5
Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Mach. Learn. 47(2–3), 235–256 (2002)
Borji, A., Itti, L.: Bayesian optimization explains human active search. In: Advances in Neural Information Processing System 26 (NIPS 2013), pp. 55–63 (2013)
Candelieri, A., Perego, R., Archetti, F.: Bayesian optimization of pump operations in water distribution systems. J. Glob. Optim. 71, 213–235 (2018)
Chapelle, O., Li, L.: An empirical evaluation of thompson sampling. In: Advances in Neural Information Processing Systems, pp. 2249–2257 (2011)
Eggensperger, K., Lindauer, M., Hutter, F.: Pitfalls and best practices in algorithm configuration. J. Artif. Intell. Res. 64, 861–893 (2019)
Gershman, S.J.: Uncertainty and exploration. bioRxiv 265504 (2018). https://doi.org/10.1101/265504
Gopnik, A., O’Grady, S., Lucas, C.G., Griffiths, T.L., Wente, A., Bridgers, S., Dahl, R.E.: Changes in cognitive flexibility and hypothesis search across human life history from childhood to adolescence to adulthood. Proc. Nat. Acad. Sci. 114(30), 7892–7899 (2017)
Kruschke, J.K.: Bayesian approaches to associative learning: from passive to active learning. Learn. Behav. 36(3), 210–226 (2008)
Jones, D.R., Schonlau, M., Welch, W.J.: Efficient global optimization of expensive black-box functions. J. Glob. Optim. 13(4), 455–492 (1998)
Li, K., Malik, J.: Learning to optimize. (2016) arXiv preprint arXiv:1606.01885
May, B.C., Korda, N., Lee, A., Leslie, D.S.: Optimistic Bayesian sampling in contextual-bandit problems. J. Mach. Learn. Res. 13(Jun), 2069–2106 (2012)
Mehlhorn, K., Newell, B.R., Todd, P.M., Lee, M.D., Morgan, K., Braithwaite, V.A., Gonzalez, C.: Unpacking the exploration–exploitation tradeoff: a synthesis of human and animal literatures. Decision 2(3), 191 (2015)
Gershman, S.J.: Quantifying mismatch in Bayesian optimization. In: NIPS Workshop on Bayesian Optimization: Black-Box Optimization and Beyond (2016)
Schulz, E., Tenenbaum, J., Duvenaud, D.K., Speekenbrink, M., Gershman, S.J.: Probing the compositionality of intuitive functions. In: Advances in Neural Information Processing Systems, pp. 3729–3737 (2016)
Schulz, E., Speekenbrink, M., Krause, A.: A tutorial on Gaussian process regression: modelling, exploring, and exploiting functions. J. Math. Psychol. 85, 1–16 (2018)
Srinivas, N., Krause, A., Kakade, S., Seeger, M.: Gaussian process optimization in the bandit setting: no regret and experimental design. In: Proceedings of the 27th International Conference on Machine Learning, pp. 1015–1022. Omnipress, June 2010
Thompson, W.R.: On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika 25(3/4), 285–294 (1933)
Wilson, R.C., Geana, A., White, J.M., Ludvig, E.A., Cohen, J.D.: Humans use directed and random exploration to solve the explore–exploit dilemma. J. Exp. Psychol. Gen. 143(6), 2074 (2014)
Wu, C.M., Schulz, E., Speekenbrink, M., Nelson, J.D., Meder, B.: Generalization guides human exploration in vast decision spaces. Nat. Hum. Behav. 2(12), 915 (2018)
Zhigljavsky, A., Zilinskas, A.: Stochastic Global Optimization, vol. 9. Springer, Berlin (2007). https://doi.org/10.1007/978-0-387-74740-8
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Candelieri, A., Perego, R., Giordani, I., Archetti, F. (2020). Are Humans Bayesian in the Optimization of Black-Box Functions?. In: Sergeyev, Y., Kvasov, D. (eds) Numerical Computations: Theory and Algorithms. NUMTA 2019. Lecture Notes in Computer Science(), vol 11974. Springer, Cham. https://doi.org/10.1007/978-3-030-40616-5_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-40616-5_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-40615-8
Online ISBN: 978-3-030-40616-5
eBook Packages: Computer ScienceComputer Science (R0)