Skip to main content
Springer Nature Link
Log in

Resource Allocation via Bayesian Optimization: an Efficient Alternative to Semi-Bandit Feedback

  • Conference paper
  • First Online:
Numerical Computations: Theory and Algorithms (NUMTA 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14476))

  • 150 Accesses

Abstract

Although optimal resource allocation is a well-known and studied problem, the recent technological innovations are bringing to light new specificities and issues. Some relevant real-life applications are the optimal management of cloud/high-performance computing resources, and the optimal budget allocation for multi-channel marketing. Recent formulations have led to the definition of the Semi-Bandit Feedback approach, that is the reference method in these emerging real-life settings. In this paper we propose a novel approach, extending the Bayesian Optimization framework to specifically deal with the resource allocation problem, and finally resulting more efficient than Semi-Bandit Feedback. Moreover, the proposed approach can also deal with specific (real-life) settings that cannot be covered by Semi-Bandit Feedback. We have validated our approach on (i) the case study reported in the original paper proposing Semi-Bandit Feedback, (ii) a multi-channel marketing application, and (iii) the optimal mix of water sources in water distribution networks.

This research was supported by the following grant: ENERGIDRICA – Efficienza energetica nelle reti idriche (CUP B42F20000390006) Programma PON “Ricerca e Innovazione” 2014- 2020 – Azione II – OS 1.b.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Archetti, F., Candelieri, A.: Bayesian optimization and data science. Springer (2019). https://doi.org/10.1007/978-3-030-24494-1

  2. Bakker, H., Dunke, F., Nickel, S.: A structuring review on multi-stage optimization under uncertainty: aligning concepts from theory and practice. Omega 96, 102080 (2020)

    Article  Google Scholar 

  3. Balcan, M.F., Dick, T., Pegden, W.: Semi-bandit optimization in the dispersed setting. In: Conference on Uncertainty in Artificial Intelligence, pp. 909–918. PMLR (2020)

    Google Scholar 

  4. Barrier, A., Garivier, A., Stoltz, G.: On best-arm identification with a fixed budget in non-parametric multi-armed bandits. In: International Conference on Algorithmic Learning Theory. pp. 136–181. PMLR (2023)

    Google Scholar 

  5. Berk, J., Gupta, S., Rana, S., Venkatesh, S.: Randomised gaussian process upper confidence bound for Bayesian optimisation. In: Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence, pp. 2284–2290 (2021)

    Google Scholar 

  6. Brandt, J., Haddenhorst, B., Bengs, V., Hüllermeier, E.: Finding optimal arms in non-stochastic combinatorial bandits with semi-bandit feedback and finite budget. arXiv preprint arXiv:2202.04487 (2022)

  7. Candelieri, A.: A gentle introduction to Bayesian optimization. In: 2021 Winter Simulation Conference (WSC), pp. 1–16. IEEE (2021)

    Google Scholar 

  8. Candelieri, A.: Sequential model based optimization of partially defined functions under unknown constraints. J. Global Optim. 79(2), 281–303 (2021)

    Article  MathSciNet  Google Scholar 

  9. Candelieri, A., Ponti, A., Giordani, I., Archetti, F.: On the use of Wasserstein distance in the distributional analysis of human decision making under uncertainty. Ann. Math. Artif. Intell., 1–22 (2022)

    Google Scholar 

  10. Carpentier, P., Chancelier, J.P., Cohen, G., De Lara, M.: Stochastic multi-stage optimization. Probability Theory and Stochastic Modelling 75 (2015)

    Google Scholar 

  11. Chen, W., Wang, L., Zhao, H., Zheng, K.: Combinatorial semi-bandit in the non-stationary environment. In: Uncertainty in Artificial Intelligence, pp. 865–875. PMLR (2021)

    Google Scholar 

  12. Dagan, Y., Koby, C.: A better resource allocation algorithm with semi-bandit feedback. In: Algorithmic Learning Theory, pp. 268–320. PMLR (2018)

    Google Scholar 

  13. Frazier, P.I.: Bayesian optimization. In: Recent Advances in Optimization and modeling of contemporary problems, pp. 255–278. Informs (2018)

    Google Scholar 

  14. Gelbart, M.A., Snoek, J., Adams, R.P.: Bayesian optimization with unknown constraints. In: 30th Conference on Uncertainty in Artificial Intelligence, UAI 2014, pp. 250–259. AUAI Press (2014)

    Google Scholar 

  15. Gelbart, M.A.: Constrained Bayesian optimization and applications. Ph.D. thesis (2015)

    Google Scholar 

  16. Gramacy, R.B.: Surrogates: Gaussian process modeling, design, and optimization for the applied sciences. Chapman and Hall/CRC (2020)

    Google Scholar 

  17. Gunjan, A., Bhattacharyya, S.: A brief review of portfolio optimization techniques. Artif. Intell. Rev., 1–40 (2022)

    Google Scholar 

  18. Jourdan, M., Mutnỳ, M., Kirschner, J., Krause, A.: Efficient pure exploration for combinatorial bandits with semi-bandit feedback. In: Algorithmic Learning Theory, pp. 805–849. PMLR (2021)

    Google Scholar 

  19. Lattimore, T., Crammer, K., Szepesvári, C.: Optimal resource allocation with semi-bandit feedback. arXiv preprint arXiv:1406.3840 (2014)

  20. Lattimore, T., Crammer, K., Szepesvári, C.: Linear multi-resource allocation with semi-bandit feedback. In: Advances in Neural Information Processing Systems, vol. 28 (2015)

    Google Scholar 

  21. Lattimore, T., Szepesvári, C.: Bandit algorithms. Cambridge University Press (2020)

    Google Scholar 

  22. Letham, B., Karrer, B., Ottoni, G., Bakshy, E.: Constrained Bayesian optimization with noisy experiments. Bayesian Anal. 14(2), 495–519 (2019)

    Article  MathSciNet  Google Scholar 

  23. Liu, B., Rao, Y., Lu, J., Zhou, J., Hsieh, C.J.: Multi-proxy Wasserstein classifier for image classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 8618–8626 (2021)

    Google Scholar 

  24. Neu, G., Bartók, G.: An efficient algorithm for learning with semi-bandit feedback. In: International Conference on Algorithmic Learning Theory, pp. 234–248. Springer (2013). https://doi.org/10.1007/978-3-642-40935-6_17

  25. Patriksson, M.: A survey on the continuous nonlinear resource allocation problem. Eur. J. Oper. Res. 185(1), 1–46 (2008)

    Article  MathSciNet  Google Scholar 

  26. Peyré, G., Cuturi, M., et al.: Computational optimal transport: with applications to data science. Found. Trends® Mach. Learn. 11(5-6), 355–607 (2019)

    Google Scholar 

  27. Ponti, A., Candelieri, A., Archetti, F.: A new evolutionary approach to optimal sensor placement in water distribution networks. Water 13(12), 1625 (2021)

    Article  Google Scholar 

  28. Ponti, A., Candelieri, A., Archetti, F.: A Wasserstein distance based multiobjective evolutionary algorithm for the risk aware optimization of sensor placement. Intell. Syst. Appl. 10, 200047 (2021)

    Google Scholar 

  29. Slivkins, A., et al.: Introduction to multi-armed bandits. Found. Trends® Mach. Learn. 12(1-2), 1–286 (2019)

    Google Scholar 

  30. Sonkar, S., Kharat, M.: A review on resource allocation and VM scheduling techniques and a model for efficient resource management in cloud computing environment. In: 2016 International Conference on ICT in Business Industry & Government (ICTBIG), pp. 1–7. IEEE (2016)

    Google Scholar 

  31. Srinivas, N., Krause, A., Kakade, S.M., Seeger, M.W.: Information-theoretic regret bounds for gaussian process optimization in the bandit setting. IEEE Trans. Inf. Theory 58(5), 3250–3265 (2012)

    Article  MathSciNet  Google Scholar 

  32. Thananjeyan, B., Kandasamy, K., Stoica, I., Jordan, M., Goldberg, K., Gonzalez, J.: Resource allocation in multi-armed bandit exploration: Overcoming sublinear scaling with adaptive parallelism. In: International Conference on Machine Learning, pp. 10236–10246. PMLR (2021)

    Google Scholar 

  33. Verma, A., Hanawal, M., Rajkumar, A., Sankaran, R.: Censored Semi-Bandits: a framework for resource allocation with censored feedback. In: Advances in Neural Information Processing Systems, vol. 32 (2019)

    Google Scholar 

  34. Villani, C.: The Wasserstein distances. In: Optimal Transport, pp. 93–111. Springer (2009). https://doi.org/10.1007/978-3-642-40935-6_17

  35. Vinothina, V.V., Sridaran, R., Ganapathi, P.: A survey on resource allocation strategies in cloud computing. Int. J. Adv. Comput. Sci. Appl. 3(6) (2012)

    Google Scholar 

  36. Wang, C., Long, S., Zeng, R., Lu, Y.: Imputation method for fetal heart rate signal evaluation based on optimal transport theory. SN Comput. Sci. 2(6), 1–12 (2021)

    Article  Google Scholar 

  37. Wang, S., Chen, W.: Thompson sampling for combinatorial semi-bandits. In: International Conference on Machine Learning, pp. 5114–5122. PMLR (2018)

    Google Scholar 

  38. Wen, Z., Kveton, B., Ashkan, A.: Efficient learning in large-scale combinatorial semi-bandits. In: International Conference on Machine Learning, pp. 1113–1122. PMLR (2015)

    Google Scholar 

  39. Williams, C.K., Rasmussen, C.E.: Gaussian Processes for Machine Learning, vol. 2. MIT Press, Cambridge, MA (2006)

    Google Scholar 

  40. Xidonas, P., Steuer, R., Hassapis, C.: Robust portfolio optimization: a categorized bibliographic review. Ann. Oper. Res. 292(1), 533–552 (2020)

    Article  MathSciNet  Google Scholar 

  41. Yousafzai, A., et al.: Cloud resource allocation schemes: review, taxonomy, and opportunities. Knowl. Inf. Syst. 50(2), 347–381 (2017)

    Article  Google Scholar 

  42. Zhang, D., et al.: Domain-oriented language modeling with adaptive hybrid masking and optimal transport alignment. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 2145–2153 (2021)

    Google Scholar 

  43. Zhang, J., Liu, T., Tao, D.: An optimal transport analysis on generalization in deep learning. IEEE Trans. Neural Netw. Learn. Syst. (2021)

    Google Scholar 

  44. Zhang, S., Huang, K., Yuan, Y.: Spare parts inventory management: a literature review. Sustainability 13(5), 2460 (2021)

    Article  Google Scholar 

  45. Ziukov, S.: A literature review on models of inventory management under uncertainty. Bus. Syst. Econ. 5(1), 26–35 (2015)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Milano-Bicocca, 20126, Milan, Italy

    Antonio Candelieri

Authors
  1. Antonio Candelieri
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Antonio Candelieri .

Editor information

Editors and Affiliations

  1. University of Calabria, Rende, Italy

    Yaroslav D. Sergeyev

  2. University of Calabria, Rende, Italy

    Dmitri E. Kvasov

  3. University of Calabria, Rende, Italy

    Annabella Astorino

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Candelieri, A. (2025). Resource Allocation via Bayesian Optimization: an Efficient Alternative to Semi-Bandit Feedback. In: Sergeyev, Y.D., Kvasov, D.E., Astorino, A. (eds) Numerical Computations: Theory and Algorithms. NUMTA 2023. Lecture Notes in Computer Science, vol 14476. Springer, Cham. https://doi.org/10.1007/978-3-031-81241-5_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-81241-5_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-81240-8

  • Online ISBN: 978-3-031-81241-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics