Abstract
The one-step anticipatory algorithm (1s-AA) is an online algorithm making decisions under uncertainty by ignoring future non-anticipativity constraints. It makes near-optimal decisions on a variety of online stochastic combinatorial problems in dynamic fleet management, reservation systems, and more.
Here we consider applications in which 1s-AA is not as close to the optimum and propose Amsaa, an anytime multi-step anticipatory algorithm. Amsaa combines techniques from three different fields to make decisions online. It uses the sampling average approximation method from stochastic programming to approximate the problem; solves the resulting problem using a search algorithm for Markov decision processes from artificial intelligence; and uses a discrete optimization algorithm for guiding the search.
Amsaa was evaluated on a stochastic project scheduling application from the pharmaceutical industry featuring endogenous observations of the uncertainty. The experimental results show that Amsaa significantly outperforms state-of-the-art algorithms on this application under various time constraints.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Barto, A.G., Bradtke, S.J., Singh, S.P.: Learning to act using real-time dynamic programming. Artificial Intelligence 72(1), 81–138 (1995)
Bent, R., Van Hentenryck, P.: Waiting and Relocation Strategies in Online Stochastic Vehicle Routing. In: Proceedings of the 20th Int. Joint Conf. on A.I. (IJCAI 2007) (January 2007)
Bent, R., Van Hentenryck, P.: Scenario-Based Planning for Partially Dynamic Vehicle Routing Problems with Stochastic Customers. Operations Research 52(6) (2004)
Bonet, B., Geffner, H.: Faster heuristic search algorithms for planning with uncertainty and full feedback. In: IJCAI, pp. 1233–1238 (2003)
Bonet, B., Geffner, H.: Learning depth-first search: A unified approach to heuristic search in deterministic and non-deterministic settings, and its application to mdps. In: ICAPS (2006)
Choi, J., Realff, M.J., Lee, J.H.: Dynamic prog. in a heuristically confined state space: A stochastic resource-constrained project scheduling appplication. Computers and Chemical Engineering (2004)
Dempster, M.A.H.: Sequential Importance Sampling Algorithms for Dynamic Stochastic Programming. Journal of Mathematical Sciences 133, 1422–1444 (2006)
Dooms, G., Van Hentenryck, P.: Gap Reduction Techniques for Online Stochastic Project Scheduling. In: CPAIOR 2008 (2008)
Dupacova, J., Consigli, G., Wallace, S.W.: Scenarios for multistage stochastic programs. Annals of Operations Research (2000)
Goel, V., Grossmann, I.E.: A class of stochastic programs with decision dependent uncertainty. Math. Program 108(2-3), 355–394 (2006)
Hansen, E.A., Zilberstein, S.: LAO: A heuristic-search algorithm that finds solutions with loops. Artificial Intelligence 129(1-2), 35–62 (2001)
Kearns, M., Mansour, Y., Ng, A.: A Sparse Sampling Algorithm for Near-Optimal Planning in Large Markov Decision Processes. In: IJCAI 1999, pp. 1231–1324 (1999)
McMahan, H.B., Likhachev, M., Gordon, G.J.: Bounded real-time dynamic programming: RTDP with monotone upper bounds and performance guarantees. In: ICML, pp. 569–576 (2005)
Mercier, L., Van Hentenryck, P.: Performance Analysis of Online Anticipatory Algorithms for Large Multistage Stochastic Programs. In: Proceedings of the 20th Int. Joint Conf. on AI (IJCAI) (2007)
Parkes, D., Duong, A.: An Ironing-Based Approach to Adaptive Online Mechanism Design in Single-Valued Domains. In: AAAI 2007, Vancouver, Canada, pp. 94–101 (2007)
Ruszczynski, A., Shapiro, A. (eds.): Stochastic Programming. Hanbooks in Operations Research and Management Series, vol. 10. Elsevier, Amsterdam (2003)
Thomas, M., Szczerbicka, H.: Evaluating Online Scheduling Techniques in Uncertain Environments. In: Proceedings of the 3rd Multidisciplinary International Scheduling Conference (MISTA 2007) (2007)
Van Hentenryck, P., Bent, R.: Online Stochastic Combinatorial Optimization. The MIT Press, Cambridge (2006)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Mercier, L., Van Hentenryck, P. (2008). Amsaa: A Multistep Anticipatory Algorithm for Online Stochastic Combinatorial Optimization. In: Perron, L., Trick, M.A. (eds) Integration of AI and OR Techniques in Constraint Programming for Combinatorial Optimization Problems. CPAIOR 2008. Lecture Notes in Computer Science, vol 5015. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-68155-7_15
Download citation
DOI: https://doi.org/10.1007/978-3-540-68155-7_15
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-68154-0
Online ISBN: 978-3-540-68155-7
eBook Packages: Computer ScienceComputer Science (R0)