Abstract
An iterative method, combining bisections and successive approximations, is proposed for computing intervals containing the Gittins indices. The intervals could be of a specified maximum length, or be merely disjoint. The first option gives approximations of the Gittins indices. The second option gives a ranking of indices, which in many applications is sufficient.
Zusammenfassung
Es wird ein iteratives Verfahren von Schranken von Gittins-Indizes vorgestellt. Der Algorithmus besteht aus Bisektion und einigen Schritten sukzessiver Approximation.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bean JC, Smith RL (1984) “Conditions for the existence of planning hoizons”, Math of OR 9: 391–401
Chen YR, Katehakis MN (1986) “Linear programming for the finite state multi-armed bandit problems”, Math of OR 11: 180–183
Denardo E (1982) Dynamic Programming, Prentice Hall
Flåm SD, Stensland G (1985) “Exploration and taxation: Some normative issues” Energy Economics: 234–240
Flåm SD, Olsen TE (1985) “Scheduling and taxation of resource deposits” The Energy Journal 6: 137–143
Gittins JC (1980) “Bandit processes and dynamic allocation indices” J Roy Satist Soc Ser B 42: 143–149
Gittins JC, Jones DM (1974) “A dynamic allocation index for the sequential design of experiments” in J Gani (editor) Progress in Statistics, North Holland, Amsterdam
Gittins JC, Glazerbrook KD (1977) “On Bayesian models in stochastic scheduling”, J Appl Prob 14: 556–565
Kallenberg LCM (1986) “A note on M.N. Katehakis' and Y.R. Chen's computation of the Gittins index”, Math of OR 11: 184–186
Katehakis MN, Veinott AF Jr (1987) “The multi-armed bandit problem: Decomposition and computation”, Math of OR 12: 262–268
Kramp M (May 1987) Models of Switching: Multi-Purpose Fleets in Fisheries, Multi-Armed Bandits and Gittins Indices, Ph D dissertation in OR, Department of Mathematical Sciences, University of Delaware
Lai TL, Ying Z (1988) “Open bandit processes and optimal scheduling of queueing networks” Adv Appl Prob 20: 447–472
Mandelbaum A (1986) “Discrete multi-armed bandits and multi-parameter processes”, Prob Theory Rel Fields 71: 129–147
Ross SM (1983) Introduction to Stochastic Dynamic Programming, Academic Press, New York
Varaiya P, Warland C, Buyukkoc C (1985) “Extensions of the multi-armed bandit problem: The discounted case“, IEEE Trans Autom Control AC-30: 426–439
Weitzmann ML (1979) “Optimal search for the best alternative”, Econometria 47: 641–654
White DJ (1987) “Elimination of non-optimal actions in Markov decisions processes” in Puterman (Editor), Dynamic Programming and its Applications, Academic Press, New York: 131–160
Whittle P (1980) “Multi-armed bandits and the Gittins index”, J Roy Statist Soc B42: 143–149
Whittle P (1981) “Arm acquiring bandits”, Ann Prob 9: 284–292
Whittle P (1982) Optimization over time, Vol I Capter 14, J. Wiley, New York
Author information
Authors and Affiliations
Additional information
Supported by the Royal Norwegian Council for Industrial and Scientific Research and by the National Science Foundation.
Rights and permissions
About this article
Cite this article
Ben-Israel, A., Flåm, S.D. A bisection/successive approximation method for computing Gittins indices. ZOR - Methods and Models of Operations Research 34, 411–422 (1990). https://doi.org/10.1007/BF01421548
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01421548