Dynamic productivity improvement in a model with multiple processes

Abstract

We study the situation where there are a number of on-going production processes each yielding a state-dependent standard reward in discrete time. At each time step one may select at most one of these processes for improvement; the selected process will yield a state-dependent non-standard reward (or cost) at that time step and change its state according to a Markov chain. We show that this model can be cast into a bandit formulation with constructed rewards and we characterize the optimal policy. Finally, we present a numerical example.