Abstract
Traditional approaches to solving stochastic optimal control problems involve dynamic programming, and solving certain optimality equations. When recast as stochastic programming problems, structural aspects such as convexity are retained, and numerical solution procedures based on decomposition and duality may be exploited. This paper explores a class of stationary, infinite-horizon stochastic optimization problems with discounted cost criterion. Constraints on both states and controls are permitted, and modeled in the objective function by allowing it to take infinite values. Approximating techniques are developed using variational analysis, and intuitive lower bounds are obtained via averaging the future. These bounds could be used in a finite-time horizon stochastic programming setting to find solutions numerically.
Similar content being viewed by others
References
Bertsekas, D. (1987). “Dynamic Programming: Deterministic and Stochastic Models.” Prentice-Hall, Englewood Cliffs.
Durrett, R. (1991). “Probability: Theory and Examples.” Brooks/Cole, Pacific Grove, CA.
Flåm, S. and R.J.-B. Wets. (1987). “Existence Results and Finite Horizon Approximates for Infinite Horizon Optimization Problems.” Econometrica 55, 1187–1209.
Flåm, S. and R.J-B. Wets. (1986). “Finite Horizon Approximates of in Finite Horizon Stochastic Programs.” Stochastic Optimization. 81, 337–350.
Gihman, I.I. and A.V. Skorohod. (1979). “Controlled Stochastic Processes.” Springer-Verlag, New York-Heidelberg.
Grinold, R.C. (1977). “Finite Horizon Approximates of Infinite Horizon Linear Programs.” Mathematical Programming 12, 1–17.
Hinderer, K.F. (1985). “On the Structure of Solutions of Stochastic Dynamic Programs.” Proceedings of the seventh conference on probability theory, pp. 173–182.
Puterman, M. (1994). “Markov Decision Processes: Discrete Stochastic Dynamic Programming.” Wiley, New York.
Rieder, U. and R. Zagst. (1994). “Monotonicity and Bounds for Convex Stochastic Control Models.” Z. Oper. Res. 39 (2), 187–207.
Rockafellar, R.T. and R.J.-B. Wets. (1998). “Variational Analysis.” Springer-Verlag.
Whitt, W. (1978). “Approximations of Dynamic Programs I.” Math. Oper. Res. 3 (3), 231–243.
Whitt, W. (1979). “Approximations of Dynamic Programs II.” Math. Oper. Res. 4 (2), 179–185.
Author information
Authors and Affiliations
Corresponding author
Additional information
Research supported in part by a grant of the National Science Foundation.
AMS Classification 46N10, 49N15, 65K10, 90C15, 90C46
Rights and permissions
About this article
Cite this article
Korf, L.A. Approximating infinite horizon stochastic optimal control in discrete time with constraints. Ann Oper Res 142, 165–186 (2006). https://doi.org/10.1007/s10479-006-6167-x
Issue Date:
DOI: https://doi.org/10.1007/s10479-006-6167-x