Abstract
A partially observed stochastic system is described by a discrete time pair of Markov processes. The observed state process has a transition probability that is controlled and depends on a hidden Markov process that also can be controlled. The hidden Markov process is completely observed in a closed set, which in particular can be the empty set and only observed through the other process in the complement of this closed set. An ergodic control problem is solved by a vanishing discount approach. In the case when the transition operators for the observed state process and the hidden Markov process depend on a parameter and the closed set, where the hidden Markov process is completely observed, is nonempty and recurrent an adaptive control is constructed based on this family of estimates that is almost optimal.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bielecki TR, Pliska SR. (1999). Risk sensitive dynamic asset management. JAMO 39:337–360
Bielecki TR, Rutkowski M. (2002). Credit risk: modelling, valuation and hedging. Springer, Berlin Heidelberg New York
Borkar VS. (2003). Dynamic programming for ergodic control with partial observation. Stoch Proc Appl 103:293–310
Borkar VS, Budhiraja A. (2004). Further remark on dynamic programming for partially observed Markov processes. Stoch Proc Appl 112(1):79–93
Borkar VS, Mundra SM. (1998). Bayesian parameter estimation and adaptive control of Markov processes with time - averaged cost. Appl Math 25(3):339–358
Brennan MJ, Schwarz ES, Lagnado R. (1997). Stategic asset allocation. J Econ Dyn Control 21:1377–1403
Di Masi GB, Stettner L. (1995). Bayesian ergodic control of discrete time Markov processes. Stoch Stoch Rep 54:301–316
Doob JL. (1953). Stochastic processes. Wiley, New York
Duncan TE, Pasik-Duncan B, Stettner L. (1998). Adaptive control of a partially observed discrete time Markov process. Appl Math Optim 37:269–293
Elliott RJ, Aggoun L, Moore JB. (1996). Hidden Markov models: estimation and control. Springer, Berlin Heidelberg New York
Hernandez-Lerma D, Lasserre JB. (1996). Discrete time Markov control processes: basic optimality criteria. Springer, Berlin Heidelberg New York
Kazangey T, Sworder DD (1971) Effective federal policies for regulating residential housing. In: Proceedings of summer computer simulation conference, Los Angeles, pp 1120–1128
Korn R. (1997). Optimal portfolios. stochastic models for optimal investment and risk management in continuous time. World Scientific, Singapore
Mariton M. (1990). Jump linear systems in automatic control. Marcel Dekker, New York and Bassel
Montgomery RC. (1983). Reliability considerations in placement of control systems components. In: Proceedings of AIAA guidance and control conference, Gatlinburg
Royden HL. (1968). Real analysis. MacMillan, New York
Runggaldier WJ, Stettner L. (1994). Approximations of discrete time partially observed control problems. Giardini, Pisa
Sethi SP, Zhang Q. (1994). Hierarchical decision making in stochastic manufacturing systems. Birkhäuser, Boston Basel Berlin
Stettner L. (1993). Ergodic control of partially observed Markov processes with equivalent transition probabilities. Appl Math 22:25–38
Stettner L. (2002). Bayesian adaptive control of discrete time partially observed Markov processes. In: Proceedings of workshop stochastic theory and control, Lect Notes Control Info Sci 280:435–446
Willsky AS, Levy BC. (1979). Stochastic stability research for complex power systems. Lab. Inf. Decision Systems, Mass. Inst. Technology, report no. ET-76-C-01-2295.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Duncan, T.E., Pasik-Duncan, B. & Stettner, L. Ergodic and adaptive control of hidden Markov models. Math Meth Oper Res 62, 297–318 (2005). https://doi.org/10.1007/s00186-005-0010-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00186-005-0010-z