Abstract
The focus of this paper is the framework of partially observable Markov decision processes (POMDPs) and its role in modeling and solving complex dynamic decision problems in stochastic and partially observable medical domains. The paper summarizes some of the basic features of the POMDP framework and explores its potential in solving the problem of the management of the patient with chronic ischemic heart disease.
This research was supported by the grant 1T15LM07092 from the National Library of Medicine. Peter Szolovits and William Long have provided valuable feedback on early versions of the paper and Hamish Fraser has helped with the ischemic heart disease example.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
References
K.J. Astrom. Optimal control of Markov decision processes with incomplete state estimation. Journal of Mathematical Analysis and Applications, 10, pp. 174–205, 1965
A.R. Cassandra. Optimal policies for partially observable Markov decision processes. Brown University, Technical report CS-94-14, 1994.
M. Hauskrecht. Planning and control in stochastic domains with imperfect information. MIT EECS PhD thesis proposal, August 1996, 133 pages.
M. Hauskrecht. Dynamic decision making in stochastic partially observable medical domains. AAAI Spring symposium, pp. 69–72, 1996.
R. Hovorka et.al., Causal probabilistic network modelling — An illustration of its role in the management of chronic diseases. IBM Systems Journal, 31:4, pp.635–648, 1992.
T.-Y. Leong. An integrated approach to dynamic decision making under uncertainty. MIT/LCS/TR-631, 1994.
W.S. Lovejoy. A survey of algorithmic methods for partially observed Markov decision processes. Annals of Operations Research, 28, pp. 47–66, 1991.
C.H. Papadimitriou, J.N. Tsitsiklis. The complexity of Markov decision processes. Mathematics of Operations Research, 12:3, pp. 441–450, 1987.
R.D. Smallwood, E.J. Sondik. The optimal control of partially observable processes over a finite horizon. Operations Research, 21, pp. 1071–1088.
J.B. Wong et.al., Myocardial revascularization for chronic stable angina. Annals of Internal Medicine, 113 (1), pp. 852–871, 1990.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1997 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hauskrecht, M. (1997). Dynamic decision making in stochastic partially observable medical domains: Ischemic heart disease example. In: Keravnou, E., Garbay, C., Baud, R., Wyatt, J. (eds) Artificial Intelligence in Medicine. AIME 1997. Lecture Notes in Computer Science, vol 1211. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0029462
Download citation
DOI: https://doi.org/10.1007/BFb0029462
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-62709-8
Online ISBN: 978-3-540-68448-0
eBook Packages: Springer Book Archive