Abstract
We present an approximation method that solves a class of Decentralized hybrid Markov Decision Processes (DEC-HMDPs). These DEC-HMDPs have both discrete and continuous state variables and represent individual agents with continuous measurable state-space, such as resources. Adding to the natural complexity of decentralized problems, continuous state variables lead to a blowup in potential decision points. Representing value functions as Rectangular Piecewise Constant (RPWC) functions, we formalize and detail an extension to the Coverage Set Algorithm (CSA) [1] that solves transition independent DEC-HMDPs with controlled error. We apply our algorithm to a range of multi-robot exploration problems with continuous resource constraints.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Becker, R., Zilberstein, S., Lesser, V., Goldman, C.V.: Solving transition independent decentralized markov decision processes. Journal of Artificial Intelligence Research 22 (2004)
Bernstein, D.S., Givan, R., Immerman, N., Zilberstein, S.: The complexity of decentralized control of markov decision processes. Mathematics of Operations Research 27(4) (2002)
Hansen, E., Bernstein, D.S., Zilberstein, S.: Dynamic programming for partially observable stochastic games. In: Proceedings of the Nineteenth National Conference on Artificial Intelligence (2004)
Feng, Z., Dearden, R., Meuleau, N., Washington, R.: Dynamic programming for structured continuous Markov decision problems. In: Proceedings of the Twentieth International Conference on Uncertainty In Artificial Intelligence, pp. 154–161 (2004)
Friedman, J.H., Bentley, J.L., Finkel, R.A.: An algorithm for finding best matches in logarithmic expected time. ACM Trans. Mathematical Software 3(3), 209–226 (1977)
Bresina, J., Dearden, R., Meuleau, N., Ramakrishnan, S., Smith, D., Washington, R.: Planning under continuous time and uncertainty: A challenge in ai. In: Proceedings of the Eighteenth International Conference on Uncertainty In Artificial Intelligence (2002)
Li, L., Littman, M.L.: Lazy approximation for solving continuous finite-horizon mdps. In: Proceedings of the Twentieth National Conference on Artificial Intelligence (2005)
Marecki, J., Koenig, S., Tambe, M.: A fast analytical algorithm for solving markov decision processes with real-valued resources. In: Proceedings of the Twentieth International Joint Conference on Artificial Intelligence (2007)
Mausam, Benazera, E., Brafman, R., Meuleau, N., Hansen, E.A.: Planning with continuous resources in stochastic domains. In: Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence, pp. 1244–1251 (2005)
Naylor, B., Amanatides, J., Thibault, W.: Merging bsp trees yields polyhedral set operations. In: SIGGRAPH 1990. Computer Graphics (1990)
Smith, D.: Choosing objectives in over-subscription planning. In: Proceedings of the Fourteenth International Conference on Automated Planning and Scheduling, pp. 393–401 (2004)
Szer, D., Charpillet, F.: Point-based dynamic programming for dec-pomdps. In: Proceedings of the Twenty First National Conference on Artificial Intelligence (2006)
Szer, D., Charpillet, F., Zilberstein, S.: Maa*: A heuristic search algorithm for solving decentralized pomdps. In: Proceedings of the Twentieth National Conference on Artificial Intelligence (2005)
van den Briel, M., Do, M.B., Sanchez, R., Kambhampati, S.: Effective approaches for partial satisfation (over-subscription) planning. In: Proceedings of the Nineteenth National Conference on Artificial Intelligence, pp. 562–569 (2004)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Benazera, E. (2007). Solving Decentralized Continuous Markov Decision Problems with Structured Reward. In: Hertzberg, J., Beetz, M., Englert, R. (eds) KI 2007: Advances in Artificial Intelligence. KI 2007. Lecture Notes in Computer Science(), vol 4667. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74565-5_26
Download citation
DOI: https://doi.org/10.1007/978-3-540-74565-5_26
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74564-8
Online ISBN: 978-3-540-74565-5
eBook Packages: Computer ScienceComputer Science (R0)