Abstract
Decision making is one of the central problems in artificial intelligence and specifically in robotics.In most cases this problem comes with uncertainty both in data received by the decision maker/agent and in the actions performed in the environment. One effective method to solve this problem is to model the environment and the agent as a Partially Observable Markov Decision Process (POMDP). A POMDP has a wide range of applications such as: Machine Vision , Marketing, Network troubleshooting, Medical diagnosis etc.
We consider a new technique, called Recursive Point Filter (RPF) based on Incremental Pruning (IP) POMDP solver to introduce an alternative method to Linear Programming (LP) filter. It identifies vectors with maximum value in each witness region known as dominated vectors, the dominated vectors at each of these points would then be part of the upper surface. RPF takes its origin from computer graphic.
In this paper, we tested this new technique against the popular Incremental Pruning (IP) exact solution method in order to measure the relative speed and quality of our new method. We show that a high-quality POMDP policy can be found in lesser time in some cases. Furthermore, RPF has solutions for several POMDP problems that LP could not converge to in 24 hours.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Monahan, G.E.: A survey of partially observable Markov decision processes. Management Science 28(1), 1–16 (1982)
Smallwood, R.D., Sondik, E.J.: The optimal control of partially observable Markov processes over a finite horizon. Operations Research 21, 1071–1088 (1973)
Caines, P.E.: Linear Stochastic Systems. John Wiley, New York (April 1988)
Spaan, M.T.J.: Cooperative active perception using POMDPs. In: AAAI 2008 Workshop on Advancements in POMDP Solvers (July 2008)
Goldsmith, J., Mundhenk, M.: Complexity issues in Markov decision processes. In: Proceedings of the IEEE Conference on Computational Complexity. IEEE, Los Alamitos (1998)
Littman, M.L., Cassandra, A.R., Kaelbling, L.P.: Efficient dynamic-programming updates in partially observable markov decision process. Technical report, Brown University, Providence, RI (1996)
Cassandra, A., Littman, M.L., Zhang, N.L.: Incremental pruning: A simple, fast, exact algorithm for partially observable Markov decision processes. In: Proceedings of the Thirteenth Annual Conference on Uncertainty in Artificial Intelligence (1997)
Pineau, J., Gordon, G., Thrun, S.: Point-based value iteration: An anytime algorithm for POMDPs. In: Proceedings of the International Joint Conference on Artificial Intelligence, Acapulco, Mexico (2003)
Smith, T., Simmons, R.G.: Heuristic search value iteration for POMDPs. In: Proc. Int. Conf. on Uncertainty in Artificial Intelligence, UAI (2004)
Spaan, M.T.J., Vlassis, N.: Randomized point-based value iteration for POMDPs. Journal of Artificial Intelligence Research 24, 195–220 (2005)
Thrun, S., Burgard, W., Fox, D.: Probablistic Robotics. The MIT Press, Cambridge (June 2006)
Pyeatt, L.D., Howe, A.E.: A parallel algorithm for POMDP solution. In: Proceedings of the Fifth European Conference on Planning, Durham, UK, pp. 73–83 (September 1999)
Cassandra, A.R.: Exact and Approximate Algorithms for Partially Observable Markov Decision Process. PhD thesis, Brown University, Department Of Computer Science (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Naser-Moghadasi, M. (2010). A New Graphical Recursive Pruning Method for the Incremental Pruning Algorithm. In: Sidorov, G., Hernández Aguirre, A., Reyes GarcÃa, C.A. (eds) Advances in Artificial Intelligence. MICAI 2010. Lecture Notes in Computer Science(), vol 6437. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-16761-4_21
Download citation
DOI: https://doi.org/10.1007/978-3-642-16761-4_21
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-16760-7
Online ISBN: 978-3-642-16761-4
eBook Packages: Computer ScienceComputer Science (R0)