Abstract
Many methods for planning under uncertainty operate in the belief space, i.e., the set of probability distributions over states. Although the problem is computationally hard, recent advances have shown that belief-space planning is becoming practical for many medium size problems. Some of the most successful methods utilize sampling and often rely on distances between beliefs to partially guide the search process. This paper deals with the question of what is a suitable distance function for belief space planning, which despite its importance remains unanswered. This work indicates that the rarely used Wasserstein distance (also known as Earth Mover’s Distance (\(\mathtt{EMD} \))) is a more suitable metric than the commonly used \(\mathtt{L1} \) and Kullback–Leibler (\(\mathtt{KL} \)) for belief-space planning. Simulation results on Non-Observable Markov Decision Problems, i.e., the simplest class of belief-space planning, indicate that as the problem becomes more complex, the differences on the effectiveness of different distance functions become quite prominent. In fact, in state spaces with more than 4 dimensions, by just replacing \(\mathtt{L1} \) or \(\mathtt{KL} \) distance with \(\mathtt{EMD} \), the problems become from virtually unsolvable to solvable within a reasonable time frame. Furthermore, preliminary results on Partially Observable Markov Decision Processes indicate that point-based solvers with \(\mathtt{EMD} \) use a smaller number of samples to generate policies with similar qualities, compared to those with \(\mathtt{L1} \) and \(\mathtt{KL} \). This paper also shows that \(\mathtt{EMD} \) carries the Lipschitz continuity of the state’s cost function to Lipschitz continuity of the expected cost of the beliefs. Such a continuity property is often critical for convergence to optimal solutions.
Zakary Littlefield is supported by the NASA Space Technology Research Fellowship (NNX13AL71H). Dimitri Klimenko is supported by the Australian Postgraduate Award. Work by the Rutgers authors has been supported by NSF CCF:13307893, NSF IIS:1451737.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Bai, H., Hsu, D., Kochenderfer, M.J., Lee, W.S.: Unmanned aircraft collision avoidance using continuous-state POMDPs. Robot. Sci. Syst. 1, 1–8 (2012)
Bai, H.Y., Hsu, D., Lee, W.S., Ngo, V.A.: Monte Carlo value iteration for continuous-state POMDPs. In: Hsu, D., et al. (eds.) WAFR. Springer, Heidelberg (2010)
Bry, A., Roy, N.: Rapidly-exploring random belief trees for motion planning under uncertainty. In: ICRA (2011)
Choset, H., Lynch, K.M., Hutchinson, S., Kantor, G., Burgard, W., Kavraki, L.E., Thrun, S.: Principles of Robot Motion. The MIT Press, Cambridge (2005)
Dudley, R.M.: Real Analysis and Probability. Cambridge University Press, Cambridge (2002)
Horowitz, M., Burdick, J.: Interactive non-prehensile manipulation for grasping via POMDPs. In: Proceedings of the IEEE International Conference on Robotics and Automation (2013)
Karaman, S., Frazzoli, E.: Sampling-based algorithms for optimal motion planning. IJRR 30(7), 846–894 (2011)
Kneebone, M., Dearden, R.: Navigation planning in probabilistic roadmaps with uncertainty. In: Proceedings of the International Conference on Automated Planning and Scheduling (2009)
Koval, M.C., Pollard, N.S., Srinivasa, S.: Pre- and post-contact policy decomposition for planar contact manipulation under uncertainty. In: RSS (2014a)
Koval, M., Pollard, N., Srinivasa, S.: Pre- and post-contact policy decomposition for planar contact manipulation under uncertainty. In: RSS, Berkeley, USA, July 2014b
Kurniawati, H., Patrikalakis. N.M.: Point-based policy transformation: adapting policy to changing POMDP models. In: WAFR (2012)
Kurniawati, H., Hsu, D., Lee, W.S.: SARSOP: Efficient point-based POMDP planning by approximating optimally reachable belief spaces. In: RSS (2008)
Kurniawati, H., Bandyopadhyay, T., Patrikalakis, N.M.: Global motion planning under uncertain motion, sensing, and environment map. Auton. Robots Spec. Issue RSS 30(3), 2012 (2011)
Kurniawati, H., Du, Y., Hsu, D., Lee, W.S.: Motion planning under uncertainty for robotic tasks with long time horizons. IJRR 30(3), 308–323 (2011)
Li, Y., Littlefield, Z., Bekris, K.E.: Sparse methods for efficient asymptotically optimal kinodynamic planning. In: WAFR (2014)
Li, Y., Littlefield, Z., Bekris, K.E.: Asymptotically optimal sampling-based kinodynamic planning. In: IJRR (2015), accepted to appear
Littlefield, Z., Li, Y., Bekris, K.E.: Efficient sampling-based motion planning with asymptotic near-optimality guarantees for systems with dynamics. In: IROS, Tokyo Big Sight, Japan (2013)
Melchior, N., Simmons, R.: Particle RRT for path planning with uncertainty. In: ICRA (2007)
Papadimitriou, C., Tsitsiklis, J.N.: The complexity of Markov decision processes. JMOR 12(3), 441–450 (1987)
Papadopoulos, G., Kurniawati, H., Patrikalakis, N.M.: Analysis of asymptotically optimal sampling-based motion planning algorithms for Lipschitz continuous dynamical systems, 12 May 2014. http://arxiv.org/abs/1405.2872
Paranjape, A.A., Meier, K.C., Shi, X., Chung, S.-J., Hutchinson, S.: Motion primitives and 3-D path planning for fast flight through a forest. In: IROS (2013)
Pele, O., Werman, M.: Fast and robust earth mover’s distances. In: ICCV (2009)
Pineau, J., Gordon, G., Thrun, S.: Pointspsbased value iteration: an anytime algorithm for POMDPs. In: IJCAI (2003)
Platt, R., Kaelbling, L., Lozano-Perez, T., Tedrake, R.: Non-gaussian belief space planning: correctness and complexity. In: ICRA (2012)
Prentice, S., Roy, N.: The belief roadmap: efficient planning in belief space by factoring the covariance. IJRR 28(11–12), 1448–1465 (2009)
Shirdhonkar, S., Jacobs, D.: Approximate earth mover’s distance in linear time. In: IEEE Conference on Computer Vision and Pattern Recognition (2008)
Smith, T., Simmons, R.: Point-based POMDP algorithms: improved analysis and implementation. In: Proceedings of the Uncertainty in Artificial Intelligence (2005)
Spaan, M.T.J., Vlassis, N.: Perseus: randomized point-based value iteration for POMDPs. J. Artif. Intell. Res. 24, 195–220 (2005)
Thrun, S.: Monte-Carlo POMDPs. In: NIPS (2000)
Urmson, C., Simmons, R.: Approaches for heuristically biasing RRT growth. In: IROS, pp. 1178–1183 (2003)
van den Berg, J., Abbeel, P., Goldberg, K.: LQG-MP: optimized path planning for robots with motion uncertainty and imperfect state information. In: RSS (2010)
van den Berg, J., Patil, S., Aterovitz, R., Abbeel, P., Goldberg, K.: Planning, sensing, and control of steerable needles. In: Workshop on the Algorithmic Foundation of Robotics (2010)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this chapter
Cite this chapter
Littlefield, Z., Klimenko, D., Kurniawati, H., Bekris, K.E. (2018). The Importance of a Suitable Distance Function in Belief-Space Planning. In: Bicchi, A., Burgard, W. (eds) Robotics Research. Springer Proceedings in Advanced Robotics, vol 3. Springer, Cham. https://doi.org/10.1007/978-3-319-60916-4_39
Download citation
DOI: https://doi.org/10.1007/978-3-319-60916-4_39
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-60915-7
Online ISBN: 978-3-319-60916-4
eBook Packages: EngineeringEngineering (R0)