Abstract
We consider the problem of robotic planning under uncertainty. This problem may be posed as a stochastic optimal control problem, complete solution to which is fundamentally intractable owing to the infamous curse of dimensionality. We report the results of an extensive simulation study in which we have compared two methods, both of which aim to salvage tractability by using alternative, albeit inexact, means for treating feedback. The first is a recently proposed method based on a near-optimal “decoupling principle” for tractable feedback design, wherein a nominal open-loop problem is solved, followed by a linear feedback design around the open-loop. The second is Model Predictive Control (MPC), a widely-employed method that uses repeated re-computation of the nominal open-loop problem during execution to correct for noise, though when interpreted as feedback, this can only said to be an implicit form. We examine a much wider range of noise levels than have been previously reported and empirical evidence suggests that the decoupling method allows for tractable planning over a wide range of uncertainty conditions without unduly sacrificing performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
In the absence of a running cost, a criterion such as state deviation could be used. Since we aim to optimize the cost, a criterion based on cost seems more reasonable.
References
Andersson, J.A.E., Gillis, J., Horn, G., Rawlings, J.B., Diehl, M.: CasADi - a software framework for nonlinear optimization and optimal control. Math. Program. Comput. 11, 1–36 (2018)
Akrour, R., Abdolmaleki, A., Abdulsamad, H., Neumann, G.: Model free trajectory optimization for reinforcement learning. In: Proceedings of the ICML (2016)
Amato, C., Chowdhary, G., Geramifard, A., Ure, N.K., Kochenderfer, M.J.: Decentralized control of partially observable Markov decision processes. In: Proceedings of the IEEE International CDC, pp. 2398–2405 (2013)
Bertsekas, D.P.: Dynamic Programming and Optimal Control, vol. I and II. Athena Scientific, Cambridge (2012)
Boutilier, C.: Planning, learning and coordination in multiagent decision processes. In: Proceedings of the 6th Conference on Theoretical Aspects of Rationality and Knowledge, pp. 195–210. Morgan Kaufmann Publishers Inc. (1996)
Bryson, A.E., Ho, Y.C.: Applied Optimal Control. Allied Publishers, Buffalo (1967)
Chisci, L., Rossiter, J.A., Zappa, G.: Systems with persistent disturbances: predictive control with restricted contraints. Automatica 37, 1019–1028 (2001)
Fleming, W.H.: Stochastic control for small noise intensities. SIAM J. Control 9(3), 473–517 (1971)
Heemels, W., Johansson, K., Tabuada, P.: An introduction to event triggered and self triggered control. In: Proceedings of IEEE International CDC (2012)
Levine, S., Abbeel, P.: Learning neural network policies with guided search under unknown dynamics. In: Advances in NIPS (2014)
Levine, S., Vladlen, K.: Learning complex neural network policies with trajectory optimization. In: Proceedings of the ICML (2014)
Li, H., She, Y., Yan, W., Johansson, K.: Periodic event-triggered distributed receding horizon control of dynamically decoupled linear systems. In: Proceedings of the IFAC World Congress (2014)
Mayne, D.Q.: Model predictive control: recent developments and future promise. Automatica 50, 2967–2986 (2014)
Mayne, D.Q., Kerrigan, E.C., van Wyk, E.J., Falugi, P.: Tube based robust nonlinear model predictive control. Int. J. Robust Nonlinear Control 21, 1341–1353 (2011)
Mohamed, M.N.G., Chakravorty, S., Shell, D.A.: Experiments with tractable feedback in robotic planning under uncertainty: insights over a wide range of noise regimes (extended report). arXiv preprint arXiv:2002.10505 (2020)
Oliehoek, F.A.: Decentralized POMDPs. Reinforcement Learning (2012)
Oliehoek, F.A., Amato, C.: A Concise Introduction to Decentralized POMDPs. Springer, Cham (2016)
Parunandi, K.S., Chakravorty, S.: T-PFC: a trajectory-optimized perturbation feedback control approach. IEEE RA-L 4(4), 3457–3464 (2019)
Pynadath, D.V., Tambe, M.: The communicative multiagent team decision problem: analyzing teamwork theories and models. J. Artif. Intel. Res. 16, 389–423 (2002)
Rawlings, J.B., Mayne, D.Q.: Model Predictive Control: Theory and Design. Nob Hill, Madison (2015)
Rossiter, J.A., Kouvaritakis, B., Rice, M.J.: A numerically stable state space approach to stable predictive control strategies. Automatica 34, 65–73 (1998)
Seuken, S., Zilberstein, S.: Formal models and algorithms for decentralized decision making under uncertainty. Int. Conf. AAMAS 17(2), 190–250 (2008)
Theodorou, E., Tassa, Y., Todorov, E.: Stochastic differential dynamic programming. In: Proceedings of the ACC (2010)
Todorov, E., Tassa, Y.: Iterative local dynamic programming. In: Proceedings of the IEEE International Symposium on ADP and RL (2009)
Wächter, A., Biegler, L.T.: On the implementation of a primal-dual interior point filter line search algorithm for large-scale nonlinear programming. Mathematical Programming (2006)
Wang, R., Parunandi, K.S., Yu, D., Kalathil, D.M., Chakravorty, S.: Decoupled data based approach for learning to control nonlinear dynamical systems. CoRR, abs/1904.08361 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Gul Mohamed, M.N., Chakravorty, S., Shell, D.A. (2021). Experiments with Tractable Feedback in Robotic Planning Under Uncertainty: Insights over a Wide Range of Noise Regimes. In: LaValle, S.M., Lin, M., Ojala, T., Shell, D., Yu, J. (eds) Algorithmic Foundations of Robotics XIV. WAFR 2020. Springer Proceedings in Advanced Robotics, vol 17. Springer, Cham. https://doi.org/10.1007/978-3-030-66723-8_25
Download citation
DOI: https://doi.org/10.1007/978-3-030-66723-8_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-66722-1
Online ISBN: 978-3-030-66723-8
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)