Abstract
In this work, we present a new algorithm that obtains a minimum-time solution in real-time to the optimal motion planning of a non-holonomic vehicle. The new algorithm is based on the combination of Cell-Mapping and reinforcement learning techniques. While the algorithm is performed on the vehicle, it learns its kinematics and dynamics from received experience with no need to have a mathematical model available. The algorithm uses a transformation of the cell-to-cell transitions in order to reduce the time spent in the knowledge of the vehicle’s parameters. The presented results have been obtained executing the algorithm with the real vehicle and generating different trajectories to specific goals.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Gómez, M., Martínez, T., Sánchez, S.: Optimal trajectory generation using the Simple Cell-Mapping method for wheeled mobile vehicles. In: Proc. of SAAEI, pp. 1–5. Universidad de Alcalá, Spain (2002)
Reeds, J.A., Shepp, R.A.: Optimal paths for a Car that Goes both Forwards and Backwards. Pacific J. Math. 145(2), 367–393 (1990)
Qin, B., Soh, Y.C., Xie, M., Wang, D.: Optimal trajectory generation for wheeled mobile robot. In: 5th International Conference on Computer Integrated Manufacturing, Singapore, March 28-30, 2000, vol. 1, pp. 434–444 (2000)
Barraquand, J., Latombe, J.C.: On nonholonomic mobile robots and optimal maneuvering. Revue d’Inteligence Artificielle 3(2), 77–103 (1989)
Hsu, S.: A discrete method of optimal control based upon the cell state space concept. J. Optim. Th. Appl. 46(4) (1985)
Papa, M., Tai, H.M., Shenoi, S.: Cell Mapping for Controller Design and Evaluation. IEEE Control Systems Magazine, 52–65 (April 1997)
Zufiria, P.J., Martínez-Marín, T.: Improved Optimal Control Methods based upon the Adjoining Cell Mapping Technique. J. Optimization Theory and Applications 118(3), 657–680 (2003)
Gómez, M., Martínez, T., Sánchez, S., Meziat, D.: Optimal control applied to Wheeled Mobile Vehicles. In: Proceedings of 2007 IEEE International Symposium on Intelligent Signal Processing, October 2007, pp. 83–88. Alcalá de Henares, Madrid (2007)
Gómez, M., Martínez, T., Sánchez, S., Meziat, D.: Optimal control applied for Wheeled Mobile Vehicles based on Cell Mapping techniques. In: IEEE Intelligent Vehicles Symposium (IV 2008), June 4-6, 2008, Conference Proceedings Book (2008)
Munos, R., Moore, A.: Variable Resolution Discretization in Optimal Control. Machine Learning Journal 49, 291–323 (2002)
CMUcam3: Open Source Programmable Embedded Color Vision Platform, http://www.cmucam.org
Sutton, R.S., Barto, A.: Reinforcement Learning: An introduction. MIT Press, Cambridge (1998)
Watkins Christopher, J.C.H.: Learning from Delayed Rewards. PhD thesis. Cambridge University, Cambridge (1989)
Kaebling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: A survey. Artificial Intelligence Research 4, 237–285 (1996)
Bryson, Ho: Applied optimal control: Optimization, estimation and control, book, p. 481. Hemisphere Pub. Corp. (1975) ISBN 0470114819
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gómez, M., Gayarre, L., Martínez-Marín, T., Sánchez, S., Meziat, D. (2009). Motion Planning of a Non-holonomic Vehicle in a Real Environment by Reinforcement Learning* . In: Cabestany, J., Sandoval, F., Prieto, A., Corchado, J.M. (eds) Bio-Inspired Systems: Computational and Ambient Intelligence. IWANN 2009. Lecture Notes in Computer Science, vol 5517. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02478-8_102
Download citation
DOI: https://doi.org/10.1007/978-3-642-02478-8_102
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-02477-1
Online ISBN: 978-3-642-02478-8
eBook Packages: Computer ScienceComputer Science (R0)