Abstract
This paper proposes a novel self-learning PD (Proportional- Derivative) control method for mobile robot path-tracking problems. In the self-learning PD control method, a reinforcement-learning (RL) module is used to automatically fine-tune the PD coefficients with only evaluative feedback. The optimization of the PD coefficients is modeled as a Markov decision problem (MDP) with continuous state space. Using an improved AHC (Adaptive Heuristic Critic) learning control method based on recursive least-squares algorithms, the near-optimal control policies of the MDP are approximated efficiently. Besides its simplicity, the self-learning PD controller can be adaptive to uncertainties in the environment as well as the mobile robot dynamics. Simulation and experimental results on a real mobile robot illustrate the effectiveness of the proposed method.
Supported by the National Natural Science Foundation of China Under Grant 60303012, 0234030, & 60225015
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Choi, S.B.: The Design of a Look-Down Feedback Adaptive Controller for the Lateral Control of Front-Wheel-Steering Autonomous Highway Vehicles. In: Proceedings of the American Control Conference, Albuquerque, New Mexico, vol. 3, pp. 1603–1607 (1997)
Shin, D.H., Singh, S., Lee, J.J.: Explicit Path Tracking by Autonomous Vehicles. Robotica 10, 539–554 (1992)
Aguilar, M., et al.: Robust Path Following Control with Exponential Stability for Mobile Robots. In: IEEE International Conference on Robotics and Automation, Leuven, Belgium, vol. 4, pp. 3279–3284 (1998)
Fierro, R., Lewis, F.L.: Control of a Nonholonomic Mobile Robot Using Neural Networks. IEEE Trans. on Neural Networks 9, 589–600 (1998)
Sanchez, O., Ollero, A., Heredia, G.: Adaptive Fuzzy Control for Automatic Path Tracking of Outdoor Mobile Robots: Application to Romeo 3R. In: IEEE International Conference on Fuzzy Systems, Barcelona, Spain, vol. 1, pp. 593–599 (1997)
Kaelbling, L.P., et al.: Reinforcement Learning: a Survey. Journal of Artificial Intelligence Research 12, 237–285 (1996)
Xu, X., He, H., Hu, D.: Efficient Reinforcement Learning using Recursive Leastsquares Methods. Journal of Artificial Intelligence Research 16, 259–292 (2002)
Lee, C.H., Teng, C.C.: Tuning of PID Controllers for Stable and Unstable Processes with Specifications on Gain and Phase Margins. International Journal of Fuzzy Systems 3(1), 346–355 (2001)
Campion, G., et al.: Structural Properties and Classification of Dynamic Models of Wheeled Mobile Robots. IEEE Trans. on Robotics and Automation 12(1), 47–62 (1996)
Puterman, M.L.: Markov Decision Processes. John Wiley and Sons, New York (1994)
Barto, A.G., Sutton, R.S., Anderson, C.W.: Neuron-like Adaptive Elements that can Solve Difficult Learning Control Problems. IEEE Transactions on System, Man and Cybernetics 13, 834–846 (1983)
Berenji, H.R., Khedkar, P.: Learning and Tuning Fuzzy Logic Controllers Through Reinforcement. IEEE Trans.on Neural Networks 3(5), 724–740 (1992)
Sutton, R.: Learning to Predict by the Method of Temporal Differences. Machine Learning 3(1), 9–44 (1988)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Xu, X., Wang, X., Hu, D. (2004). Mobile Robot Path-Tracking Using an Adaptive Critic Learning PD Controller. In: Yin, FL., Wang, J., Guo, C. (eds) Advances in Neural Networks - ISNN 2004. ISNN 2004. Lecture Notes in Computer Science, vol 3174. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-28648-6_5
Download citation
DOI: https://doi.org/10.1007/978-3-540-28648-6_5
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-22843-1
Online ISBN: 978-3-540-28648-6
eBook Packages: Springer Book Archive