Abstract
This paper presents a performance comparison of mobile robot obstacle avoidance between using Deep Reinforcement Learning (DRL) and two classical Reinforcement Learning (RL). For the DRL-based method, Deep Q-Learning (DQN) algorithm was used whereas for the RL-based method, Q-Learning and Sarsa algorithms were used. In our experiments, we have used the extended OpenAI Gym ToolKit to compare the performances of DQN, Q-Learning, and Sarsa algorithms in both simulated and real-world environments. Turtlebot3 Burger was used as the mobile robot hardware to evaluate the performance of the RL models in the real-world environment. The average rewards, episode steps, and rate of successful navigation were used to compare the performance of the navigation ability of the RL agents. Based on the simulated and real-world results, DQN has performed significantly better than both Q-Learning and Sarsa. It has achieved 100% success rates during the simulated and real-world tests.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Siegwart, R., Nourbakhsh, I.R., Scaramuzza, D.: Introduction to Autonomous Mobile Robots. MIT Press, Cambridge (2011)
Raguraman, S.M., Tamilselvi, D., Shivakumar, N.: Mobile robot navigation using fuzzy logic controller. In: 2009 International Conference on Control, Automation, Communication and Energy Conservation, pp. 1–5. IEEE (2009)
Ge, S.S., Cui, Y.J.: Dynamic motion planning for mobile robots using potential field method. Auton. Robots 13(3), 207–222 (2002)
Hu, Y., Yang, S.X.: A knowledge based genetic algorithm for path planning of a mobile robot. In: IEEE International Conference on Robotics and Automation 2004, Proceedings, ICRA 2004, vol. 5, pp. 4350–4355. IEEE (2004)
Watkins, C.J.C.H., Dayan, P.: Q-learning. Mach. Learn. 8(3), 279–292 (1992)
Reinforcement learning: Q-Learning vs Sarsa. http://www.cse.unsw.edu.au/cs9417ml/RL1/algorithms.html. Accessed 14 June 2021
Watkins, C.: Learning from Delayed Rewards. Cambridge, England (1989)
Rummery, G.A., Niranjan, M.: On-line Q-learning using connectionist systems, vol. 37. University of Cambridge, Department of Engineering Cambridge, UK (1994)
Reinforcement learning: Sarsa vs Qlearn. https://studywolf.wordpress.com/2013/07/01/reinforcement-learning-sarsa-vs-q-learning. Accessed 14 June 2021
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)
Sewak, M.: Deep Reinforcement Learning Frontier of Artificial Intelligence, p. 215 (2019). ISBN: 9789811382840
A hands-on introduction to deep Q-learning using OpenAI Gym in Python. https://www.analyticsvidhya.com/blog/2019/04/introduction-deep-q-learning-python/. Accessed 14 June 2021
Zamora, I., et al.: Extending the OpenAI Gym for robotics: a toolkit for reinforcement learning using ROS and Gazebo. arXiv: 1608.05742 [cs.RO] (2017)
Wang, Y.-H., Li, T.-H.S., Lin, C.-J.: Backward Q-learning: the combination of Sarsa algorithm and Q-learning. Eng. Appl. Artif. Intell. 26(9), 2184–2193 (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Anas, H., Ong, W.H., Malik, O.A. (2022). Comparison of Deep Q-Learning, Q-Learning and SARSA Reinforced Learning for Robot Local Navigation. In: Kim, J., et al. Robot Intelligence Technology and Applications 6. RiTA 2021. Lecture Notes in Networks and Systems, vol 429. Springer, Cham. https://doi.org/10.1007/978-3-030-97672-9_40
Download citation
DOI: https://doi.org/10.1007/978-3-030-97672-9_40
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-97671-2
Online ISBN: 978-3-030-97672-9
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)