Abstract
Aiming at the problem of the slow speed of reinforcement learning, a tentative Q learning algorithm is proposed. By improving the number of exploration in each learning iteration and the updating method of Q table, tentative Q learning algorithm accelerates the learning speed and ensures the balance between exploration and exploitation. Finally, the feasibility and effectiveness of the algorithm are proved by the experiment of robot path planning.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Kober, J., Bagnell, J.A., et al.: Reinforcement learning in robotics: a survey. Int. J. Robot. Res. 32(11), 1238–1274 (2013)
Polydoros, A., Nalpantidis, L.: Survey of model-based reinforcement learning: applications on robotics. J. Intell. Rob. Syst. 86(2), 153–173 (2017)
Vieira, A., Ribeiro, B.: Reinforcement Learning and Robotics, Introduction to Deep Learning Business Applications for Developers. Apress, pp. 137–168. A Press, Berkeley (2018)
Kormushev, P., Calinon, S., et al.: Reinforcement learning in robotics: applications and real-world challenges. Robotics 2(3), 122–148 (2013)
Wawrzynski, P.: Control policy with autocorrelated noise in reinforcement learning for robotics. Int. J. Mach. Learn. Comput. 5(2), 91 (2015)
Sutton, R.S., Barto, A.G.: Reinforcement learning: an introduction (2011)
Ravishankar, R., Vijayakumar, V.: Reinforcement learning algorithms: survey and classification. Indian J. Sci. Technol. 10(1), 1–8 (2017)
Koga, M.L., Silva, V.F., et al.: Speeding-up reinforcement learning through abstraction and transfer learning. In: Proc. of Int. Cof. on Autonomous Agents and Multi-agent Systems. International Foundation for Autonomous Agents and Multiagent Systems, pp. 119–126 (2013)
Azar, G., Munos, R., Ghavamzadeh, M., et al.: Speedy Q-learning: a computationally efficient reinforcement learning algorithm with a near optimal rate of convergence (2013)
Matignon, L., Laurent, G., et al.: Improving reinforcement learning speed for robot control. In: IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pp. 3172–3177 (2008)
Tokic, M.: Adaptive ε-greedy exploration in reinforcement learning based on value differences. In: Annual Conference on Artificial Intelligence, pp. 203–210 (2010)
Achbany, Y., Fouss, F., et al.: Tuning continual exploration in reinforcement learning: An optimality property of the Boltzmann strategy. Neurocomputing 71(13–15), 2507–2520 (2008)
Viet, H.H., Kyaw, P.H., et al.: Simulation-based evaluations of reinforcement learning algorithms for autonomous mobile robot path planning. In: Park, J., Arabnia, H., Chang, H.B., Shon, T. (eds.) IT Convergence and Services, pp. 467–476. Springer, Dordrecht (2011)
Wang, Z., Ren, J., et al.: A deep-learning based feature hybrid framework for spatiotemporal saliency detection inside videos. Neurocomputing 287, 68–83 (2018)
Ren, J., Jiang, J.: Hierarchical modeling and adaptive clustering for real-time summarization of rush videos. IEEE Trans. Multimed. 11(5), 906–917 (2009)
Han, J., Zhang, D., et al.: Object detection in optical remote sensing images based on weakly supervised learning and high-level feature learning. IEEE Trans. Geosci. Remote Sens. 53(6), 3325–3337 (2015)
Chen, J., Ren, J.: Modelling of content-aware indicators for effective determination of shot boundaries in compressed MPEG videos. Multimed. Tools Appl. 54(2), 219–239 (2011)
Ren, J., Vlachos, T.: Immersive and perceptual human–computer interaction using computer vision techniques. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 66–72. IEEE (2010)
Yan, Y., Ren, J., et al.: Cognitive fusion of thermal and visible imagery for effective detection and tracking of pedestrians in videos. Cogn. Comput. 61, 1–11 (2017)
Yan, Y., Ren, J., et al.: Unsupervised image saliency detection with Gestalt-laws guided optimization and visual attention based refinement. Pattern Recogn. 79, 65–78 (2018)
Acknowledgements
This research project was supported by two Shaanxi Province Founds (Program No. 2017ZDXM-GY-008 and 2016MSZD-G-8-1), and supported by two National Funds (Program No. 2017KF100037 and MJ-2015-D-66).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, L., Zhu, Y., Duan, J. (2018). An Improved Tentative Q Learning Algorithm for Robot Learning. In: Ren, J., et al. Advances in Brain Inspired Cognitive Systems. BICS 2018. Lecture Notes in Computer Science(), vol 10989. Springer, Cham. https://doi.org/10.1007/978-3-030-00563-4_84
Download citation
DOI: https://doi.org/10.1007/978-3-030-00563-4_84
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-00562-7
Online ISBN: 978-3-030-00563-4
eBook Packages: Computer ScienceComputer Science (R0)