Abstract
Most mobile robots are powered by batteries, which must be charged before their level become too low to continue providing services. This paper contributes a novel method based on Reinforcement Learning (RL) for the autonomous docking of mobile robots at their charging stations. Our proposal considers a RL network that is fed with images to visually sense the environment and with distance measurements to safely avoid obstacles, and produces motion commands to be executed by the robot. Additionally, since the autonomous docking is in essence a sparse reward task (the only state that returns a positive reward is when the robot docks at the charging station), we propose the usage of reward shaping to successfully learn to dock. For that we have designed extrinsic rewards that are built on the results of a Convolutional Neural Network in charge of detecting the pattern typically used to visually identify charging stations. The experiments carried out support our design decisions and validate the method implementation, reporting a \(\sim \)100% of success in the docking task with obstacle-free paths, and \(\sim \)93% when obstacles are considered, along with short execution times (10 s and 14 s on average, respectively).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Luperto, M., et al.: Towards long-term deployment of a mobile robot for at-home ambient assisted living of the elderly. In: 2019 European Conference on Mobile Robots (ECMR), pp. 1–6. IEEE (2019)
Tussyadiah, I.P., Park, S.: Consumer evaluation of hotel service robots. In: Stangl, B., Pesonen, J. (eds.) Information and Communication Technologies in Tourism 2018, pp. 308–320. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-72923-7_24
Ruiz-Sarmiento, J.R., Galindo, C., González-Jiménez, J.: Robot@ home, a robotic dataset for semantic mapping of home environments. Int. J. Robot. Res. 36(2), 131–141 (2017)
González-Jiménez, J., Galindo, C., Ruiz-Sarmiento, J.R.: Technical improvements of the Gira telepresence robot based on users’ evaluation. In: 2012 IEEE RO-MAN, pp. 827–832 (2012)
Wang, Y., et al.: Autonomous target docking of nonholonomic mobile robots using relative pose measurements. IEEE Trans. Industr. Electron. 68, 1 (2020)
Burgue no Romero, A.M., et al.: A collection of Jupyter notebooks covering the fundamentals of computer vision. In: ICERI2020 Proceedings. Online Conference, pp. 5495–5505 (2020)
Yahya, M.F., Arshad, M.R.: Detection of markers using deep learning for docking of autonomous underwater vehicle. In: 2017 IEEE I2CACIS, pp. 179–184 (2017)
Kriegler, A., Wöber, W.: Vision-based docking of a mobile robot. Tech. rep. EasyChair (2021)
Ebert, F., et al.: Visual foresight: model-based deep reinforcement learning for vision-based robotic control. arXiv: 1812.00568 [cs.RO] (2018)
Tai, L., Paolo, G., Liu, M.: Virtual-to-real deep reinforcement learning: continuous control of mobile robots for mapless navigation. In: 2017 IEEE/RSJ IROS, pp. 31–36 (2017)
H3 Dynamics homepage. https://www.h3dynamics.com/. Accessed 20 Apr 2021
Juliani, A., et al.: Unity: a general platform for intelligent agents. arXiv:1809.02627 (2018)
He, K., Girshick, R., Dollar, P.: Rethinking ImageNet pre-training. In: Proceedings of the IEEE/CVF ICCV, October 2019
Gonzalez-Jimenez, J., Galindo, C., Gutierrez-Castaneda, C.: Evaluation of a telepresence robot for the elderly: a Spanish experience. In: Ferrández Vicente, J.M., Álvarez Sánchez, J.R., de la Paz López, F., Toledo Moreo, F.J. (eds.) IWINAC 2013. LNCS, vol. 7930, pp. 141–150. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-38637-4_15
Torabi, F., Warnell, G., Stone, P.: Behavioral cloning from observation (2018). arXiv: 1805.01954 [cs.AI]
Schulman, J., et al.: Proximal policy optimization algorithms. CoRR arXiv:1707.06347 (2017)
Haarnoja, T., et al.: Soft actor-critic algorithms and applications (2019). arXiv: 1812.05905 [cs.LG]
Hare, J.: Dealing with sparse rewards in reinforcement learning (2019). arXiv:1910.09281 [cs.LG]
Vecerik, M., et al.: Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards. Presented at the arXiv:1707.08817 (2018) [cs.AI]
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)
Acknowledgements
This work was supported by the research projects WISER (DPI2017-84827-R) and ARPEGGIO (PID2020-117057).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Burgueño-Romero, A.M., Ruiz-Sarmiento, J.R., Gonzalez-Jimenez, J. (2021). Autonomous Docking of Mobile Robots by Reinforcement Learning Tackling the Sparse Reward Problem. In: Rojas, I., Joya, G., Català, A. (eds) Advances in Computational Intelligence. IWANN 2021. Lecture Notes in Computer Science(), vol 12862. Springer, Cham. https://doi.org/10.1007/978-3-030-85099-9_32
Download citation
DOI: https://doi.org/10.1007/978-3-030-85099-9_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-85098-2
Online ISBN: 978-3-030-85099-9
eBook Packages: Computer ScienceComputer Science (R0)