Abstract
Reinforcement Learning is a commonly used technique in robotics, however, traditional algorithms are unable to handle large amounts of data coming from the robot’s sensors, require long training times, are unable to re-use learned policies on similar domains, and use discrete actions. This work introduces TS-RRLCA, a two stage method to tackle these problems. In the first stage, low-level data coming from the robot’s sensors is transformed into a more natural, relational representation based on rooms, walls, corners, doors and obstacles, significantly reducing the state space. We also use Behavioural Cloning, i.e., traces provided by the user to learn, in few iterations, a relational policy that can be re-used in different environments. In the second stage, we use Locally Weighted Regression to transform the initial policy into a continuous actions policy. We tested our approach with a real service robot on different environments for different navigation and following tasks. Results show how the policies can be used on different domains and perform smoother, faster and shorter paths than the original policies.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Aznar, F., Pujol, F.A., Pujol, M., Rizo, R.: Using gaussian processes in bayesian robot programming. In: Distributed Computing, Artificial Intelligence, Bioinformatics, Soft Computing, and Ambient Assisted Living, pp. 547–553 (2009)
Bratko, I., Urbancic, T., Sammut, C.: Behavioural cloning of control skill. Machine Learning and Data Mining, 335–351 (1998)
Cocora, A., Kersting, K., Plagemanny, C., Burgardy, W., De Raedt, L.: Learning relational navigation policies. Journal of Intelligent and Robotics System, 2792–2797 (October 2006)
Conn, K., Peters, R.A.: Reinforcement learning with a supervisor for a mobile robot in a real-world environment. In: Proc. of the IEEE CIRA (2007)
Gaskett, C., Wettergreen, D., Zelinsky, A.: Q-learning in continuous state and action spaces. In: Foo, N.Y. (ed.) AI 1999. LNCS, vol. 1747, pp. 417–428. Springer, Heidelberg (1999)
Hernández, S.F., Morales, E.F.: Global localization of mobile robots for indoor environments using natural landmarks. In: Proc. of the IEEE 2006 ICRAM, September 2006, pp. 29–30 (2006)
Romero, L., Morales, E.F., Sucar, L.E.: An exploration and navigation approach for indoor mobile robots considering sensor’s perceptual limitations. In: Proc. of the IEEE ICRA, pp. 3092–3097 (2001)
Torrey, L., Shavlik, J., Walker, T., Maclin, R.: Relational macros for transfer in reinforcement learning. In: ILP, pp. 254–268 (2008)
Vaughan, R., Gerkey, B., Howard, A.: On device abstractions for portable, reusable robot code. In: Proc. of the 2003 IEEE/RSJ IROS, pp. 11–15 (2003)
Vega, J.H.: Mobile robot localization in topological maps using visual information. Masther’s thesis (to be publised, 2009)
Wang, Y., Huber, M., Papudesi, V.N., Cook, D.J.: User-guided reinforcement learning of robot assistive tasks for an intelligent environment. In: Proc. of the The 2003 IEEE/RSJ IROS, October 2003, pp. 27–31 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zaragoza, J.H., Morales, E.F. (2009). A Two-Stage Relational Reinforcement Learning with Continuous Actions for Real Service Robots. In: Aguirre, A.H., Borja, R.M., Garciá, C.A.R. (eds) MICAI 2009: Advances in Artificial Intelligence. MICAI 2009. Lecture Notes in Computer Science(), vol 5845. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-05258-3_30
Download citation
DOI: https://doi.org/10.1007/978-3-642-05258-3_30
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-05257-6
Online ISBN: 978-3-642-05258-3
eBook Packages: Computer ScienceComputer Science (R0)