Abstract
Applying conventional Q-Learning to Multi-Component Robotic Systems (MCRS) increasing the number of components produces an exponential growth of state storage requirements. Modular approaches limit the state size growth to be polynomial on the number of components, allowing more manageable state representation and manipulation. In this article, we advance on previous works on a modular Q-learning approach to learn the distributed control of a Linked MCRS. We have chosen a paradigmatic application of this kind of systems using only local rewards: a set of robots carrying a hose from some initial configuration to a desired goal. The hose dynamics are simplified to be a distance constraint on the robots positions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Busoniu, L., Babuska, R., Schutter, B.D.: A comprehensive survey of multiagent reinforcement learning. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 38(2), 156–172 (2008)
Duro, R.J., Graña, M., de Lope, J.: On the potential contributions of hybrid intelligent approaches to multicomponen robotic system development. Information Sciences 180(14), 2635–2648 (2010)
Echegoyen, Z., Villaverde, I., Moreno, R., Graña, M., d’Anjou, A.: Linked multi-component mobile robots: modeling, simulation and control. Robotics and Autonomous Systems 58(12), 1292–1305 (2010)
Fernandez-Gauna, B., Graña, M., Lopez-Guede, J.M.: Towards concurrent q-learning on linked multi-component robotic systems. In: HAIS 2011. LNCS (LNAI), Springer, Heidelberg (2011) (in press)
Fernandez-Gauna, B., Lopez-Guede, J.M., Zulueta, E., Graña, M.: Learning hose transport control with Q-learning. Neural Network World 20(7), 913–923 (2010)
Fernandez-Gauna, B., Lopez-Guede, J.M., Zulueta, E.: Linked multicomponent robotic systems: Basic assessment of linking element dynamical effect. In: Corchado, E., Graña, M., Savio, A. (eds.) Hybrid Artificial Intelligence Systems, Part I, vol. 6076, pp. 73–79. Springer, Heidelberg (2010)
Fernandez-Gauna, B., Lopez-Guede, J.M., Zulueta, E., Echegoyen, Z., Graña, M.: Basic results and experiments on robotic multi-agent system for hose deployment and transportation. International Journal of Artificial Intelligence 6(S11), 183–202 (2011)
Graña, M., Torrealdea, F.J.: Hierarchically structured systems. European Journal of Operational Research 25, 20–26 (1986)
Maravall, D., de Lope, J., Martin, J.A.: Hybridizing evolutionary computation and reinforcement learning for the design of almost universal controllers for autonomous robots. Neurocomputing 72(4-6), 887–894 (2009)
Panait, L., Luke, S.: Cooperative multi-agent learning: The state of the art. Autonomous Agents and Multi-Agent Systems 11(3), 387–434 (2005)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Whitehead, S., Karlsson, J., Tenenberg, J.: Learning multiple goal behavior via task decomposition and dynamic policy merging. In: Robot Learning, pp. 45–78. Kluwer Academic Publisher, Dordrecht (1993)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Fernandez-Gauna, B., Lopez-Guede, J.M., Graña, M. (2011). Concurrent Modular Q-Learning with Local Rewards on Linked Multi-Component Robotic Systems. In: Ferrández, J.M., Álvarez Sánchez, J.R., de la Paz, F., Toledo, F.J. (eds) Foundations on Natural and Artificial Computation. IWINAC 2011. Lecture Notes in Computer Science, vol 6686. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-21344-1_16
Download citation
DOI: https://doi.org/10.1007/978-3-642-21344-1_16
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-21343-4
Online ISBN: 978-3-642-21344-1
eBook Packages: Computer ScienceComputer Science (R0)