Abstract
This research aims at studying the effects of exchanging information during the learning process in Multiagent Systems. The concept of advice-exchange, introduced in previous contributions, consists in enabling an agent to request extra feedback, in the form of episodic advice, from other agents that are solving similar problems. The work that was previously focused on the exchange of information between agents that were solving detached problems is now concerned with groups of learning-agents that share the same environment. This change added new difficulties to the task. The experiments reported below were conducted to detect the causes and correct the shortcomings that emerged when moving from environments where agents worked in detached problems to those where agents are interacting in the same environment. New concepts, such as self confidence, trust and advisor preference are introduced in this text.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Clouse, J.A.: On integrating apprentice learning and reinforcement learning. PhD thesis, University of Massachusetts, Department of Computer Science (1997)
Nunes, L., Oliveira, E.: Advice-exchange in heterogeneous groups of learning agents. Technical Report 1 12/02, FEUP/LIACC (2002)
Whitehead, S.D.: A complexity analysis of cooperative mechanisms in reinforcement learning. In: Proc. of the 9th National Conf. on AI (AAAI 1991), pp. 607–613 (1991)
Clouse, J.A., Utgoff, P.E.: Two kinds of training information for evaluation function learning. In: Proc. of AAAI 1991 (1991)
Clouse, J., Utgoff, P.: A teaching method for reinforcement learning. In: Proc. of the 9th Int. Conf. on Machine Learning, pp. 92–101 (1992)
Lin, L.J.: Self-improving reactive agents based on reinforcement learning, planning and teaching. Machine Learning 8, 293–321 (1992)
Tan, M.: Multi-agent reinforcement learning: Independent vs. cooperative agents. Proc. of the Tenth Int. Conf. on Machine Learning, 330–337 (1993)
Price, B., Boutilier, C.: Implicit imitation in multiagent reinforcement learning. Proc. of the Sixteenth Int. Conf. on Machine Learning, 325–334 (1999)
Sen, S., Kar, P.P.: Sharing a concept. In: Working Notes of the AAAI 2002 Spring Symposium on Collaborative Learning Agents (2002)
Watkins, C.J.C.H., Dayan, P.D.: Technical note: Q-learning. Machine Learning 8, 279–292 (1992)
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning internal representations by error propagation. Parallel Distributed Processing: Exploration in the Microstructure of Cognition 1, 318–362 (1986)
Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: A survey. Journal of Artificial Intelligence Research 4 (1996)
Nunes, L., Oliveira, E.: Advice exchange between evolutionary algorithms and reinforcement learning agents: Experimental results in the pursuit domain. Technical Report 2 03/03, FEUP/LIACC (2003)
Holland, J.H.: Adaptation in Natural and Artificial Systems. University of Michigan Press (1975)
Koza, J.R.: Genetic programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge (1992)
Salustowicz, R.: A Genetic Algorithm for the Topological Optimization of Neural Networks. PhD thesis, Tech. Univ. Berlin (1995)
Yao, X.: Evolving artificial neural networks. Proc. of IEEE. 87, 1423–1447 (1999)
Glickman, M., Sycara, K.: Evolution of goal-directed behavior using limited information in a complex environment. In: Proc. of the Genetic and Evolutionary Computation Conference, GECCO 1999 (1999)
Nunes, L., Oliveira, E.: On learning by exchanging advice. In: Proc. of the First Symposium on Adaptive Agents and Multi-Agent Systems, AISB 2002 (2002)
Benda, M., Jagannathan, V., Dodhiawalla, R.: On optimal cooperation of knowledge resources. Technical Report BCS G-2012-28, Boeing AI Center, Boeing Computer Services, Bellevue, WA (1985)
Haynes, T., Wainwright, R., Sen, S., Schoenfeld, D.: Strongly typed genetic programming in evolving cooperation strategies. In: Proc. of the Sixth Int. Conf. on Genetic Algorithms, pp. 271–278 (1995)
Sen, S., Sekaran, M., Hale, J.: Lerning to coordinate without sharing information. In: Proc. of the National Conf. on AI, pp. 426–431 (1994)
Sen, S., Sekaran, M.: Individual learning of coordination knowledge. Journal of Experimental, Theoretical Artificial Intelligence 10, 333–356 (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Nunes, L., Oliveira, E. (2005). Advice-Exchange Between Evolutionary Algorithms and Reinforcement Learning Agents: Experiments in the Pursuit Domain. In: Kudenko, D., Kazakov, D., Alonso, E. (eds) Adaptive Agents and Multi-Agent Systems II. AAMAS AAMAS 2004 2003. Lecture Notes in Computer Science(), vol 3394. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-32274-0_12
Download citation
DOI: https://doi.org/10.1007/978-3-540-32274-0_12
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-25260-3
Online ISBN: 978-3-540-32274-0
eBook Packages: Computer ScienceComputer Science (R0)