Abstract
One of the most important features of “intelligent behaviour” is the ability to learn from experience. The introduction of Multiagent Systems brings new challenges to the research in Machine Learning. New difficulties, but also new advantages, appear when learning takes place in an environment in which agents can communicate and cooperate. The main question that drives this work is “How can agents benefit from communication with their peers during the learning process to improve their individual and global performances? ” We are particularly interested in environments where speed and band-width limitations do not allow highly structured communication, and where learning agents may use different algorithms. The concept of advice-exchange, which started out as mixture of reinforced and supervised learning procedures, is developing into a meta-learning architecture that allows learning agents to improve their learning skills by exchanging information with their peers. This paper reports the latest experiments and results in this subject.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Nunes, L., Oliveira, E.: On learning by exchanging advice. In: Proc. of the First Symposium on Adaptive Agents and Multi-Agent Systems (AAMAS/AISB 2002), pp. 29–40 (2002)
Dorigo, M., Colombetti, M.: The role of the trainer in reinforcement learning. In: Mahadevan, S. (ed.) Proc. of MLC-COLT 1994, pp. 37–45 (1994)
Clouse, J.A.: On integrating apprentice learning and reinforcement learning. PhD thesis, University of Massachusetts, Department of Computer Science (1997)
Nunes, L., Oliveira, E.: Advice-exchange in heterogeneous groups of learning agents. Technical Report 1 12/02, FEUP/LIACC (2002)
Nunes, L., Oliveira, E.: Advice exchange between evolutionary algorithms and reinforcement learning agents: Experimental results in the pursuit domain. In: Proc. of the Second Symposium on Adaptive Agents and Multi-Agent Systems, AAMAS/AISB 2003 (2003)
Nunes, L., Oliveira, E.: Advice exchange architecture. Technical Report 3 04/03, FEUP/LIACC (2003)
Rumelhart, D.E., Zipser, D.: Feature discovery by competitive learning. Cognitive Science 9 (1985)
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning internal representations by error propagation. Parallel Distributed Processing: Exploration in the Microstructure of Cognition 1, 318–362 (1986)
Watkins, C.J.C.H., Dayan, P.D.: Technical note: Q-learning. Machine Learning 8, 279–292 (1992)
Whitehead, S.D.: A complexity analisys of cooperative mechanisms in reinforcement learning. In: Proc. of the 9th National Conf. on AI (AAAI 1991), pp. 607–613 (1991)
Holland, J.H.: Adaptation in Natural and Artificial Systems. University of Michigan Press (1975)
Koza, J.R.: Genetic programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge (1992)
Glickman, M., Sycara, K.: Evolution of goal-directed behavior using limited information in a complex environment. In: Proc. of the Genetic and Evolutionary Computation Conference (GECCO 1999) (1999)
Benda, M., Jagannathan, V., Dodhiawalla, R.: On optimal cooperation of knowledge resources. Technical Report BCS G-2012-28, Boeing AI Center, Boeing Computer Services, Bellevue, WA (1985)
Tan, M.: Multi-agent reinforcement learning: Independent vs. cooperative agents. In: Proc. of the Tenth Int. Conf. on Machine Learning, pp. 330–337 (1993)
Haynes, T., Wainwright, R., Sen, S., Schoenfeld, D.: Strongly typed genetic programming in evolving cooperation strategies. In: Proc. of the Sixth Int. Conf. on Genetic Algorithms, pp. 271–278 (1995)
Sen, S., Sekaran, M., Hale, J.: Learning to coordinate without sharing information. In: Proc. of the National Conf. on AI, pp. 426–431 (1994)
Lin, L.J.: Self-improving reactive agents based on reinforcement learning, planning and teaching. Machine Learning 8, 293–321 (1992)
Littman, M.L.: Markov games as a framework for multi-agent reinforcement learning. In: Proc. of the Eleventh International Conference on Machine Learning, pp. 157–163 (1994)
Thrun, S., Mitchell, T.: Lifelong robot learning. Robotics and Autonomous Systems 15, 25–46 (1995)
Maclin, R., Shavlik, J.: Creating advicetaking reinforcement learners. Machine Learning 22, 251–281 (1996)
Matarić, M.J.: Using communication to reduce locality in distributed multi-agent learning. Computer Science Technical Report CS-96-190, Brandeis University (1996)
Claus, C., Boutilier, C.: The dynamics of reinforcement learning in cooperative multiagent systems. In: Proc. of the Fifteenth National Conference on Artificial Intelligence, Madison, WI, pp. 746–752 (1998)
Price, B., Boutilier, C.: Implicit imitation in multiagent reinforcement learning. In: Proc. of the Sixteenth Int. Conf. on Machine Learning, pp. 325–334 (1999)
Berenji, H.R., Vengerov, D.: Advantages of cooperation between reinforcement learning agents in difficult stochastic problems. In: Proc. of the Nineth IEEE International Conference on Fuzzy Systems (FUZZ-IEEE 2000) (2000)
Price, B., Boutilier, C.: Imitation and reinforcement learning in agents with heterogeneous actions. In: Proc. of the Seveteenth Int. Conf. on Machine Learning (ICML2000)(2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Nunes, L., Oliveira, E. (2003). Exchanging Advice and Learning to Trust. In: Klusch, M., Omicini, A., Ossowski, S., Laamanen, H. (eds) Cooperative Information Agents VII. CIA 2003. Lecture Notes in Computer Science(), vol 2782. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-45217-1_19
Download citation
DOI: https://doi.org/10.1007/978-3-540-45217-1_19
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40798-0
Online ISBN: 978-3-540-45217-1
eBook Packages: Springer Book Archive