Abstract
We outline a new model of multi-agent coalition formation which focuses on how collaborative agents can improve their coalition formation skills over time, learning from their prior interactions. The proposed research direction builds on our prior work on distributed coalition formation in collaborative multi-agent systems (MAS), centered at partitioning the underlying network of agents into non-overlapping cliques. At the core of that prior research is the MCDCF algorithm which provides a semantically simple, fully decentralized, local and (for sufficiently sparse networks) scalable mechanism for multi-agent coalition formation [1–3]. Our goal is to extend the MCDCF-based coalition formation along several new dimensions. First, we want to consider candidate coalitions that (i) no longer have to be cliques but can be more general types of (connected) subgraphs, and (ii) that also satisfy additional, more complex “compatibility” properties stemming from individual agents’ capabilities and preferences. Second, we begin exploration of semantically more rich and versatile ways of capturing this inter-agent compatibility than what’s found in the existing literature. In particular, we propose applying graph pattern techniques to capture a variety of qualitative “compatibility relationships” among agents. Next, we revisit approaches to and benefits of reinforcement learning (RL) in the context of autonomous agents repeatedly engaging in coalition formation. Last but not least, we discuss benefits of each agent maintaining other agents’ reputations that quantify those agents’ coalition formation effectiveness in the past. With these extensions, we argue that the resulting modeling framework adequately captures core aspects of a much richer class of multi-agent coalition formation scenarios, as well as, more broadly, of a variety of distributed consensus reaching problems in collaborative MAS.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Tosic, P., Agha, G.: Maximal clique based distributed group formation for autonomous agent coalitions. In: Proceedings of Coalitions and Teams Workshop (W10), pp. 1–8. Within the 3rd International Joint Conference on Agents and Multi Agent Systems (AAMAS 2004), New York City, New York, USA (2004)
Tošić, P.T., Agha, G.: Maximal clique based distributed coalition formation for task allocation in large-scale multi-agent systems. In: Ishida, T., Gasser, L., Nakashima, H. (eds.) MMAS 2005. LNCS (LNAI), vol. 3446, pp. 104–120. Springer, Heidelberg (2005)
Tosic, P.: Distributed coalition formation for collaborative large-scale multi-agent systems. MS thesis, University of Illinois at Urbana-Champaign, Urbana, Illinois, USA (2006)
Tosic, P., et al.: Modeling a system of UAVs on a mission, invited session on agent-based computing. In: Proceedings of the Seventh World Multiconference on Systemics, Cybernetics, and Informatics (SCI 2003), pp. 508–514 (2003)
Shehory, O., Kraus, S.: Task allocation via coalition formation among autonomous agents. In: Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI 1995), Montréal, Canada, pp. 655–661 (1995)
Shehory, O., Sycara, K., Jha, S.: Intelligent agents IV multi-agent coordination through coalition formation. In: Singh, M.P., Rao, A., Wooldridge, M.J. (eds.) ATAL 1997. LNCS (LNAI), vol. 1365, pp. 143–154. Springer, Heidelberg (1997)
Tosic, P., Ginne, N.K.: Distributed coalition formation for collaborative multi-agent systems: a performance case study on random graphs. In: Proceedings of the 9th European Workshop on Multi-agent Systems (EUMAS 2011), Maastricht, The Netherlands, December 2011
Tosic, P., Ginne, N.K.: Challenges in distributed coalition formation among collaborative multi-agent systems: an experimental case study on small-world networks. In: Proceedings of the International Conference on Artificial Intelligence (ICAI 2011), Las Vegas, Nevada, USA (2011)
Sandholm, T., et al.: Coalition structure generation with worst case guarantees. AI J. 111(1–2), 209–238 (1999)
Cerquides, J., et al.: A tutorial on optimization for multi-agent systems. Comput. J. 57 (6), 799–824 (2014). British Computer Society, UK
Tosic, P., Vilalta, R.: A unified framework for reinforcement learning, co-learning and meta-learning how to coordinate in collaborative multi-agent systems. In: Proceedings of the International Conference on Computational Science ICCS-2010 (Track on Cognitive Agents: Theory and Practice), Amsterdam, The Netherlands, June 2010. In: Procedia Comput. Sci. 1 (1), 2217–2226 (2010)
Fan, W., et al.: Association rules with graph patterns. In: Proceedings of the International Conference on Very Large Data Bases (VLDB 2015), pp. 1502–1513. Kohala Coast, Hawai’i, USA (2015)
Soh, L.K., Li, X.: An integrated multilevel learning approach to multiagent coalition formation. In: Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI 2003), pp. 619–624. Acapulco, Mexico (2003)
Tošić, P.T., Ordonez, C.: Distributed protocols for multi-agent coalition formation: a negotiation perspective. In: Huang, R., Ghorbani, A.A., Pasi, G., Yamaguchi, T., Yen, N.Y., Jin, B. (eds.) AMT 2012. LNCS, vol. 7669, pp. 93–102. Springer, Heidelberg (2012)
Tosic, P., Wu, Y.: Towards networks of search engines and other digital experts: a distributed intelligence approach. In: Proceedings of the 8th International Conference on u- and e-Service, Science and Technology (UNESST 2015), pp. 35–38. IEEE Computer Society (2015)
Tran, T.T.: Reputation-oriented reinforcement learning strategies for economically-motivated agents in electronic market environments. Ph.D. thesis, University of Waterloo, Waterloo, Ontario, Canada (2004)
Tosic, P., Vilalta, R.: Learning and meta-learning for coordination of autonomous unmanned vehicles - a preliminary analysis. In: Proceedings of the European Conference on Artificial Intelligence (ECAI-2010), pp. 163–168. Lisbon, Portugal, August 2010
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Tošić, P.T. (2016). Graph Patterns, Reinforcement Learning and Models of Reputation for Improving Coalition Formation in Collaborative Multi-agent Systems. In: Rovatsos, M., Vouros, G., Julian, V. (eds) Multi-Agent Systems and Agreement Technologies. EUMAS AT 2015 2015. Lecture Notes in Computer Science(), vol 9571. Springer, Cham. https://doi.org/10.1007/978-3-319-33509-4_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-33509-4_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-33508-7
Online ISBN: 978-3-319-33509-4
eBook Packages: Computer ScienceComputer Science (R0)