Zhaoqing Peng
ABSTRACT
Many tasks in AI require the collaboration of multiple agents. Generally, these agents cooperate with each other by message-passing communication. However, agents may suffer from being overwhelmed by massive received messages and have difficulties in obtaining useful information. To this end, we use an attention-based message processing (AMP) method to model agents' interactions by considering the relevance of each received message. To improve the efficiency of learning correct interactions, a supervised variant SAMP is then proposed to directly optimize the attentional weights in AMP with a target auxiliary interaction matrix from the environment. The empirical results demonstrate our proposal outperforms other competing multi-agent methods in "predator-prey-toxin" domain, and prove the superiority of SAMP in correctly guiding the optimization of attentional weights in AMP.
- Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014).Google Scholar
- Jakob Foerster, Ioannis Alexandros Assael, Nando de Freitas, and Shimon Whiteson. 2016. Learning to communicate with deep multi-agent reinforcement learning. In Advances in Neural Information Processing Systems. 2137--2145. Google ScholarDigital Library
- Jakob Foerster, Gregory Farquhar, Triantafyllos Afouras, Nantas Nardelli, and Shimon Whiteson. 2017. Counterfactual multi-agent policy gradients. arXiv preprint arXiv:1705.08926 (2017).Google Scholar
- Jakob Foerster, Nantas Nardelli, Gregory Farquhar, Philip Torr, Pushmeet Kohli, Shimon Whiteson, et al. 2017. Stabilising experience replay for deep multi-agent reinforcement learning. arXiv preprint arXiv:1702.08887 (2017).Google Scholar
- Jayesh K Gupta, Maxim Egorov, and Mykel Kochenderfer. 2017. Cooperative multi-agent control using deep reinforcement learning. In International Conference on Autonomous Agents and Multiagent Systems. Springer, 66--83.Google ScholarCross Ref
- Matthew Hausknecht and Peter Stone. 2016. Grounded Semantic Networks for Learning Shared Communication Protocols. In International Conference on Machine Learning (Workshop).Google Scholar
- Yedid Hoshen. 2017. Vain: Attentional multi-agent predictive modeling. In Advances in Neural Information Processing Systems. 2698--2708.Google Scholar
- Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016).Google Scholar
- Tejas D Kulkarni, Karthik Narasimhan, Ardavan Saeedi, and Josh Tenenbaum. 2016. Hierarchical deep reinforcement learning: Integrating temporal abstraction and intrinsic motivation. In Advances in neural information processing systems. 3675--3683. Google ScholarDigital Library
- Joel Z Leibo, Vinicius Zambaldi, Marc Lanctot, Janusz Marecki, and Thore Graepel. 2017. Multi-agent reinforcement learning in sequential social dilemmas. In Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems. International Foundation for Autonomous Agents and Multiagent Systems, 464--473. Google ScholarDigital Library
- Hangyu Mao, Yan Ni, Zhibo Gong, Weichen Ke, Chao Ma, Yang Xiao, Yuan Wang, Jiakang Wang, Quanbin Wang, Xiangyu Liu, et al. 2017. ACCNet: Actor-Coordinator-Critic Net for" Learning-to-Communicate" with Deep Multi-agent Reinforcement Learning. arXiv preprint arXiv:1706.03235 (2017).Google Scholar
- Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. 2016. Asynchronous methods for deep reinforcement learning. In International Conference on Machine Learning. 1928--1937. Google ScholarDigital Library
- Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. 2015. Human-level control through deep reinforcement learning. Nature 518, 7540 (2015), 529.Google Scholar
- Igor Mordatch and Pieter Abbeel. 2017. Emergence of grounded compositional language in multi-agent populations. arXiv preprint arXiv:1703.04908 (2017).Google Scholar
- Kozue Noro, Hiroshi Tenmoto, and Akimoto Kamiya. 2014. Signal learning with messages by reinforcement learning in multi-agent pursuit problem. Procedia Computer Science 35 (2014), 233--240.Google ScholarCross Ref
- Shayegan Omidshafiei, Jason Pazis, Christopher Amato, Jonathan P How, and John Vian. 2017. Deep Decentralized Multi-task Multi-Agent RL under Partial Observability, In International Conference on Machine Learning. arXiv preprint arXiv:1703.06182, 2681--2690.Google Scholar
- Liviu Panait and Sean Luke. 2005. Cooperative multi-agent learning: The state of the art. Autonomous agents and multi-agent systems 11, 3 (2005), 387--434. Google ScholarDigital Library
- Peng Peng, Quan Yuan, Ying Wen, Yaodong Yang, Zhenkun Tang, Haitao Long, and Jun Wang. 2017. Multiagent Bidirectionally-Coordinated nets for learning to play StarCraft combat games. arXiv preprint arXiv:1703.10069 (2017).Google Scholar
- Zhaoqing Peng, Takumi Kato, Hideyuki Takahashi, and Tetsuo Kinoshita. 2015. Intelligent home security system using agent-based IoT Devices. In Consumer Electronics (GCCE), 2015 IEEE 4th Global Conference on. IEEE, 313--314.Google ScholarCross Ref
- Sainbayar Sukhbaatar, Rob Fergus, et al. 2016. Learning multiagent communication with backpropagation. In Advances in Neural Information Processing Systems. 2244--2252. Google ScholarDigital Library
- Richard S Sutton, David A McAllester, Satinder P Singh, and Yishay Mansour. 2000. Policy gradient methods for reinforcement learning with function approximation. In Advances in neural information processing systems. 1057--1063. Google ScholarDigital Library
Index Terms
- Learning to Communicate via Supervised Attentional Message Processing
Recommendations
Learning Summarised Messaging Through Mediated Differentiable Inter-Agent Learning
Multi-Agent Systems and Agreement TechnologiesAbstractIn recent years, notable research has been done in the area of communication in multi-agent systems. When agents have a partial view of the environment, communication becomes essential for collaboration. We propose a Deep Q-Learning based multi-...
Learning Group-Level Information Integration in Multi-Agent Communication
AAMAS '23: Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent SystemsIn multi-agent systems, it's hard to make proper decisions for agents due to the partial observability of the environment. Among categories of multi-agent reinforcement learning (MARL) algorithms, communication learning is a common approach to solving ...
Pheromone-inspired Communication Framework for Large-scale Multi-agent Reinforcement Learning
Artificial Neural Networks and Machine Learning – ICANN 2022AbstractBeing difficult to scale poses great problems in multi-agent coordination. Multi-agent Reinforcement Learning (MARL) algorithms applied in small-scale multi-agent systems are hard to extend to large-scale ones because the latter is far more ...
Comments