Abstract
As a method of multi-agent system cooperation, multi-agent communication can help agents negotiate and adjust behavior decisions by exchanging information such as observation, intention, or experience during operation, improve the overall learning performance, and achieve their learning objectives. However, there are still some challenging problems in multi-agent communication. With the expansion of the multi-agent system scale, the global complete massive information will bring great resource overhead, and the introduction of redundant communication will lead to the difficulty of agent policy convergence, and affect the joint action and target completion. In addition, predefined communication structures have potential cooperation limitations in dynamic environments. In this paper, we introduce a dynamic communication model based on the graph convolution neural network called DCGN. Empirically, we show that DCGN can better cope with the dynamic update of tasks in the process of helping agents complete task information interaction, and can formulate more coordinated strategies than the existing methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press (2018)
Rizk, Y., Awad, M., Tunstel, E.W.: Cooperative heterogeneous multi-robot systems: a survey. ACM Comput. Surv. (CSUR) 52(2), 1–31 (2019)
Foerster, J., Assael, I.A., De Freitas, N., Whiteson, S.: Learning to communicate with deep multi-agent reinforcement learning. In: Advances in Neural Information Processing Systems, vol. 29 (2016)
Jiang, J., Lu, Z.: Learning attentional communication for multi-agent cooperation. In: Advances in Neural Information Processing Systems, vol. 31 (2018)
Du, Y., et al.: Learning correlated communication topology in multi-agent reinforcement learning. In: Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems, pp. 456–464 (2021)
Sukhbaatar, S., Fergus, R., et al.: Learning multiagent communication with backpropagation. In: Advances in Neural Information Processing Systems, vol. 29 (2016)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Singh, A., Jain, T., Sukhbaatar, S.: Learning when to communicate at scale in multiagent cooperative and competitive tasks. arXiv preprint arXiv:1812.09755 (2018)
Hoshen, Y.: Vain: attentional multi-agent predictive modeling. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Das, A., et al.: TarMAC: targeted multi-agent communication. In: International Conference on Machine Learning, PMLR 2019, pp. 1538–1546 (2019)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)
Jiang, J., Dun, C., Huang, T., Lu, Z.: Graph convolutional reinforcement learning. arXiv preprint arXiv:1810.09202 (2018)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Yuan, Q., Fu, X., Li, Z., Luo, G., Li, J., Yang, F.: GraphComm: efficient graph convolutional communication for multiagent cooperation. IEEE IoT J. 8(22), 16 359–16 369 (2021)
Hu, G., Zhu, Y., Zhao, D., Zhao, M., Hao, J.: Event-triggered multi-agent reinforcement learning with communication under limited-bandwidth constraint. arXiv preprint arXiv:2010.04978 (2020)
Ding, Z., Huang, T., Lu, Z.: Learning individually inferred communication for multi-agent cooperation. In: Advances in Neural Information Processing Systems, vol. 33, pp. 22 069–22 079 (2020)
Liu, Y., Wang, W., Hu, Y., Hao, J., Chen, X., Gao, Y.: Multi-agent game abstraction via graph attention neural network. Proc. AAAI Conf. Artif. Intell. 34(05), 7211–7218 (2020)
Malysheva, A., Sung, T.T., Sohn, C.-B., Kudenko, D., Shpilman, A.: Deep multi-agent reinforcement learning with relevance graphs. arXiv preprint arXiv:1811.12557 (2018)
Kim, D., et al.: Learning to schedule communication in multi-agent reinforcement learning. arXiv preprint arXiv:1902.01554 (2019)
Niu, Y., Paleja, R.R., Gombolay, M.C.: Multi-agent graph-attention communication and teaming. In: AAMAS 2021, pp. 964–973 (2021)
Acknowledgment
This work was partially supported by the National Science Fund for Distinguished Young Scholars (62025205), and the National Natural Science Foundation of China (No. 62002292, 62032020, 61960206008, 62102322).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Zhang, Y., Liu, J., Ren, H., Guo, B., Yu, Z. (2024). Autonomous Communication Decision Making Based on Graph Convolution Neural Network. In: Jin, H., Yu, Z., Yu, C., Zhou, X., Lu, Z., Song, X. (eds) Green, Pervasive, and Cloud Computing. GPC 2023. Lecture Notes in Computer Science, vol 14504. Springer, Singapore. https://doi.org/10.1007/978-981-99-9896-8_20
Download citation
DOI: https://doi.org/10.1007/978-981-99-9896-8_20
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-9895-1
Online ISBN: 978-981-99-9896-8
eBook Packages: Computer ScienceComputer Science (R0)