Abstract
Cooperative multi-agent reinforcement learning (CMARL) has shown promise in solving real-world scenarios. The interaction information between agents contains rich global information, which is easily neglected after perceiving other agents’ behavior. To tackle this problem, we propose Collaboration Interaction Information Modelling via Hypergraph (CIIMH), which first perceives the behavior of other agents by mutual information optimization and constructs the dynamic interaction information via hypergraph. Perceived behavioral features of other agents are further aggregated in the hypergraph convolutional network to obtain interaction information. We compare our method with three existing baselines on StarCraft II micromanagement tasks (SMAC), Level-based Foraging (LBF), and Hallway. Empirical results show that our method outperforms baseline methods on all maps.
Supported by National Natural Science Foundation of China (61772355, 61702055, 61876217, 62176175). Project Funded by the Priority Academic Program Development of Jiangsu Higher Education Institutions (PAPD).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alemi, A.A., Fischer, I., Dillon, J.V., Murphy, K.: Deep variational information bottleneck. arXiv preprint arXiv:1612.00410 (2016)
Bai, Y., Gong, C., Zhang, B., Fan, G., Hou, X., Lu, Y.: Cooperative multi-agent reinforcement learning with hypergraph convolution. In: 2022 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2022)
Bhalla, S., Ganapathi Subramanian, S., Crowley, M.: Deep multi agent reinforcement learning for autonomous driving. In: Goutte, C., Zhu, X. (eds.) Canadian AI 2020. LNCS (LNAI), vol. 12109, pp. 67–78. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-47358-7_7
Bretto, A.: Hypergraph Theory. An Introduction. Mathematical Engineering, Springer, Cham (2013). https://doi.org/10.1007/978-3-319-00080-0
Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014)
Feng, Y., You, H., Zhang, Z., Ji, R., Gao, Y.: Hypergraph neural networks. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 3558–3565 (2019)
Kiran, B.R., et al.: Deep reinforcement learning for autonomous driving: a survey. IEEE Trans. Intell. Transp. Syst. (2021)
Li, J., Yu, T.: Large-scale multi-agent deep reinforcement learning-based coordination strategy for energy optimization and control of proton exchange membrane fuel cell. Sustain. Energy Technol. Assess. 48, 101568 (2021)
Luo, Y.C., Tsai, C.W.: Multi-agent reinforcement learning based on two-step neighborhood experience for traffic light control. In: Proceedings of the 2021 ACM International Conference on Intelligent Computing and its Emerging Applications, pp. 28–33 (2021)
Van der Maaten, L., Hinton, G.: Visualizing data using T-SNE. J. Mach. Learn. Res. 9(11), 2579–2605 (2008)
Mahajan, A., Rashid, T., Samvelyan, M., Whiteson, S.: Maven: multi-agent variational exploration. In: Advances in Neural Information Processing Systems, vol. 32 (2019)
Oliehoek, F.A., Amato, C.: A concise introduction to decentralized pomdps (2015)
Papoudakis, G., Christianos, F., Schäfer, L., Albrecht, S.V.: Benchmarking multi-agent deep reinforcement learning algorithms in cooperative tasks. arXiv preprint arXiv:2006.07869 (2020)
Rashid, T., Samvelyan, M., Witt, C.S., Farquhar, G., Foerster, J., Whiteson, S.: Qmix: monotonic value function factorisation for deep multi-agent reinforcement learning. In: International Conference on Machine Learning, pp. 4292–4301 (2018)
Samvelyan, M., et al.: The starcraft multi-agent challenge. arXiv preprint arXiv:1902.04043 (2019)
Sunehag, P., et al.: Value-decomposition networks for cooperative multi-agent learning. arXiv preprint arXiv:1706.05296 (2017)
Tufano, R., Scalabrino, S., Pascarella, L., Aghajani, E., Oliveto, R., Bavota, G.: Using reinforcement learning for load testing of video games. In: Proceedings of the 44th International Conference on Software Engineering, pp. 2303–2314 (2022)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Wang, J., Ren, Z., Liu, T., Yu, Y., Zhang, C.: Qplex: duplex dueling multi-agent q-learning. In: International Conference on Learning Representations (ICLR) (2021)
Wang, T., Wang, J., Zheng, C., Zhang, C.: Learning nearly decomposable value functions via communication minimization. arXiv preprint arXiv:1910.05366 (2019)
Wen, G., Fu, J., Dai, P., Zhou, J.: DTDE: a new cooperative multi-agent reinforcement learning framework. Innovation 2(4) (2021)
Wu, T., et al.: Multi-agent deep reinforcement learning for urban traffic light control in vehicular networks. IEEE Trans. Veh. Technol. 69(8), 8243–8256 (2020)
Zhang, B., Bai, Y., Xu, Z., Li, D., Fan, G.: Efficient policy generation in multi-agent systems via hypergraph neural network. In: Tanveer, M., Agarwal, S., Ozawa, S., Ekbal, A., Jatowt, A. (eds.) ICONIP 2022. LNCS, vol. 13624, pp. 219–230. Springer, Cham (2022)
Zhang, R., Zong, Q., Zhang, X., Dou, L., Tian, B.: Game of drones: multi-UAV pursuit-evasion game with online motion planning by deep reinforcement learning. IEEE Trans. Neural Networks Learn. Syst. 10, 7900–7909 (2022)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Shi, M., Liu, Q., Huang, Z. (2024). Efficient Collaboration via Interaction Information in Multi-agent System. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Lecture Notes in Computer Science, vol 14448. Springer, Singapore. https://doi.org/10.1007/978-981-99-8082-6_31
Download citation
DOI: https://doi.org/10.1007/978-981-99-8082-6_31
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8081-9
Online ISBN: 978-981-99-8082-6
eBook Packages: Computer ScienceComputer Science (R0)