Abstract
Swarm systems with simple, homogeneous and autonomous individuals can efficiently accomplish specified complex tasks. Recent works have shown the power of deep reinforcement learning (DRL) methods to learn cooperative policies for swarm systems. However, most of them show poor adaptability when applied to new environments or tasks. In this paper, we propose a novel semantic perception swarm policy with DRL for distributed swarm systems. This policy implements innovative semantic perception, which enables agents to understand their observation information, yielding semantic information, to promote agents’ adaptability. In particular, semantic disentangled representation with posterior distribution and semantic mixture representation with network mapping are realized to represent semantic information of agents’ observations. Moreover, in the semantic representation, heterogeneous graph attention network is adopted to effectively model individual-level and group-level relational information. The distributed and transferable swarm policy can perceive the information of uncertain number of agents in swarm environments. Various simulations and real-world experiments on several challenging tasks, i.e., sheep food collection and wolves predation, demonstrate the superior effectiveness and adaptability performance of our method compared with existing methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Agarwal, A., Kumar, S., Sycara, K.: Learning transferable cooperative behavior in multi-agent teams (2019). arXiv preprint arXiv:1906.01202
Arnold, R.D., Yamaguchi, H., Tanaka, T.: Search and rescue with autonomous flying robots through behavior-based cooperative intelligence. J. Int. Humanit. Action 3(1), 1–18 (2018). https://doi.org/10.1186/s41018-018-0045-4
Azzouni, J.: Semantic Perception: How the Illusion of a Common Language Arises and Persists. Oxford University Press, Oxford (2015)
Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013)
Blei, D.M., Kucukelbir, A., McAuliffe, J.D.: Variational inference: a review for statisticians. J. Am. Stat. Assoc 112(518), 859–877 (2017)
Henson, C., Sheth, A., Thirunarayan, K.: Semantic perception: converting sensory observations to abstractions. IEEE Internet Comput. 16(2), 26–34 (2012)
Higgins, I., et al.: beta-vae: learning basic visual concepts with a constrained variational framework (2016)
Hostallero, W.J.K.D.E., Son, K., Kim, D., Qtran, Y.Y.: Learning to factorize with transformation for cooperative multi-agent reinforcement learning. In: Proceedings of the 31st International Conference on Machine Learning, Proceedings of Machine Learning Research. PMLR (2019)
Hüttenrauch, M., Adrian, S., Neumann, G., et al.: Deep reinforcement learning for swarm systems. J. Mach. Learn. Res. 20(54), 1–31 (2019)
Hüttenrauch, M., Šošić, A., Neumann, G.: Guided deep reinforcement learning for swarm systems (2017). arXiv preprint arXiv:1709.06011
Jiang, J., Lu, Z.: Learning attentional communication for multi-agent cooperation. In: Advances in Neural Information Processing Systems, pp. 7254–7264 (2018)
Kingma, D.P., Welling, M.: Auto-encoding variational bayes (2013). arXiv preprint arXiv:1312.6114
Liu, Y., Wang, L., Huang, H., Liu, M., Xu, C.Z.: A novel swarm robot simulation platform for warehousing logistics. In: 2017 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 2669–2674. IEEE (2017)
Lowe, R., Wu, Y.I., Tamar, A., Harb, J., Abbeel, O.P., Mordatch, I.: Multi-agent actor-critic for mixed cooperative-competitive environments. In: Advances in Neural Information Processing Systems, pp. 6379–6390 (2017)
Mao, H., et al.: Neighborhood cognition consistent multi-agent reinforcement learning (2019). arXiv preprint arXiv:1912.01160
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms (2017). arXiv preprint arXiv:1707.06347
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks (2017). arXiv preprint arXiv:1710.10903
Wang, W., et al.: From few to more: large-scale dynamic multiagent curriculum learning. In: AAAI, pp. 7293–7300 (2020)
Wang, X., et al.: Heterogeneous graph attention network. In: The World Wide Web Conference, pp. 2022–2032 (2019)
Zhou, Z., Zhang, W., Ding, J., Huang, H., Stipanović, D.M., Tomlin, C.J.: Cooperative pursuit with voronoi partitions. Automatica 72, 64–72 (2016)
Acknowledgment
This work was supported by the National Key Research and Development Program of China under Grant 2018AAA0102402, the National Natural Science Foundation of China under Grant 62073323, the Strategic Priority Research Program of Chinese Academy of Sciences under Grant No. XDA27030403, and the External Cooperation Key Project of Chinese Academy Sciences No. 173211KYSB20200002.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, T., Liu, Z., Pu, Z., Yi, J. (2021). Semantic Perception Swarm Policy with Deep Reinforcement Learning. In: Mantoro, T., Lee, M., Ayu, M.A., Wong, K.W., Hidayanto, A.N. (eds) Neural Information Processing. ICONIP 2021. Lecture Notes in Computer Science(), vol 13110. Springer, Cham. https://doi.org/10.1007/978-3-030-92238-2_10
Download citation
DOI: https://doi.org/10.1007/978-3-030-92238-2_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-92237-5
Online ISBN: 978-3-030-92238-2
eBook Packages: Computer ScienceComputer Science (R0)