Gaurav Dixit
ABSTRACT
Evolutionary optimization is difficult in domains that require heterogeneous agents to coordinate on diverse tasks as agents often converge to a limited set of "acceptable" behaviors. Quality-Diversity methods alleviate this problem by shifting the focus from optimizing to finding a diverse repertoire of behaviors. However, in multiagent environments with diverse and tightly-coupled tasks, exploring the entire space of behaviors is often intractable. Agents must focus on searching regions of the behavior space that yield behaviors for good team performance. We extend Behavior Exploration for Heterogeneous Teams (BEHT)[4], a multi-level training framework that allows systematic exploration of the agents' behavior space required to complete diverse tasks as a coordinated team in a dynamic environment. We show that BEHT allows agents to learn diverse synergies that are demonstrated by the diversity of acquired agent behavior in response to the changing environment and agent behaviors.
- Adrian Agogino and Kagan Tumer. 2009. Learning indirect actions in complex domains: action suggestions for air traffic control. Advances in Complex Systems 12, 04n05 (2009), 493--512.Google Scholar
Cross Ref
- Stefano V Albrecht and Peter Stone. 2018. Autonomous agents modelling other agents: A comprehensive survey and open problems. Artificial Intelligence 258 (2018), 66--95.Google ScholarCross Ref
- Jonathan C Brant and Kenneth O Stanley. 2020. Diversity preservation in minimal criterion coevolution through resource limitation. In Proceedings of the 2020 Genetic and Evolutionary Computation Conference. 58--66.Google ScholarDigital Library
- Gaurav Dixit and Kagan Tumer. 2022. Behavior Exploration and Team Balancing for Heterogeneous Multiagent Coordination. In Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems.Google ScholarDigital Library
- Gaurav Dixit, Nicholas Zerbel, and Kagan Tumer. 2019. Dirichlet-Multinomial Counterfactual Rewards for Heterogeneous Multiagent Systems. In 2019 International Symposium on Multi-Robot and Multi-Agent Systems (MRS). IEEE, 209--215.Google Scholar
- Shauharda Khadka, Somdeb Majumdar, Santiago Miret, Stephen McAleer, and Kagan Tumer. 2019. Evolutionary reinforcement learning for sample-efficient multiagent coordination. arXiv preprint arXiv:1906.07315 (2019).Google Scholar
- Joel Lehman and Kenneth O Stanley. 2011. Evolving a diversity of virtual creatures through novelty search and local competition. In Proceedings of the 13th annual conference on Genetic and evolutionary computation. 211--218.Google ScholarDigital Library
- Joel Z. Leibo, Julien Perolat, Edward Hughes, Steven Wheelwright, Adam H. Marblestone, Edgar Duéñez Guzmán, Peter Sunehag, Iain Dunning, and Thore Graepel. 2019. Malthusian Reinforcement Learning. In Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems (Montreal QC, Canada) (AAMAS '19). International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, 1099--1107.Google ScholarDigital Library
- Justin K Pugh, Lisa B Soros, and Kenneth O Stanley. 2016. Quality diversity: A new frontier for evolutionary computation. Frontiers in Robotics and AI 3 (2016), 40.Google ScholarCross Ref
- John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017).Google Scholar
Index Terms
- Balancing teams with quality-diversity for heterogeneous multiagent coordination
Recommendations
Diversifying behaviors for learning in asymmetric multiagent systems
GECCO '22: Proceedings of the Genetic and Evolutionary Computation ConferenceTo achieve coordination in multiagent systems such as air traffic control or search and rescue, agents must not only evolve their policies, but also adapt to the behaviors of other agents. However, extending coevolutionary algorithms to complex domains ...
Heterogeneous agent coordination via adaptive quality diversity and specialization
GECCO '21: Proceedings of the Genetic and Evolutionary Computation Conference CompanionIn many real-world multiagent systems, agents must learn diverse tasks and coordinate with other agents. This paper introduces a method to allow heterogeneous agents to specialize and only learn complementary divergent behaviors needed for coordination ...
Behavior Exploration and Team Balancing for Heterogeneous Multiagent Coordination
AAMAS '22: Proceedings of the 21st International Conference on Autonomous Agents and Multiagent SystemsDiversity in behaviors is instrumental for robust team performance in many multiagent tasks which require agents to coordinate. Unfortunately, exhaustive search through the agents' behavior spaces is often intractable. This paper introduces Behavior ...
Comments