Competitive Collaboration for Complex Task Learning in Agent Systems

Abstract

This paper presents a novel competitive collaboration-based reinforcement learning strategy to improve the performance of goal-oriented autonomous agent systems. Competitive characteristics are introduced while agents are required to collaboratively share and update a single knowledge pool. Experimental evaluations are conducted in a reward collection task where the goal is to collect as many reward points as possible across a set space navigating through multiple channels while avoiding costs associated with channel switching. Evaluations across 50 challenge levels of the task in comparison to state-of-the-art reinforcement learning models reveal that introducing a combination of competition and collaboration attributes facilitate statistically significant performance improvements. The compared incremental models carry forward knowledge from previous challenge levels to learn more complex challenges. However, they resort to sub-optimal solutions where cost minimisation is given priority over higher reward collection. Traditional reinforcement learning is generally incapable of identifying a balance between reward collection and cost minimisation leading to poor performance. In contrast, the proposed competitive collaboration-based model is capable of strategically planning the best route while taking appropriate risks with higher costs for better rewards in the long run. Further, investigations with multiple reward schemes illustrate that frequent rewards to guide the agents’ strategies as opposed to delayed rewards that are awarded at less frequent intervals are imperative for agent performance. The presented evaluations lend insights into future research for optimising agent behaviours in complex real-world applications with competitive collaboration.