Processing math: 100%
State Augmentation via Self-Supervision in Offline Multiagent Reinforcement Learning | IEEE Journals & Magazine | IEEE Xplore

State Augmentation via Self-Supervision in Offline Multiagent Reinforcement Learning


Abstract:

The utilization of precollected offline data sets for learning in the absence of environmental interaction has enabled reinforcement learning (RL) to make significant str...Show More

Abstract:

The utilization of precollected offline data sets for learning in the absence of environmental interaction has enabled reinforcement learning (RL) to make significant strides in real-world circumstances. This approach is also attractive for multiagent RL (MARL) tasks, given the complex interactions that occur between agents and the environment. However, when compared to the single-agent approach, offline MARL faces more challenges due to the larger state and action space, particularly with regard to poor out-of-distribution generalization to the environment. The present study demonstrates the ineffectiveness of directly transferring conservative offline RL algorithms from single-agent settings to multiagent environments, which is due to the accumulating extrapolation errors that increase in proportion to the number of agents. In this article, we explore the efficacy of three types of data augmentation techniques that can be applied to the state representation in the context of MARL. By combining the proposed data augmentation techniques with a state-of-the-art offline multiagent algorithm, we improve the function approximation of centralized Q -networks. The experimental results conducted on StarCraft II strongly support the effectiveness of the data augmentation techniques in enhancing the performance of offline MARL in the state space.
Page(s): 1051 - 1062
Date of Publication: 23 October 2023

ISSN Information:

Funding Agency:


References

References is not available for this document.