Task Inference for Offline Meta Reinforcement Learning via Latent Shared Knowledge

Abstract

Offline Reinforcement Learning (RL) has emerged as a promising approach for learning from existing data without requiring online interactions. However, traditional offline RL algorithms often suffer from poor generalization and overfitting due to limited task diversity in the training data. In this paper, we propose a novel framework called Meta-Task (MeTask) for offline RL that leverages meta-learning techniques to learn a task representation from a diverse set of offline training tasks. Specifically, we introduce a task-shared meta-learning objective that extracts meta-knowledge from the context data of each task and uses it to learn a more generalizable task representation. Additionally, we design a task-infer module that restores the learned meta-knowledge and task-specific information between different tasks to achieve efficient transfer of knowledge. Experiments on a variety of benchmark tasks demonstrate that MeTask achieves state-of-the-art performance compared to traditional offline RL algorithms. These results suggest that leveraging task diversity and meta-learning techniques can significantly improve the efficiency of offline RL methods.