Inter-task Similarity Measure for Heterogeneous Tasks

Abstract

A long term goal in robotics is lifelong learning, in which a robot learns to perform new tasks without human intervention. For this a promising alternative is reinforcement learning (RL), in which an agent learns how to solve sequential decision-making problems by interacting with its environment. Despite the effectiveness of RL, learning in a trial-and-error way can be infeasible in robotics applications, in which long training periods can significantly deteriorate hardware. For robots that face long sequences of tasks, transfer learning represents an alternative to reduce training time, by transferring knowledge from previously learned tasks. However, in domains such as service robotics, identifying reusable pieces of knowledge and transferring them across tasks can be challenging, due to the mismatch between the state-action spaces (heterogeneous tasks). Thus, in this article we present an inter-task similarity measure and a transfer learning method for heterogeneous tasks. Through experimental evaluations, we show that the proposed measure is able to rank tasks in a way that prioritizes those that cause a larger increment of the learning performance in a target task. Additionally, results show that for some tasks the similarity measure can be computed with few data, enabling significant speedups in the learning process by transferring knowledge at an early stage.