Authors:
Mohamed Jabri
1
;
2
;
3
;
4
;
Panagiotis Papadakis
1
;
4
;
Ehsan Abbasnejad
2
;
3
;
4
;
Gilles Coppin
1
;
4
and
Javen Shi
2
;
3
;
4
Affiliations:
1
IMT Atlantique, Lab-STICC, UMR CNRS 6285, F-29238 Brest, France
;
2
The University of Adelaide, Adelaide, Australia
;
3
Australian Institute for Machine Learning, Adelaide, Australia
;
4
IRL CROSSING, CNRS, Adelaide, Australia
Keyword(s):
Imitation Learning, Causality, Structural Causal Models (SCMs), Counterfactual Reasoning.
Abstract:
Imitation learning has emerged as a pragmatic alternative to reinforcement learning for teaching agents to execute specific tasks, mitigating the complexity associated with reward engineering. However, the deployment of imitation learning in real-world scenarios is hampered by numerous challenges. Often, the scarcity and expense of demonstration data hinder the effectiveness of imitation learning algorithms. In this paper, we present a novel approach to enhance the sample efficiency of goal-conditioned imitation learning. Leveraging the principles of causality, we harness structural causal models as a formalism to generate counterfactual data. These counterfactual instances are used as additional training data, effectively improving the learning process. By incorporating causal insights, our method demonstrates its ability to improve imitation learning efficiency by capitalizing on generated counterfactual data. Through experiments on simulated robotic manipulation tasks, such as pus
hing, moving, and sliding objects, we showcase how our approach allows for the learning of better reward functions resulting in improved performance with a limited number of demonstrations, paving the way for a more practical and effective implementation of imitation learning in real-world scenarios.
(More)