A Reinforcement Learning Approach for Abductive Natural Language Generation

Abstract

Teaching deep learning models commonsense knowledge is a crucial yet challenging step towards building human-level artificial intelligence. Abductive Commonsense Reasoning ($\mathcal {ART}$) is a benchmark that investigates model’s ability on inferencing the most plausible explanation within the given context, which requires model using commonsense knowledge about the world. $\mathcal {ART}$ consists of two datasets, $\alpha$NLG and $\alpha$NLI, that challenge models from generative and discriminative settings respectively. Despite the fact that both of the datasets investigate the same ability, existing work solves them independently. In this work, we address $\alpha$NLG in a teacher-student setting by getting help from another model with adequate commonsense knowledge fully-trained on $\alpha$NLI. We fulfill this intuition by representing the desired optimal generation model as an Energy-Based Model and training it using a reinforcement learning algorithm. Experiment results showed that our model achieve state-of-the-art results on both automatic and human evaluation metrics, which have demonstrated the effectiveness and feasibility of our model (Code available in https://github.com/Huanghongru/commonsense-generation).