Neural Paraphrase Generation with Multi-domain Corpus

Abstract

Automatic paraphrase generation is an important task for natural language processing. However, progress in paraphrase generation has been hindered for a long time by the lack of large monolingual parallel corpora. We can alleviate the data shortage by effectively using multi-domain corpus. In this paper, we propose a novel model to exploit information from other source domains (out-of-domains) which benefits our target domain (in-domain). In our method, we maintain a private encoder and a private decoder for each domain which are used to model domain-specific information. In the meantime, we introduce a shared encoder and a shared decoder shared by all domains which only contain domain-independent information. Besides, we add a domain discriminator to the shared encoder to reinforce the ability to capture common features of shared encoder by adversarial training. Experimental results show that our method not only perform well in traditional domain adaptation tasks but also improve performance in all domains together. Moreover, we show that the shared layer learned by our proposed model can be regarded as an off-the-shelf layer and can be easily adapted to new domains.