How to Generate Reasonable Texts with Controlled Attributes

Abstract

The controllable text generation (CTG) task is crucial for text-related applications, such as goal-oriented dialogue systems and text style-transfer applications, etc. However, existing CTG methods commonly ignore the co-occurrence dependencies between multiple controlled attributes, which are implicit in domain knowledge. As a result, rarely co-occurring controlled values are highly likely to be given by users, which finally leads to non-committal generated texts that are out of control. To address this problem, we initially propose the Dependency-aware Controllable Text Generation (DCTG) model that reduces trivial generations by automatically learning the co-occurrence dependencies and adjusting rarely co-occurring controlled values. Our DCTG highlights in (1) modeling the co-occurrence dependencies between controlled attributes with neural networks, (2) integrating dependency losses to guide each component of our model to collaboratively work for generating reasonable texts based on the learned dependencies, and (3) proposing a novel Reasonableness metric measuring to which degree generations comply with real co-occurrence dependencies. Experiments prove that DCTG outperforms state-of-the-art baselines on three datasets in multiple aspects.

Y. Zheng and Y. Wang—Both are first authors with equal contributions.