Combining Domain Knowledge and Deep Learning Makes NMT More Adaptive

Abstract

In both SMT (statistical machine translation) and NMT (neural machine translation), training data often varies in source, theme and genre. It is less likely that the training data and texts in practical translation fall into a same domain, leading to a sub-optimal performance. Domain adaptation is to address such problems. Existing domain adaptive approach in machine translation employs topic model to obtain topic information. However, thus domain labels can be very much limited to in-domain and out-of-domain, when dividing topics into two types, without any more specific labels. We propose a novel domain adaptive approach to annotate Chinese sentences with CLCN (Chinese Library Classification Number) as the domain labels. We design a deep fusion model of neural network to combine two annotating models, including one applying a domain knowledge base built on thesis keywords and Chinese Scientific and Technical Vocabulary System, and the other applying deep learning method based on a CNN. Then, we have the fused domain annotator to filter the training data of NMT according to the test data. After running two predefined domain test sets on a NMT system trained by only partial of the original training data, we achieve an average 1.3 BLEU score improvement (5.4% relative), which demonstrates the feasibility and validity of proposed approach.