Binary Label-Aware Transfer Learning for Cross-Domain Slot Filling

Abstract

Slot filling plays an important role in spoken language understanding. Slot prediction need to use a lot of labeled data in a specific field for training, but in the real situation, there is often a lack of training data in a specific field, which is the biggest problem in cross-domain slot filling. In the previous works on cross-domain slot filling, many methods train their model through the sufficient source domain data, so that the model could predict the slot type in the unknown domain. However, previous approaches do not make good use of the small amount of labeled target domain data. In this paper, we proposed a cross-domain slot filling model with label-aware transfer learning. First, we classify words into three categories based on their BIO labels, calculate the MMD (maximum mean discrepancy) by computing hidden representations between two domains with the same ground truth label, which can participate in the loss function calculation, so that the model can better capture the overall characteristics of the target domain. Experimental results show that our proposed models significantly outperform other methods on average F1-score.