Distant Supervision Relation Extraction with Improved PCNN and Multi-level Attention

Abstract

In the research field of relation extraction, a large amount of labeled training data can be quickly obtained through distant supervision. However, distant supervision will be inevitably accompanied by the problem of wrong labelling, and the noisy data can substantially hurt the performance of relation extraction model. Previous studies mainly focus on de-noising by designing neural networks with intra-bag or inter-bag attention, but do not sufficiently take noise interference at different levels into account. Moreover, the conventional approach Piecewise Convolutional Neural Network (PCNN) encodes sentences based on an assumption that each segmented feature contributes equally to the relation, which is unreasonable. To alleviate these issues, we propose a distant supervision relation extraction model with Improved PCNN (I-PCNN) and multi-level attention. By incorporating word-level, sentence-level and bag-level attention into the model, it can effectively reduce the noisy data in a data set. Besides, it also enhances PCNN by augmenting self-attention, which can promote the encoding quality of sentences. Experiments conducted on New York Times (NYT) data set show that the proposed model evidently outperforms several baseline models in terms of the metrics $Precision$, $Recall$, $AUC$, and $P@N$.