Abstract
Distant supervision is an effective method to generate large-scale labeled data for relation extraction without expensive manual annotation, but it inevitably suffers from the wrong labeling problem, which would make the corpus much noisy. However, the existing research work mainly focuses on sentence-level noise filtering, without considering noisy words which widely exist inside sentences. In this paper, we propose an attention-based gated piecewise convolutional neural networks (AGPCNNs) for distant supervised relation extraction, which can effectively reduce word-level noise by selecting the inner-sentence features. On the one hand, we construct a piecewise convolutional neural network with gate mechanism to extract features that are related to relations. On the other hand, we employ a soft-label strategy to enable model to select important features automatically. Furthermore, we adopt an attention mechanism after the piecewise pooling layer to obtain high-level positive features for relation predicting. Experimental results show that our method can effectively filter word-level noise and outperforms all baseline systems significantly.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Mintz, M., Bills, S., Snow, R., Jurafsky, D.: Distant supervision for relation extraction without labeled data. In: Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pp. 1003–1011 (2009)
Riedel, S., Yao, L., McCallum, A.: Modeling relations and their mentions without labeled text. In: Machine Learning and Knowledge Discovery in Databases, pp. 148–163 (2010)
Dietterich, T.G., Lathrop, R.H., Lozano-Pérez, T.: Solving the multiple instance problem with axis-parallel rectangles. Artif. Intell. 89(1–2), 31–71 (1997)
Hoffmann, R., Zhang, C., Ling, X., Zettlemoyer, L., Weld, D. S.: Knowledge-based weak supervision for information extraction of overlapping relations. In: Proceedings of ACL, pp. 541–550. Association for Computational Linguistics (2011)
Surdeanu, M., Tibshirani, J., Nallapati, R., Manning, C.D.: Multi-instance multi-label learning for relation extraction. In: Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 455–465. Association for Computational Linguistics (2012)
Zeng, D., Liu, K., Chen, Y., Zhao, J.: Distant supervision for relation extraction via piecewise convolutional neural networks. In: Proceedings of EMNLP, pp. 1753–1762 (2015)
Lin, Y., Shen, S., Liu, Z., Luan, H., Sun, M.: Neural relation extraction with selective attention over instances. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2124–2133 (2016)
Ji, G., Liu, K., He, S., Xu, L., Zhao, J.: Distant supervision for relation extraction with sentence-level attention and entity descriptions. In: AAAI, pp. 3060–3066 (2017)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Wu, W., Chen, Y., Xu, J., Zhang, Y.: Attention-Based Convolutional Neural Networks for Chinese Relation Extraction. In: Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data, pp. 147–158. Springer, Cham (2018)
Wang, G., Zhang, W., Wang, R., Zhou, Y., Chen, X., Zhang, W., Zhu, H., Chen, H.: Label-free distant supervision for relation extraction via knowledge graph embedding. In: Proceedings of EMNLP, pp. 2246–2255 (2018)
Kalchbrenner, N., Espeholt, L., Simonyan, K., Oord, A., Graves, A., Kavukcuoglu, K.: Neural machine translation in linear time. arXiv preprint arXiv, 1610.10099 (2016)
Gehring, J., Auli, M., Grangier, D., Yarats, D., Dauphin, Y.: Convolutional sequence to sequence learning. In: Proceedings of the 34th International Conference on Machine Learning-Volume 70, pp. 1243–1252. JMLR. org (2017)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv, 1301.3781 (2013)
Liu, T., Zhang, X., Zhou, W., Jia, W.: Neural relation extraction via inner-sentence noise reduction and transfer learning. In: Proceedings of EMNLP, pp. 2195–2204 (2018)
Feng, J., Huang, M., Zhao, L., Yang, Y., Zhu X.: Reinforcement learning for relation classification from noisy data. In: AAAI, pp. 5779–5786 (2018)
Qin, P., Xu, W., Wang, W.Y.: Robust distant supervision relation extraction via deep reinforcement learning. In: Proceedings of ACL, pp. 2137–2147 (2018)
Acknowledgments
The authors are supported by the National Nature Science Foundation of China (Nos. 61473294, 61370130 and 61876198), the Fundamental Research Funds for the Central Universities (Nos. 2015JBM033), and the International Science and Technology Cooperation Program of China (No. K11F100010).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Li, X., Chen, Y., Xu, J., Zhang, Y. (2019). Attention-Based Gated Convolutional Neural Networks for Distant Supervised Relation Extraction. In: Sun, M., Huang, X., Ji, H., Liu, Z., Liu, Y. (eds) Chinese Computational Linguistics. CCL 2019. Lecture Notes in Computer Science(), vol 11856. Springer, Cham. https://doi.org/10.1007/978-3-030-32381-3_20
Download citation
DOI: https://doi.org/10.1007/978-3-030-32381-3_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32380-6
Online ISBN: 978-3-030-32381-3
eBook Packages: Computer ScienceComputer Science (R0)