Abstract
Distant Supervision is a common technique for relation extraction from large amounts of free texts, but introduces wrong labeled sentences at the same time. Existing deep learning approaches mainly rely on CNN-based models. However, they fail to capture spatial patterns due to the inherent drawback of pooling operations and thus lead to suboptimal performance. In this paper, we propose a novel framework based on Selective Capsule Network for distant supervision relation extraction. Compared with traditional CNN-based models, the involvement of capsule layers in the sentence encoder makes it more powerful in encoding spatial patterns, which is very important in determining the relation expressed in a sentence. To address the wrong labeling problem, we introduce a high-dimensional selection mechanism over multiple instances. It is one generalization of traditional selective attention mechanism and can be seamlessly integrated with the capsule network based encoder. Experimental results on a widely used dataset (NYT) show that our model significantly outperform all the state-of-the-art methods.
This work was supported by NSFC (91646202), National Key R&D Program of China (SQ2018YFB140235).
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
Each capsule array here is a matrix rather than vector.
References
Bollacker, K.D., Evans, C., Paritosh, P., Sturge, T., Taylor, J.: Freebase: a collaboratively created graph database for structuring human knowledge. In: SIGMOD, pp. 1247–1250 (2008)
Bunescu, R.C., Mooney, R.J.: Subsequence kernels for relation extraction. In: NIPS, pp. 171–178 (2005)
Gormley, M.R., Yu, M., Dredze, M.: Improved relation extraction with feature-rich compositional embedding models. In: EMNLP, pp. 1774–1784 (2015)
Gregor, K., LeCun, Y.: Emergence of complex-like cells in a temporal product network with local receptive fields. CoRR abs/1006.0448 (2010). http://arxiv.org/abs/1006.0448
Hinton, G.E., Krizhevsky, A., Wang, S.D.: Transforming auto-encoders. In: Honkela, T., Duch, W., Girolami, M., Kaski, S. (eds.) ICANN 2011. LNCS, vol. 6791, pp. 44–51. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-21735-7_6
Hinton, G.E., Sabour, S., Frosst, N.: Matrix capsules with EM routing. In: ICLR (2018)
Hoffmann, R., Zhang, C., Ling, X., Zettlemoyer, L.S., Weld, D.S.: Knowledge-based weak supervision for information extraction of overlapping relations. In: ACL, pp. 541–550 (2011)
Jaiswal, A., AbdAlmageed, W., Natarajan, P.: Capsulegan: generative adversarial capsule network. CoRR abs/1802.06167 (2018)
Jiang, X., Wang, Q., Li, P., Wang, B.: Relation extraction with multi-instance multi-label convolutional neural networks. In: COLING, pp. 1471–1480 (2016)
Lin, Y., Shen, S., Liu, Z., Luan, H., Sun, M.: Neural relation extraction with selective attention over instances. In: ACL (2016)
Liu, L., et al.: Heterogeneous supervision for relation extraction: a representation learning approach. In: EMNLP, pp. 46–56 (2017)
Luo, B., et al.: Learning with noise: enhance distantly supervised relation extraction with dynamic transition matrix. In: ACL, pp. 430–439 (2017)
Luo, L., et al.: Beyond polarity: interpretable financial sentiment analysis with hierarchical query-driven attention. In: IJCAI, pp. 4244–4250 (2018)
Mintz, M., Bills, S., Snow, R., Jurafsky, D.: Distant supervision for relation extraction without labeled data. In: ACL, pp. 1003–1011 (2009)
Miwa, M., Bansal, M.: End-to-end relation extraction using LSTMs on sequences and tree structures. In: ACL (2016)
Riedel, S., Yao, L., McCallum, A.: Modeling relations and their mentions without labeled text. In: Balcázar, J.L., Bonchi, F., Gionis, A., Sebag, M. (eds.) ECML PKDD 2010. LNCS (LNAI), vol. 6323, pp. 148–163. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15939-8_10
Sabour, S., Frosst, N., Hinton, G.E.: Dynamic routing between capsules. In: NIPS, pp. 3859–3869 (2017)
Su, Y., Liu, H., Yavuz, S., Gur, I., Sun, H., Yan, X.: Global relation embedding for relation extraction. In: NAACL-HLT, pp. 820–830 (2018)
Surdeanu, M., Tibshirani, J., Nallapati, R., Manning, C.D.: Multi-instance multi-label learning for relation extraction. In: ACL, pp. 455–465 (2012)
Takamatsu, S., Sato, I., Nakagawa, H.: Reducing wrong labels in distant supervision for relation extraction. In: ACL, pp. 721–729 (2012)
Wang, J., Wang, Z., Zhang, D., Yan, J.: Combining knowledge with deep convolutional neural networks for short text classification. In: IJCAI, pp. 2915–2921 (2017)
Wang, W.Y., Xu, W., Qin, P.: DSGAN: generative adversarial training for distant supervision relation extraction. In: ACL, pp. 496–505 (2018)
Wang, W.Y., Xu, W., Qin, P.: Robust distant supervision relation extraction via deep reinforcement learning. In: ACL, pp. 2137–2147 (2018)
Wang, Y., Sun, A., Han, J., Liu, Y., Zhu, X.: Sentiment analysis by capsules. In: WWW, pp. 1165–1174 (2018)
Wu, F., Weld, D.S.: Open information extraction using Wikipedia. In: ACL, pp. 118–127 (2010)
Yan, Y., Okazaki, N., Matsuo, Y., Yang, Z., Ishizuka, M.: Unsupervised relation extraction by mining Wikipedia texts using information from the web. In: ACL, pp. 1021–1029 (2009)
Ye, H., Chao, W., Luo, Z., Li, Z.: Jointly extracting relations with class ties via effective deep ranking. In: ACL, pp. 1810–1820 (2017)
Zelenko, D., Aone, C., Richardella, A.: Kernel methods for relation extraction. J. Mach. Learn. Res. 3, 1083–1106 (2003)
Zeng, D., Liu, K., Chen, Y., Zhao, J.: Distant supervision for relation extraction via piecewise convolutional neural networks. In: EMNLP, pp. 1753–1762 (2015)
Zhao, K., et al.: Modeling patient visit using electronic medical records for cost profile estimation. In: Pei, J., Manolopoulos, Y., Sadiq, S., Li, J. (eds.) DASFAA 2018. LNCS, vol. 10828, pp. 20–36. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-91458-9_2
Zhao, W., Ye, J., Yang, M., Lei, Z., Zhang, S., Zhao, Z.: Investigating capsule networks with dynamic routing for text classification. CoRR abs/1804.00538 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, Z., Zhang, Y., Xing, C. (2019). Reducing Wrong Labels for Distant Supervision Relation Extraction with Selective Capsule Network. In: Shao, J., Yiu, M., Toyoda, M., Zhang, D., Wang, W., Cui, B. (eds) Web and Big Data. APWeb-WAIM 2019. Lecture Notes in Computer Science(), vol 11641. Springer, Cham. https://doi.org/10.1007/978-3-030-26072-9_6
Download citation
DOI: https://doi.org/10.1007/978-3-030-26072-9_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-26071-2
Online ISBN: 978-3-030-26072-9
eBook Packages: Computer ScienceComputer Science (R0)