Abstract
Relation extraction (RE) aims at identifying the relationship between two given entities and plays an essential role in natural language processing (NLP). Most of existing relation extraction models use convolutional or recurrent neural network and fail to capture the in-depth semantic features from the entities. These models also only focus on the training data and ignore external knowledge. In this paper, we propose a relation extraction model that makes use of external knowledge and the semantic roles of entities. In our model, we first adopt RoBERTa to make use of the knowledge learned from the unsupervised pretraining corpus. Then we obtain the semantic role embeddings and propose an entity attention network to select important words for relation extraction. We also offer the multi-task learning module and further improve our model by learning from auxiliary tasks. Our model obtains a Macro-F1 score of 89.96% on the benchmark dataset, outperforming most of the existing methods. More ablation experiments on two different datasets show that semantic role information and multi-task learning can help improve the relation extraction.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abujabal, A., Yahya, M., Riedewald, M., Weikum, G.: Automated template generation for question answering over knowledge graphs. In: WWW, pp. 1191–1200 (2017)
Zeng, D., Liu, K., Lai, S.: Relation classification via convolutional deep neural network. In: COLING 2014: Technical Papers, pp. 2335–2344. ACL, Stroudsburg (2014)
Socher, R., Huval, B., Manning, C.D.: Semantic compositionality through recursive matrix-vector spaces. In: EMNLP-CoNLL, pp. 1201–1211. ACL, Stroudsburg (2012)
Mikolov, T., Sutskever, I., Chen, K.: Distributed representations of words and phrases and their compositionality. In: NIPS, pp. 3111–3119. MIT Press, Cambridge (2013)
Devlin, J., Chang, M.W., Lee, K., et al.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Liu, Y., Ott, M., Goyal, N., et al.: RoBERTa: a robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019)
Yang, Z., Dai, Z., Yang, Y., et al.: XLNet: generalized autoregressive pretraining for language understanding. In: NIPS. Cambridge, MIT Press, pp. 5754–5764 (2019)
Liu, X., He, P., Chen, W., et al.: Multi-task deep neural networks for natural language understanding. arXiv preprint arXiv:1901.11504 (2019)
Rink, B., Harabagiu, S.: UTD: classifying semantic relations by combining lexical and semantic resources. In: SemEval, pp. 256–259. ACL, Stroudsburg (2010)
Collins, M., Duffy, N. :Convolution kernels for natural language. In: Advances in Neural Information Processing Systems, pp. 625–632. MIT Press, Cambridge (2002)
Santos, C.N., Xiang, B., Zhou, B.: Classifying relations by ranking with convolutional neural networks. arXiv preprint arXiv:1504.06580 (2015)
Lee, J., Seo, S., Choi, Y.S.: Semantic relation classification via bidirectional LSTM networks with entity-aware attention using latent entity typing. Symmetry 11(6), 785 (2019)
Mandya, A., Bollegala, D., Coenen, F.: Contextualised graph attention for improved relation extraction. arXiv preprint arXiv:2004.10624 (2020)
Wu, S., He, Y.: Enriching pre-trained language model with entity information for relation classification. In: CIKM, pp. 2361–2364. ACM, New York (2019)
Hoffmann, R., Zhang, C., Ling, X.: Knowledge-based weak supervision for information extraction of overlapping relations. In: ACL HLT, vol. 1, pp. 541–550 (2011)
Surdeanu, M., Tibshirani, J., Nallapati, R.: Multi-instance multi-label learning for relation extraction. In: Empirical Methods in Natural Language Processing, pp. 455–465 (2012)
Gardner, M., Grus, J., Neumann, M., et al.: AllenNLP: a deep semantic natural language processing platform. arXiv preprint arXiv:1803.07640 (2018)
Warstadt, A., Singh, A., Bowman, S.R.: Neural network acceptability judgments. Trans. Assoc. Comput. Linguist. 7, 625–641 (2019)
Socher, R., Perelygin, A., Wu, J., et al.: Recursive deep models for semantic compositionality over a sentiment treebank. In: EMNLP, pp. 1631–1642. ACL, Stroudsburg (2013)
Rajpurkar, P., Zhang, J., Lopyrev, K., et al.: Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250 (2016)
Conneau, A., Kiela, D.: SentEval: an evaluation toolkit for universal sentence representations. arXiv preprint arXiv:1803.05449 (2018)
Williams, A., Nangia, N., Bowman, S.R.: A broad-coverage challenge corpus for sentence understanding through inference. arXiv preprint arXiv:1704.05426 (2017)
Hendrickx, I., Kim, S.N., Kozareva, Z.: SemEval-2010 task 8: multi-way classification of semantic relations between pairs of nominals. In: SemEval, Stroudsburg, PA, pp. 94–99 (2009)
Han, X., Gao, T., Yao, Y., et al.: OpenNRE: an open and extensible toolkit for neural relation extraction. arXiv preprint arXiv:1909.13078 (2019)
Tao, Q., Luo, X., Wang, H., et al.: Enhancing relation extraction using syntactic indicators and sentential contexts. In: ICTAI, Piscataway, NJ, pp. 574–1580. IEEE (2019)
Acknowledgments
This work was supported in part by Research and Development Program in Key Areas of Guangdong Province under Grant 2018B010109004, in part by the National Natural Science Foundation of China under Grant 61936003, and in part by the Applied Scientific and Technology Special Project of Department of Science and Technology of Guangdong Province under Grant 20168010124010.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Zhu, Z., Su, J., Hong, X. (2021). Improving Relation Extraction Using Semantic Role and Multi-task Learning. In: Chen, H., Liu, K., Sun, Y., Wang, S., Hou, L. (eds) Knowledge Graph and Semantic Computing: Knowledge Graph and Cognitive Intelligence. CCKS 2020. Communications in Computer and Information Science, vol 1356. Springer, Singapore. https://doi.org/10.1007/978-981-16-1964-9_8
Download citation
DOI: https://doi.org/10.1007/978-981-16-1964-9_8
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-1963-2
Online ISBN: 978-981-16-1964-9
eBook Packages: Computer ScienceComputer Science (R0)