skip to main content
10.1145/3409501.3409535acmotherconferencesArticle/Chapter ViewAbstractPublication PageshpcctConference Proceedingsconference-collections
research-article

Improving relation extraction by multi-task learning

Published: 25 August 2020 Publication History

Abstract

Relation extraction is a subtask of information extraction. Current relation extraction methods are mainly designed for relation extraction tasks, and they use limited knowledge. In this paper, we propose a relation extraction method based on multi-task learning. It uses multiple tasks to learn features that are hard to learn from the relation extraction task, and we add knowledge distillation to help the multi-task model perform better than its single-task counterparts. The experiments based on the pre-trained language model BERT show that our method performs better than most relation extraction methods on the SemEval2010-task8 dataset.

References

[1]
Dongsheng Wang, Prayag Tiwari, Sahil Garg, et al. 2020. Structural block driven enhanced convolutional neural representation for relation extraction. Applied Soft Computing, Volume 86, 105913.
[2]
Dos Santos C, Xiang B, Zhou B. 2015. Classifying Relations by Ranking with Convolutional Neural Networks. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 626--634.
[3]
Huang X. 2016. Attention-based convolutional neural network for semantic relation extraction. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, 2526--2536.
[4]
Wang L, Cao Z, De Melo G, et al. 2016. Relation Classification via Multi-Level Attention CNNs. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 1298--1307.
[5]
Lei, M., Huang, H., Feng, C. et al. 2019. An input information enhanced model for relation extraction. Neural Comput & Applic, 31, 9113--9126.
[6]
Sun, J., Li, Y., Shen, Y., Ding, W., Shi, X., et al. 2019. Joint Self-Attention Based Neural Networks for Semantic Relation Extraction. Journal of Information Hiding and Privacy Protection, 1(2), 69--75.
[7]
ZhiQiang Geng, GuoFei Chen, YongMing Han, et al. 2020. Semantic relation extraction using sequential and tree-structured LSTM with attention. Information Sciences, Volume 509, Pages 183--192.
[8]
Liu Y, Li F W S. 2015. A dependency-based neural network for relation classification. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers).
[9]
Xu K, Feng Y, Huang S, et al. 2015. Semantic Relation Classification via Convolutional Neural Networks with Simple Negative Sampling. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, 536--540.
[10]
Xu K, Feng Y, Huang S, et al. 2015. Semantic Relation Classification via Convolutional Neural Networks with Simple Negative Sampling. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, 536--540.
[11]
Xu Y, Jia R, Mou L, et al. 2016. Improved relation classification by deep recurrent neural networks with data augmentation. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, 1461--1470.
[12]
Cai R, Zhang X, Wang H. 2016. Bidirectional recurrent convolutional neural network for relation classification. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 756--765.
[13]
Zeng D, Liu K, Chen Y, et al. 2015. Distant supervision for relation extraction via piecewise convolutional neural networks. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, 1753--1762.
[14]
Ji G, Liu K, He S, et al. 2017. Distant supervision for relation extraction with sentence-level attention and entity descriptions. In Thirty-First AAAI Conference on Artificial Intelligence.
[15]
Ruder S. 2017. An overview of multi-task learning in deep neural networks. arXiv preprint arXiv:1706.05098.
[16]
Aone C, Halverson L, Hampton T, et al. 1998. SRA: Description of the IE2 system used for MUC-7. In Seventh Message Understanding Conference (MUC-7): Proceedings of a Conference Held in Fairfax, Virginia.
[17]
Kambhatla N. 2004. Combining lexical, syntactic, and semantic features with maximum entropy models for extracting relations. In Proceedings of the ACL 2004 on Interactive poster and demonstration sessions, 22.
[18]
Zhao S, Grishman R. 2005. Extracting relations with integrated information using kernel methods. In Proceedings of the 43rd annual meeting on association for computational linguistics, 419--426.
[19]
Brin S. 1998. Extracting patterns and relations from the world wide web. In International Workshop on The World Wide Web and Databases, 172--183.
[20]
Agichtein E, Gravano L. 2000. Snowball: extracting relations from large plain-text collections. In Proceedings of the fifth ACM conference on Digital libraries, 85--94.
[21]
Hasegawa T, Sekine S, Grishman R. 2004. Discovering relations among named entities from large corpora. In Proceedings of the 42nd annual meeting on association for computational linguistics, 415.
[22]
Shinyama Y, Sekine S. 2006. Preemptive information extraction using unrestricted relation discovery. In Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics, 304--311.
[23]
Devlin J, Chang M W, Lee K, et al. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
[24]
Zhang Y, Yang Q. 2017. A survey on multi-task learning. arXiv preprint arXiv:1707.08114.
[25]
Alonso H M, Plank B. 2016. When is multitask learning effective? Semantic sequence prediction under varying data conditions. arXiv preprint arXiv:1612.02251.
[26]
Li F, Zhang M, Fu G, et al. 2017. A neural joint model for entity and relation extraction from biomedical text. BMC bioinformatics, 18(1), 198.
[27]
Suncong Zheng, Jiaming Xu, et al. 2016. A neural network framework for relation extraction: Learning entity semantic and relation pattern. Knowledge-Based Systems, Volume 114, Pages 12--23
[28]
Wang A, Singh A, Michael J, et al. 2018. GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, 353--355.
[29]
Hinton G, Vinyals O, Dean J. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531.
[30]
Hendrickx I, Kim S N, Kozareva Z, et al. 2009. Semeval-2010 task 8: Multi-way classification of semantic relations between pairs of nominals. In Proceedings of the Workshop on Semantic Evaluations: Recent Achievements and Future Directions, 94--99.
[31]
Zhang Y, Qi P, Manning C D. 2018. Graph Convolution over Pruned Dependency Trees Improves Relation Extraction. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 2205--2215.
[32]
Samuel R Bowman, Ellie Pavlick, Edouard Grave, et al. 2018. Looking for ELMo's friends: Sentence-level pretraining beyond language modeling. arXiv preprint arXiv:1812.10860.
[33]
Yi Zhao, Huaiyu Wan, Jianwei Gao, Youfang Lin. 2019. Improving Relation Classification by Entity Pair Graph. In Proceedings of The Eleventh Asian Conference on Machine Learning, PMLR 101, 1156--1171.

Cited By

View all
  • (2023)Analysis of Digital Information in Storage Devices Using Supervised and Unsupervised Natural Language Processing TechniquesFuture Internet10.3390/fi1505015515:5(155)Online publication date: 23-Apr-2023
  • (2021)A Multitask Learning Model with Multiperspective Attention and Its Application in RecommendationComputational Intelligence and Neuroscience10.1155/2021/85502702021Online publication date: 15-Oct-2021
  • (2021)Multi-Entity Collaborative Relation ExtractionICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP39728.2021.9413673(7678-7682)Online publication date: 6-Jun-2021

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
HPCCT & BDAI '20: Proceedings of the 2020 4th High Performance Computing and Cluster Technologies Conference & 2020 3rd International Conference on Big Data and Artificial Intelligence
July 2020
276 pages
ISBN:9781450375603
DOI:10.1145/3409501
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

  • Xi'an Jiaotong-Liverpool University: Xi'an Jiaotong-Liverpool University

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 August 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Information Extraction
  2. Knowledge Distillation
  3. Multi-task Learning
  4. Relation Extraction

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

HPCCT & BDAI 2020

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)11
  • Downloads (Last 6 weeks)0
Reflects downloads up to 20 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2023)Analysis of Digital Information in Storage Devices Using Supervised and Unsupervised Natural Language Processing TechniquesFuture Internet10.3390/fi1505015515:5(155)Online publication date: 23-Apr-2023
  • (2021)A Multitask Learning Model with Multiperspective Attention and Its Application in RecommendationComputational Intelligence and Neuroscience10.1155/2021/85502702021Online publication date: 15-Oct-2021
  • (2021)Multi-Entity Collaborative Relation ExtractionICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP39728.2021.9413673(7678-7682)Online publication date: 6-Jun-2021

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media