An entity relation extraction algorithm based on BERT(wwm-ext)-BiGRU-Attention
Pages 130 - 135
Abstract
Entity relation extraction is one of the basic steps of knowledge Graph. It identifies the relations between entities. A BERT-Bidirectional gated recurrent units-Attention mechanism (BERT-BiGRU-Attention) model has been proposed, but it is based on the single Chinese character based masking. Due to the complexity of Chinese grammar structure and the semantic diversity, a BERT(wwm-ext) was proposed based on the whole Chinese word masking. In this paper we propose a BERT(wwm-ext)-BiGRU-Attention model. The experimental result shows that for the purpose of entity relation extraction the precision is 93.60%, recall rate is 91.90%, and F1 value 92.53%, which are higher than the BERT-BiGRU-Attention and its precision is 91.80%, recall rate is 90.16%, and F1 value is 90.97%. Since BERT(wwm-ext)-BIGRU-Attention gets higher precision, F1 value, and higher recall rate, it has better effects on the Chinese entity relation extraction tasks.
References
[1]
Muyijie Z, Bingkun B, Changsheng X U. Research progress on development and construction of knowledge graph[J]. Journal of Nanjing University of Information ence & Technology (Natural ence Edition), 2017.
[2]
Pao-li Li, Yu-zhong Chen, Shi-man Yu. Overview of information Extraction research [J]. Computer Engineering and Application, 2003, 039 (010):1--5, 66.
[3]
Wang L, Chang H, Chang C. Recognizing unregistered names for Mandarin word identification[C]// International Conference on Computation Linguistics, 1992:1239--1243.
[4]
Aone C, Halverson L, Hampton T, et al. SRA: Description of the IE2 system used for MUC-7[C]//Seventh Message Understanding Conference (MUC-7): Proceedings of a Conference Held in Fairfax, Virginia, April 29-May 1, 1998. 1998.
[5]
KAMBHATLA N. Combining lexical, syntactic, and semantic features with maximum entropy models for extracting relations[C]// Proceedings of the 43rd ACL International Conference.[S.l.]: Association for Computational Linguistics, 2004: 178--181.
[6]
Xiao Sun, Chongyuan Sun, Fuji Ren. Biomedical named entity Recognition based on deep condition random field [J]. Pattern recognition and artificial intelligence, 2016, 29(11):997--1008.
[7]
Jiahui Hu, An Fang, Wanqing Zhao, et al. Research on Chinese Electronic Medical Record Annotation Method for knowledge Discovery [J]. Data Analysis and Knowledge Discovery, 2019, (7): 123--132.
[8]
Wanxiang Che, Ting Liu, Sheng Li. Automatic extraction of entity relationship [J]. Chinese Journal of Information Technology, 2005(02):2--7.
[9]
Socher R, Pennington J, Huang E H, et al. Semi-supervised recursive autoencoders for predicting sentiment distributions[C]//Proceedings of the 2011 conference on empirical methods in natural language processing. 2011: 151--161.
[10]
Yan X, Mou L, Li G, et al. Classifying relations via long short term memory networks along shortest dependency paths[J]. computer science, 2015.
[11]
Li J, Luong M T, Jurafsky D, et al. When are tree structures necessary for deep learning of representations?[J]. arXiv preprint arXiv:1503.00185, 2015.
[12]
Miwa M, Bansal M. End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures[J]. 2016.
[13]
Cui Y, Che W, Liu T, et al. Pre-Training with Whole Word Masking for Chinese BERT[J]. 2019.
[14]
Che W, Li Z, Liu T. LTP: A Chinese Language Technology Platform[C]//COLING 2010, 23rd International Conference on Computational Linguistics, Demonstrations Volume, 23--27 August 2010, Beijing, China. 2010.
[15]
Devlin J, Chang MW, Lee K, et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding[J]. 2018.
[16]
Hochreiter S. The Vanishing Gradient Problem During Learning Recurrent Neural Nets and Problem Solutions[J]. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 1998, 06(2):-.
[17]
Zaremba W, Sutskever I, Vinyals O. Recurrent neural network regularization[J]. arXiv preprint arXiv:1409.2329, 2014.
[18]
Liao S, Grishman R. Using Document Level Cross-Event Inference to Improve Event Extraction.[C]// Acl, Meeting of the Association for Computational Linguistics, July, Uppsala, Sweden. DBLP, 2010.
[19]
Raffel C, Ellis D P W. Feed-forward networks with attention can solve some long-term memory problems[J]. arXiv preprint arXiv:1512.08756, 2015.
Index Terms
- An entity relation extraction algorithm based on BERT(wwm-ext)-BiGRU-Attention
Recommendations
Research on Progress and Inspiration of Entity Relation Extraction in English Open Domain
Machine Learning for Cyber SecurityAbstractIn the era of big data, how to extract unrestricted type of entity relations from open domain text is a challenging topic. In order to further understand related deep issues, this paper summarized the latest progress in the field of English entity ...
Document-Level Iterative Entity and Relation Extraction for Materials Scientific Literature
Advanced Intelligent Computing Technology and ApplicationsAbstractThe existing document-level entity and relation extraction methods mainly concentrate on generic semantics. However, for scientific literature, especially in the materials domain, there are a large number of entities that are iteratively generated ...
Developing Position Structure-Based Framework for Chinese Entity Relation Extraction
Relation extraction is the task of finding semantic relations between two entities in text, and is often cast as a classification problem. In contrast to the significant achievements on English language, research progress in Chinese relation extraction ...
Comments
Information & Contributors
Information
Published In
December 2020
597 pages
ISBN:9781450387828
DOI:10.1145/3444370
Copyright © 2020 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
In-Cooperation
- Sun Yat-Sen University
- CARLETON UNIVERSITY: INSTITUTE FOR INTERDISCIPLINARY STUDIES
- Beijing University of Posts and Telecommunications
- Guangdong University of Technology: Guangdong University of Technology
- Deakin University
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 04 January 2021
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
CIAT 2020
CIAT 2020: 2020 International Conference on Cyberspace Innovation of Advanced Technologies
December 4 - 6, 2020
Guangzhou, China
Acceptance Rates
CIAT 2020 Paper Acceptance Rate 94 of 232 submissions, 41%;
Overall Acceptance Rate 94 of 232 submissions, 41%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 163Total Downloads
- Downloads (Last 12 months)11
- Downloads (Last 6 weeks)1
Reflects downloads up to 08 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in