Abstract
Entity and relationship extraction is an important step in building a knowledge base, which is the basis for many artificial intelligence products to be used in life, such as Amazon Echo and Intelligent Search. We propose a new artificial neural network model to identify entities and their relationships without any handcrafted features. The neural network model mainly includes the CNN module for extracting text features and relationship classifications, and a bidirectional LSTM module for obtaining context information of the entity. The context information and entity tags between the entities obtained in the entity identification process are further passed to the CNN module of the relationship classification to improve the effectiveness of the relationship classification and achieve the purpose of joint processing. We conducted experiments on the public datasets CoNLL04 (Conference on Computational Natural Language Learning), ACE04 and ACE05 (Automatic Content Extraction program) to verify the effectiveness of our approach. The method we proposed achieves the state-of-the-art results on entity and relation extraction task.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Chiu J, Nichols E (2015) Named entity recognition with bidirectional LSTM-SNNS. Assoc Comput Linguist Trans. https://doi.org/10.1162/tacl_a_00104
Chowdary D, Vanukuri L, Sistla V, Kishore K (2019) Sentiment analysis and text categorization of cancer medical records with LSTM. J Ambient Intell Hum Comput. https://doi.org/10.1007/s12652-019-01399-8
Collobert R, Jason W, Leon B, Michael KK, Kuksa P (2011) Natural language processing (almost) from scratch. Comput Res Repos CORR. http://www.jmlr.org/papers/volume12/collobert11a/collobert11a.pdf
Dahl G, Sainath T , Geoffrey H (2013) Improving deep neural networks for LVCSR using rectified linear units and dropout. 26:8609–8613. https://doi.org/10.1109/ICASSP.2013.6639346
Duan K, Keerthi S, Chu W, Shevade S, Poo A-N(2003) Multi-category classification by soft-max combination of binary classifiers. 125–134. https://doi.org/10.1007/3-540-44938-8_13
Duchi J, Hazan E, Singer Y (2011) Adaptive subgradient methods for online learning and stochastic optimization. J Mach Learn Res 12:2121–2159. https://doi.org/10.1109/TNN.2011.2146788
Ebrahimi J, Dou D (2015) Chain based RNN for relation classification. 1244–1249. https://doi.org/10.3115/v1/N15-1133
Graves A (2012) Supervised sequence labelling with recurrent neural networks. 385. https://doi.org/10.1007/978-3-642-24797-2
Huang Z, Xu W, Yu K (2015) Bidirectional LSTM-CRF models for sequence tagging. arxiv:1508.01991
Kala M, Nandhini KU (2019) Context-category specific sequence aware point-of-interest recommender system with multi-gated recurrent unit. J Ambient Intell Hum Comput. https://doi.org/10.1007/s12652-019-01583-w. ISSN 1868-5145
Kalchbrenner N, Grefenstette E, Blunsom P (2014) A convolutional neural network for modelling sentences. Eprint arxiv. https://doi.org/10.3115/v1/P14-1062
Kambhatla N (2004) Combining lexical, syntactic and semantic features with maximum entropy models for extracting relations. Proc ACL 10(3115/1219044):1219066
Kim N, Choi Y, Lee H, Choi J, Kim S, Kim J-A, Cho Y, Lee J-H (2017) Detection of document modification based on deep neural networks. J Ambient Intell Hum Comput. https://doi.org/10.1007/s12652-017-0617-y
Lample G, Ballesteros M, Subramanian S, Kawakami K, Dyer C (2016) Neural architectures for named entity recognition. 260–270. https://doi.org/10.18653/v1/N16-1030
LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86 (11):2278–2324. https://doi.org/10.1109/5.726791. ISSN 0018-9219
Li Q, Ji H (2014) Incremental joint extraction of entity mentions and relations. In: The association for computer linguistics. http://dblp.unitrier.de/db/conf/acl/acl2014-1.html
Liu J, Yang Y, Lv S, Wang J, Chen H (2019) Attention-based BIGRU-CNN for Chinese question classification. J Ambient Intell Hum Comput. https://doi.org/10.1007/s12652-019-01344-9
Ma X, Hovy E (2016) End-to-end sequence labeling via bi-directional LSTM-CNNS-CRF. 1064–1074. https://doi.org/10.18653/v1/P16-1101
Miwa M, Bansal M (2016) End-to-end relation extraction using LSTMS on sequences and tree structures. 1105–1116. https://doi.org/10.18653/v1/P16-1105
Miwa M, Sasaki Y (2014) Modeling joint entity and relation extraction with table representation. In: EMNLP. ACL. http://dblp.unitrier.de/db/conf/emnlp/emnlp2014.html
Nguyen N-V, Nguyen T-L, Nguyen V, Mai T, Nguyen T-T, Ha Q (2019) Improving named entity recognition in vietnamese texts by a character-level deep lifelong learning model. Vietnam J Comput Sci. https://doi.org/10.1142/S219688881950026X
Nguyen T-VT, Alessandro M, Giuseppe R (2009) Convolution kernels on constituent, dependency and sequential structures for relation extraction, pp 1378–1387. http://dblp.unitrier.de/db/conf/emnlp/emnlp2009.html#NguyenMR09
Qin P, Weiran X, Guo J (2016) An empirical convolutional neural network approach for semantic relation classification. Neurocomputing 190. https://doi.org/10.1016/j.neucom.2015.12.091
Qin P, Xu W, Wang W (2018) Dsgan: generative adversarial training for distant supervision relation extraction. https://doi.org/10.18653/v1/P18-1046
Ratinov L, Roth D (2009) Conll09 design challenges and misconceptions in named entity recognition. https://doi.org/10.3115/1596374.1596399
Rink B, Harabagiu S (2010) UTD: classifying semantic relations by combining lexical and semantic resources. In: Proceedings of the 5th international workshop on semantic evaluation, pp 256–259. http://dblp.unitrier.de/db/conf/semeval/semeval2010.html#RinkH10
Shen Y, Yun H, Lipton Z, Kronrod Y, Anandkumar A (2018) Deep active learning for named entity recognition. https://doi.org/10.18653/v1/W17-2630
Singh G, Bhatia P (2019) Relation extraction using explicit context conditioning. 1442–1447. https://doi.org/10.18653/v1/N19-1147
Singh S, Riedel S, Martin B, Zheng J, Mccallum A (2013) Joint inference of entities, relations, and coreference. 1–6. https://doi.org/10.1145/2509558.2509559
Sun L, Han X (2014) A feature-enriched tree kernel for relation extraction. In: Proceedings of the 52nd annual meeting of the Association for Computational Linguistics (volume 2: short papers), pp 61–67. Association for Computational Linguistics. http://www.aclweb.org/anthology/P14-2011
Tieleman HGT (2012) Lecture 6.5-rmsprop: divide the gradient by a running average of its recent magnitude
Yan X, Mou L, Li G, Chen Y, Peng H, Jin Z (2015) Classifying relations via long short term memory networks along shortest dependency path. https://doi.org/10.18653/v1/D15-1206
Yih W, Roth D (2007) Global inference for entity and relation identification via a linear programming formulation. In: An introduction to statistical relational learning. MIT Press. https://ieeexplore.ieee.org/document/6278198
Yu M, Gormley MR, Dredze M (2014) Factor-based compositional embedding models. In: NIPS workshop on learning semantics
Zeng D, Liu K, Lai S, Zhou G, Zhao J (2014) Relation classification via convolutional deep neural network. In: The 25th international conference on computational linguistics: technical papers, pp 2335–2344. http://dblp.uni-trier.de/db/conf/coling/coling2014.html#ZengLLZZ14
Zhang D, Wang D (2015) Relation classification via recurrent neural network. arxiv:1508.01006
Zheng S, Hao Y, Dongyuan L, Bao H, Jiaming X, Hao H, Bo X (2017) Joint entity and relation extraction based on a hybrid neural network. Neurocomputing 257. https://doi.org/10.1016/j.neucom.2016.12.075
Zhou L, Gu X (2019) Deep supervised visual saliency model addressing low-level features. J Ambient Intell Human Comput. https://doi.org/10.1007/s12652-019-01441-9
Zhou GS, Jian JZ, Zhang M (2005) Exploring various knowledge in relation extraction. https://doi.org/10.3115/1219840.1219893
Zhu X, Sobhani P, Guo H (2015) Long short-term memory over tree structures. arxiv:1503.04881
Funding
This study was funded by National Natural Science Foundation of China (Grant no. 61371156).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zhang, Z., Zhan, S., Zhang, H. et al. Joint model of entity recognition and relation extraction based on artificial neural network. J Ambient Intell Human Comput 13, 3503–3511 (2022). https://doi.org/10.1007/s12652-020-01949-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12652-020-01949-5