Abstract
Joint information extraction with high quality and low annotation costs plays an important role in many natural language processing (NLP) scenarios. To tackle this challenging problem, we firstly propose a joint machine extraction method based on a hybrid neural network which takes three common NLP tasks—named entity recognition (NER), relation extraction (RE) and event extraction (EE) into consideration. Then, based on the joint model, we propose an efficient active learning algorithm to select the most beneficial sentences to be annotated for further improving the model quality in a batch mode. Experimental results show that the proposed joint framework achieves better performance than state-of-the-art information extraction approaches on standard datasets, and our active algorithm surpasses all baseline methods with just 25% of the original training data and saves more than 70% annotation costs in testing data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ayana, S.S., Zhao, Y., Liu, Z., Sun, M.: Neural headline generation with minimum risk training. CoRR, abs/1604.01904 (2016)
Beluch, W.H., Köhler, J.M.: The power of ensembles for active learning in image classification. In: CVPR 2018, Salt Lake City, UT, USA, 18–22 June 2018, pp. 9368–9377 (2018)
Chakraborty, S., Ye, J.: Active batch selection via convex relaxations with guaranteed solution bounds. IEEE Trans. Pattern Anal. Mach. Intell. 37(10), 1945–1958 (2015)
Chen, Y., Xu, L., Liu, K., Zhao, J.: Event extraction via dynamic multi-pooling convolutional neural networks. In: ACL 2015, 26–31 July 2015, Beijing, China, pp. 167–176 (2015)
Ding, X., Liao, K.: ELG: an event logic graph. CoRR, abs/1907.08015 (2019)
Greenberg, N., Bansal, T., Verga, P., McCallum, A.: Marginal likelihood training of BILSTM-CRF for biomedical named entity recognition from disjoint label sets. In: EMNLP, Brussels, Belgium, 31 October–4 November 2018, pp. 2824–2829 (2018)
Katiyar, A., Cardie, C.: Going out on a limb: joint extraction of entity mentions and relations without dependency trees. In: ACL, Vancouver, Canada, pp. 917–928 (2017)
Li, F., Zhang, M., Fu, G., Ji, D.: A neural joint model for entity and relation extraction from biomedical text. BMC Bioinform. 18(1), 198:1–198:11 (2017)
Li, J., et al.: WCP-RNN: a novel RNN-based approach for bio-NER in Chinese EMRs. J. Supercomput. 76(3), 1450–1467 (2020)
Li, Q., Ji, H.: Incremental joint extraction of entity mentions and relations. In: ACL 2014, vol. 1, pp. 402–412, Baltimore, MD, USA, 22–27 June 2014 (2014)
Liao, S., Grishman, R.: Using document level cross-event inference to improve event extraction. In: Hajic, J., Carberry, S., Clark, S. (eds.) ACL 2010, pp. 789–797 (2010)
Lin, Y., Liu, Z., Sun, M., Liu, Y., Zhu, X.: Learning entity and relation embeddings for knowledge graph completion. In: AAAI, Austin, Texas, USA, 25–30 January 2015, pp. 2181–2187 (2015)
Liu, X., Huang, H.: Jointly multiple events extraction via attention-based graph information aggregation. In: EMNLP, Brussels, Belgium, 31 October 2018, pp. 1247–1256 (2018)
Marcheggiani, D., Titov, I.: Encoding sentences with graph convolutional networks for semantic role labeling. In: EMNLP, Copenhagen, Denmark, 9 September (2017)
Mayer, C., Timofte, R.: Adversarial sampling for active learning. In: WACV 2020, Snowmass Village, CO, USA, 1–5 March 2020, pp. 3060–3068 (2020)
Miwa, M., Bansal, M.: End-to-end relation extraction using LSTMS on sequences and tree structures. In: ACL, 7–12 August 2016, Berlin, Germany (2016)
Nguyen, T.H., Cho, K., Grishman, R.: Joint event extraction via recurrent neural networks. In: NAACL, USA, 12–17 June 2016, pp. 300–309 (2016)
Nguyen, T.H., Grishman, R.: Event detection and domain adaptation with convolutional neural networks. In: ACL, 26–31 July 2015, Beijing, China, pp. 365–371 (2015)
Nguyen, T.H., Grishman, R.: Graph convolutional networks with argument-aware pooling for event detection. In: AAAI, New Orleans, Louisiana, USA, pp. 5900–5907 (2018)
Ohno-Machado, L.: Natural language processing: algorithms and tools to extract computable information from EHRs and from the biomedical literature. JAMIA 20(5), 805 (2013)
Pennington, J., Socher, R., Manning, C.D.: Glove: global vectors for word representation. In: EMNLP, 25–29 October 2014, Doha, Qatar, pp. 1532–1543 (2014)
Sener, O., Savarese, S.: Active learning for convolutional neural networks: a core-set approach. In: ICLR 2018, Vancouver, BC, Canada, 30 April–3 May 2018 (2018)
Shen, Y., Yun, H., Lipton, Z.C., Kronrod, Y., Anandkumar, A.: Deep active learning for named entity recognition. In: ICLR 2018, Vancouver, BC, Canada, 30April–3 May 2018 (2018)
Smith, D.A., Eisner, J.: Minimum risk annealing for training log-linear models. In: ACL 2006, Sydney, Australia, 17–21 July 2006 (2006)
Sun, C., Wu, Y., Lee, K., Wu, K.: Extracting entities and relations with joint minimum risk training. In: EMNLP 2018, Brussels, Belgium, 31 October 2018, pp. 2256–2265 (2018)
Velickovic, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., Bengio, Y.: Graph attention networks. In: ICLR, Vancouver, BC, Canada, 30 April–3 May 2018 (2018)
Yang, S., Li, D.: Exploring pre-trained language models for event extraction and generation. In: ACL, Florence, Italy, 28 July 2019, pp. 5284–5294 (2019)
Yoo, D., Kweon, I.S.: Learning loss for active learning. In: CVPR 2019, Long Beach, CA, USA, 16–20 June 2019, pp. 93–102. IEEE (2019)
Yuan, X., Zhang, T.: Truncated power method for sparse eigenvalue problems. J. Mach. Learn. Res. 14(1), 899–925 (2013)
Zhang, M., Zhang, Y., Fu, G.: End-to-end neural relation extraction with global optimization. In: EMNLP, Copenhagen, Denmark, 9–11 September 2017, pp. 1730–1740 (2017)
Zheng, S., Hao, Y., Lu, D., Bao, H., Xu, J., Xu, B.: Joint entity and relation extraction based on a hybrid neural network. Neurocomputing 257, 59–66 (2017)
Acknowledgement
This work was supported by Key Projects of Military Logistics Research (BHJ14L010), and medical AI research and development project of PLAGH (2019MBD-046).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhuang, Y., Li, G., Xue, W., Zhu, F. (2020). An Active Learning Based Hybrid Neural Network for Joint Information Extraction. In: Huang, Z., Beek, W., Wang, H., Zhou, R., Zhang, Y. (eds) Web Information Systems Engineering – WISE 2020. WISE 2020. Lecture Notes in Computer Science(), vol 12343. Springer, Cham. https://doi.org/10.1007/978-3-030-62008-0_6
Download citation
DOI: https://doi.org/10.1007/978-3-030-62008-0_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-62007-3
Online ISBN: 978-3-030-62008-0
eBook Packages: Computer ScienceComputer Science (R0)