Abstract
Relation classification is an essential task in natural language processing (NLP) in order to extract structured data from sentences. In this paper, we propose a novel model Att-ComNN combining convolutional neural network (CNN) and bidirectional recurrent neural network (RNN) for relation classification. By combining RNN and CNN, we obtain more accurate context representations of words, which benefits classifying relations. Besides, with both shortest dependency path (SDP) attention and pooling attention added, this model captures the most informative context representation for better classification without using other handcrafted features. The results of experiments show that our model improves the relation classification performance on the SemEval-2010 Task 8 and outperforms most of previous state-of-the-art methods, including those depending on much richer forms of handcrafted features and prior knowledge.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The dimension we apply softmax function is different from Wang et al. [12].
References
Bunescu, R.C., Mooney, R.J.: A shortest path dependency kernel for relation extraction. In: Proceedings of the Conference Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, HLT/EMNLP 2005, Vancouver, British Columbia, Canada, 6–8 October 2005, pp. 724–731 (2005)
Cai, R., Zhang, X., Wang, H.: Bidirectional recurrent convolutional neural network for relation classification. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, Volume 1: Long Papers, Berlin, Germany, 7–12 August 2016 (2016)
Chen, J., Ji, D., Tan, C.L., Niu, Z.: Unsupervised feature selection for relation extraction. In: Second International Joint Conference on Natural Language Processing - IJCNLP 2005 - Companion Volume to the Proceedings of Conference including Posters/Demos and tutorial abstracts, Jeju Island, Republic of Korea, 11–13 October 2005 (2005)
Cho, K., van Merrienboer, B., Gülçehre, Ç., Bougares, F., Schwenk, H., Bengio, Y.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. CoRR abs/1406.1078 (2014)
Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, AISTATS 2010, Chia Laguna Resort, Sardinia, Italy, 13–15 May 2010, pp. 249–256 (2010)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. CoRR abs/1301.3781 (2013)
Rink, B., Harabagiu, S.M.: UTD: classifying semantic relations by combining lexical and semantic resources. In: Proceedings of the 5th International Workshop on Semantic Evaluation, SemEval@ACL 2010, Uppsala University, Uppsala, Sweden, 15–16 July 2010, pp. 256–259 (2010)
Rotsztejn, J., Hollenstein, N., Zhang, C.: Eth-ds3lab at semeval-2018 task 7: effectively combining recurrent and convolutional neural networks for relation classification and extraction. CoRR abs/1804.02042 (2018)
dos Santos, C.N., Xiang, B., Zhou, B.: Classifying relations by ranking with convolutional neural networks. CoRR abs/1504.06580 (2015)
Socher, R., Huval, B., Manning, C.D., Ng, A.Y.: Semantic compositionality through recursive matrix-vector spaces. In: Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, EMNLP-CoNLL 2012, Jeju Island, Korea, 12–14 July 2012, pp. 1201–1211 (2012)
Socher, R., Pennington, J., Huang, E.H., Ng, A.Y., Manning, C.D.: Semi-supervised recursive autoencoders for predicting sentiment distributions. In: Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, EMNLP 2011, A meeting of SIGDAT, a Special Interest Group of the ACL, John McIntyre Conference Centre, Edinburgh, UK, 27–31 July 2011, pp. 151–161 (2011)
Wang, L., Cao, Z., de Melo, G., Liu, Z.: Relation classification via multi-level attention CNNs. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, Volume 1: Long Papers, Berlin, Germany, 7–12 August 2016 (2016)
Xu, K., Feng, Y., Huang, S., Zhao, D.: Semantic relation classification via convolutional neural networks with simple negative sampling. CoRR abs/1506.07650 (2015)
Xu, Y., Mou, L., Li, G., Chen, Y., Peng, H., Jin, Z.: Classifying relations via long short term memory networks along shortest dependency paths. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, 17–21 September 2015, pp. 1785–1794 (2015)
Yu, J., Jiang, J.: Pairwise relation classification with mirror instances and a combined convolutional neural network. In: COLING 2016, 26th International Conference on Computational Linguistics, Proceedings of the Conference: Technical Papers, Osaka, Japan, 11–16 December 2016, pp. 2366–2377 (2016)
Zeng, D., Liu, K., Lai, S., Zhou, G., Zhao, J.: Relation classification via convolutional deep neural network. In: COLING 2014, 25th International Conference on Computational Linguistics, Proceedings of the Conference: Technical Papers, Dublin, Ireland, 23–29 August 2014, pp. 2335–2344 (2014)
Zheng, S., Wang, F., Bao, H., Hao, Y., Zhou, P., Xu, B.: Joint extraction of entities and relations based on a novel tagging scheme. CoRR abs/1706.05075 (2017)
Zhou, P., Shi, W., Tian, J., Qi, Z., Li, B., Hao, H., Xu, B.: Attention-based bidirectional long short-term memory networks for relation classification. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, Volume 2: Short Papers, Berlin, Germany, 7–12 August 2016
Acknowledgements
This work is supported by National Key R&D Program of China (No. 2017YFB1400200).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Guo, X., Zhang, H., Liu, R., Ding, X., Tian, R., Wang, B. (2018). Attention-Based Combination of CNN and RNN for Relation Classification. In: Cheng, L., Leung, A., Ozawa, S. (eds) Neural Information Processing. ICONIP 2018. Lecture Notes in Computer Science(), vol 11304. Springer, Cham. https://doi.org/10.1007/978-3-030-04212-7_21
Download citation
DOI: https://doi.org/10.1007/978-3-030-04212-7_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-04211-0
Online ISBN: 978-3-030-04212-7
eBook Packages: Computer ScienceComputer Science (R0)