Abstract
Deep Neural Network has been applied to many Natural Language Processing tasks. Instead of building hand-craft features, DNN builds features by automatic learning, fitting different domains well. In this paper, we propose a novel convolution network, incorporating lexical features, applied to Relation Extraction. Since many current deep neural networks use word embedding by word table, which, however, neglects semantic meaning among words, we import a new coding method, which coding input words by synonym dictionary to integrate semantic knowledge into the neural network. We compared our Convolution Neural Network (CNN) on relation extraction with the state-of-art tree kernel approach, including Typed Dependency Path Kernel and Shortest Dependency Path Kernel and Context-Sensitive tree kernel, resulting in a 9% improvement competitive performance on ACE2005 data set. Also, we compared the synonym coding with the one-hot coding, and our approach got 1.6% improvement. Moreover, we also tried other coding method, such as hypernym coding, and give some discussion according the result.
This research was supported by Research Fund for the Doctoral Program for Higher Education of China (New teacher Fund), Contract No. 20101102120016.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Grishman, R.: Information Extraction: Capabilities and Challenges. Lecture Notes of 2012 International Winter School in Language and Speech Technologies, Rovirai Virgili University (2012)
Kambhatla, N.: Combining lexical, syntactic, and semantic features with maximum entropy models for information extraction. In: The Companion Volume to the Proceedings of 42st Annual Meeting of the Association for Computational Linguistics, vol. 1, pp. 178–181 (2004)
Zelenko, D., Aone, C., Richardella, A.: Kernel methods for relation extraction. Journal Machine Learning Research 3, 1083–1106 (2003)
Culotta, A., Sorensen, J.: Dependency tree kernels for relation extraction. In: Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics, pp. 423–430 (2004)
Collobert, R., Weston, J.: Fast Semantic Extraction Using a Novel Neural Network Architecture. In: Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pp. 560–567 (2008)
Schwenk, H., Rousseau, A., Attik, M.: Large, pruned or continuous space language models on a gpu-forstatistical machine translation. In: Workshop on the Future of Language Modeling for HLT (2012)
Collobert, R., Weston, J.: A unified architecture for natural language processing: Deep neural networks with multitask learning. In: ICML 2008 (2008)
Grolot, X., Bordes, A., Bengio, Y.: Domain adaptation for large-scale sentiment classification: A deep learning approach.In ICML 2011 (2011)
Bourlard, H., Kamp, Y.: Auto-association by multilayer perceptrons and singular value decomposition. Biological Cybernetics 59, 291–294 (1988)
LeCun, Y., Bengio, Y.: Convolutional Networks for Images, Speech, and Time-Series. The Handbook of Brain Theory and Neural Networks. MIT Press (1995)
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C.: A Neural Probabilistic Language Model. Journal of Machine Learning Research 3, 1137–1155 (2003)
Walker, C., Strassel, S., Medero, J., Maeda, K.: ACE 2005 Multilingual Training Corpus. Linguistic Data Consortium, Philadelphia (2006)
Klein, D., Manning, C.: Accurate Unlexicalized Parsing. In: Proceedings of the 41st Meeting of the Association for Computational Linguistics, pp. 423–430 (2003)
Fellbaum, C.: WordNet: An Electronic Lexical Database. MIT Press, Cambridge (1998)
Toutanova, K., Klein, D., Manning, C., Singer, Y.: Feature-Rich Part-of-Speech Tagging with a Cyclic Dependency Network. In: Proceedings of HLT-NAACL, pp. 252–259 (2003)
Bunescu, R., Mooney, R.: Proceedings of the 19th Conference on Neural Information Processing Systems (NIPS), Vancouver, BC (2005)
Reichartz, F., Korte, H., Paass, G.: Semantic Relation Extraction with Kernels Over Typed Dependency Trees. In: KDD 2010, Washington, DC (2010)
Zhou, G., Zhang, M., Ji, D., Zhu, Q.: Tree Kernel-based Relation Extractionwith Context-Sensitive Structured Parse Tree Information. In: EMNLP 2010, Prague, pp. 728–736 (2007)
Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology 2, 27:1–27:27 (2011)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Liu, C., Sun, W., Chao, W., Che, W. (2013). Convolution Neural Network for Relation Extraction. In: Motoda, H., Wu, Z., Cao, L., Zaiane, O., Yao, M., Wang, W. (eds) Advanced Data Mining and Applications. ADMA 2013. Lecture Notes in Computer Science(), vol 8347. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-53917-6_21
Download citation
DOI: https://doi.org/10.1007/978-3-642-53917-6_21
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-53916-9
Online ISBN: 978-3-642-53917-6
eBook Packages: Computer ScienceComputer Science (R0)