Abstract
Recently, attention mechanism has been transferred to relation classification task. Since relation classification is a sequence-to-label task, the challenge is how to generate the deciding factor to calculate attention weights. The previous solution randomly initializes a global deciding factor, which is easy to suffer from over-fitting. To solve the problem, we propose instance-adaptive attention mechanism, which generates a specially designed deciding factor for each sentence. The experimental result on SemEval-2010 Task 8 dataset shows that our method can outperform most state-of-the-art systems without external linguistic features.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
References
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. Comput. Sci. (2014)
Luo, Z., Osborne, M., Petrovi, S., et al.: Improving Twitter retrieval by exploiting structural information. In: Twenty-Sixth AAAI Conference on Artificial Intelligence (2012)
Bienvenu, M., Bourgaux, C., Goasdou, F.: Explaining inconsistency-tolerant query answering over description logic knowledge bases. In: AAAI Conference on Artificial Intelligence (2016)
Yao, X., Durme, B.V.: Information extraction over structured data: question answering with freebase. In: Meeting of Association for Computational Linguistics, pp. 956–966 (2014)
Santos, C.N.D., Xiang, B., Zhou, B.: Classifying relations by ranking with convolutional neural networks. Comput. Sci. (2015)
Yan, X., Mou, L., Li, G., et al.: Classifying relations via long short term memory networks along shortest dependency path. In: Conference on Empirical Methods in Natural Language Processing. 56–61, arXiv (2015)
Xu, Y., Jia, R., Mou, L., et al.: Improved relation classification by deep recurrent neural networks with data augmentation. In: COLING, arXiv (2016)
Xu, K., Feng, Y., Huang, S., et al.: Semantic relation classification via convolutional neural networks with simple negative sampling. Comput. Sci. 71(7), 941–949 (2015)
Mikolov, T., Sutskever, I., Chen, K., et al.: Distributed representations of words and phrases and their compositionality. Adv. Neural Inf. Process. Syst. 26, 3111–3119 (2013)
Zeng, D., Liu, K., Lai, S., et al.: Relation classification via convolutional deep neural network (2014)
Zhang, D., Wang, D.: Relation classification via recurrent neural network. Comput. Sci. (2015)
Zhang, S., Zheng, D., Hu, X., et al.: Bidirectional long short-term memory networks for relation classification (2015)
Zhou, P., Shi, W., Tian, J., et al.: Attention-based bidirectional long short-term memory networks for relation classification. In: Meeting of Association for Computational Linguistics, pp. 207–212 (2016)
Cho, K., Merrienboer, B.V., Gulcehre, C., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. Comput. Sci. (2014)
Rink, B., Harabagiu, S.: Classifying semantic relations by combining lexical and semantic resources. In: International Workshop on Semantic Evaluation, pp. 256–259. Association for Computational Linguistics (2010)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Hinton, G.E., Srivastava, N., Krizhevsky, A., et al.: Improving neural networks by preventing co-adaptation of feature detectors. Comput. Sci. 3(4), 212–223 (2012)
Acknowledgments
This work was supported by 111 Project of China under Grant No. B08004, National Natural Science Foundation of China (61273217, 61300080, 61671078), the Ph.D. Programs Foundation of Ministry of Education of China (20130005110004).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Lu, Y., Zhang, C., Xu, W. (2017). Instance-Adaptive Attention Mechanism for Relation Classification. In: Lintas, A., Rovetta, S., Verschure, P., Villa, A. (eds) Artificial Neural Networks and Machine Learning – ICANN 2017. ICANN 2017. Lecture Notes in Computer Science(), vol 10614. Springer, Cham. https://doi.org/10.1007/978-3-319-68612-7_37
Download citation
DOI: https://doi.org/10.1007/978-3-319-68612-7_37
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-68611-0
Online ISBN: 978-3-319-68612-7
eBook Packages: Computer ScienceComputer Science (R0)