Abstract
As an essential sub-task of frame-semantic parsing, Frame Identification (FI) is a fundamentally important research topic in shallow semantic parsing. However, most existing work is based on sophisticated, hand-crafted features which might not be compatible with FI procedure. Besides that, they usually heavily rely on available natural language processing (NLP) toolkits and various lexical resources. Thus existing methods with hand-crafted features may not achieve satisfactory performance. In this paper, we propose a two-stage attention-based convolutional neural network (TSABCNN) to alleviate this problem and capture the most important context features for FI task. In order to dynamically adjust the weight of each feature, we build two levels of attention over instances at input layer and pooling layer respectively. Furthermore, the proposed model is an end-to-end learning framework which does not need any complicated NLP toolkits and feature engineering, and can be applied to any language. Experiments results on FrameNet and Chinese FrameNet (CFN) show the effectiveness of the proposed approach for the FI task.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Baker, C.F., Fillmore, C.J., Lowe, J.B.: The Berkeley FrameNet project. In: Meeting of the Association for Computational Linguistics and, International Conference on Computational Linguistics, vol. 47, pp. 86–90 (1998)
Fillmore, C.J.: Frame semantics. In: Linguistics in the Morning Calm, pp. 111–138 (1982)
Hermann, K.M., et al.: Teaching machines to read and comprehend. In: NIPS, pp. 1693–1701 (2015)
Das, D., Schneider, N., Chen, D., Smith, N.A.: Probabilistic frame-semantic parsing. In: Human Language Technologies: The 2010 Conference of the North American Chapter of the Association for Computational Linguistics, vol. 40, pp. 948–956 (2010)
Das, D., Chen, D., Schneider, N., Smith, N.A.: Frame-semantic parsing. Comput. Linguist. 40(1), 9–56 (2014)
Carreras, X., Litkowski, K.C., Stevenson, S.: Semantic role labeling: an introduction to the special issue. Comput. Linguist. 34(2), 145–159 (2008)
Kalchbrenner, N., Grefenstette, E., Blunsom, P.: A Convolutional Neural Network for Modelling Sentences. eprint arxiv (2014)
Kim, Y.: Convolutional Neural Networks for Sentence Classification. eprint arxiv (2014)
Zeng, D.J., Liu, K., Lai, S.W., Zhou, G.Y., Zhao, J.: Relation classification via convolutional deep neural network. In: Proceedings of COLING, Dublin, Ireland (2014)
Chen, Y., Xu, L., Liu, K., Zeng, D., Zhao, J.: Event extraction via dynamic multi-pooling convolutional neural networks. In: The Meeting of the Association for Computational Linguistics, Beijing, China, pp. 167–176 (2015)
Dong, L., Wei, F., Zhou, M., Xu, K.: Question answering over freebase with multi-column convolutional neural networks. In: Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing, Beijing, China, pp. 260–269 (2015)
Severyn, A., Moschitti, A.: Learning to rank short text pairs with convolutional deep neural networks. In: The International ACM SIGIR Conference, pp. 373–382. ACM (2015)
Yin, W., Schütze, H.: MultiGranCNN: an architecture for general matching of text chunks on multiple levels of granularity. In: Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing, Beijing, China, pp. 63–73 (2015)
Gildea, D., Jurafsky, D.: Automatic labeling of semantic roles. Comput. Linguist. 28(28), 245–288 (2002)
Carreras, X.: Introduction to the CoNLL-2004 shared task: semantic role labeling. In: Proceedings of 8th Conference on Natural Language Learning (CoNLL), Boston, MA, vol. 47, pp. 5–9 (2004)
Carreras, X., Mà rquez, L.: Introduction to the CoNLL-2005 shared task: semantic role labeling. In: Proceedings of the 9th Conference on Computational Natural Language Learning (CoNLL), ANN Arbor, MI, pp. 152–164 (2005)
Fleischman, M., Kwon, N., Hovy, E.: Maximum entropy models for FrameNet classification. In: Conference on Empirical Methods in Natural Language Processing, Sapporo, pp. 49–56 (2003)
Erk, K.: Frame assignment as word sense disambiguation. In: proceedings of the 6th International Workshop on Computational Semantic (IWCS-6) (2005)
Johansson, R., Nugues, P.: LTH: semantic structure extraction using nonprojective dependency trees. In: Proceedings of SemEval, vol. 13, no. 4, pp. 227–230 (2007)
Bejan, C.A., Hathaway, C.: UTD-SRL: a pipeline architecture for extracting frame semantic structures. In: International Workshop on Semantic Evaluations, vol. 8764, pp. 460–463 (2007)
Hermann, K.M., Das, D., Weston, J., Ganchev, K.: Semantic frame identification with distributed word representations. In: Meeting of the Association for Computational Linguistics, vol. 1, pp. 1448–1458 (2014)
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. arXiv: 1409.0473 (2014)
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12(1), 2493–2537 (2011)
Yin, W.P., Schütze, H., Xiang, B., Zhou, B.W.: ABCNN: attention-based convolutional neural network for modeling sentence pairs. Action Editor: Brian Roark, vol. 4, pp. 259–272 (2016)
Collobert R., Weston J.: A unified architecture for natural language processing: deep neural networks with multitask learning. International Conference on Machine Learning, pp.160–167. ACM (2008)
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R. R.: Improving neural networks by preventing co-adaptation of feature detectors. Comput. Sci. vol. 3(4), 212–223 (2012)
Zhang, Y., Wallace, B.: A sensitivity analysis of (and practitioners’ guide to) convolutional neural networks for sentence classification. In: Proceedings of the 8th International Joint Conference on Natural Language Processing, Taipei, Taiwan, pp. 253–263 (2017)
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems, vol. 26, pp. 3111–3119 (2013)
Zhou, P., et al.: Attention-based bidirectional long short-term memory networks for relation classification. In: Meeting of the Association for Computational Linguistics, pp. 207–212 (2016)
Li, R., Liu, H.J., Li, S.H.: Chinese frame identification using T-CRF model. In: International Conference on Computational Linguistics, COLING 2010, pp. 674–682 (2010)
Zhao, H.Y., Li, R., Zhang, S., Zhang, L.W.: Chinese frame identification with deep neural network. J. Chin. Inf. Process. 30(6), 75–83 (2016)
Acknowledgements
This work is supported by National Natural Science Foundation of China (No. 61772324, No. 61673248), and Shanxi Province Postgraduate Joint Training Base Talent Training Project (No. 2018JD01, No. 2018JD02).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhao, H., Li, R., Duan, F., Wu, Z., Guo, S. (2018). TSABCNN: Two-Stage Attention-Based Convolutional Neural Network for Frame Identification. In: Sun, M., Liu, T., Wang, X., Liu, Z., Liu, Y. (eds) Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data. CCL NLP-NABD 2018 2018. Lecture Notes in Computer Science(), vol 11221. Springer, Cham. https://doi.org/10.1007/978-3-030-01716-3_24
Download citation
DOI: https://doi.org/10.1007/978-3-030-01716-3_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-01715-6
Online ISBN: 978-3-030-01716-3
eBook Packages: Computer ScienceComputer Science (R0)