Skip to main content
SpringerLink
Log in

Improving Cascade Decoding with Syntax-Aware Aggregator and Contrastive Learning for Event Extraction

  • Conference paper
  • First Online:
Chinese Computational Linguistics (CCL 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14232))

Included in the following conference series:

  • 298 Accesses

Abstract

Cascade decoding framework has shown superior performance on event extraction tasks. However, it treats a sentence as a sequence and neglects the potential benefits of the syntactic structure of sentences. In this paper, we improve cascade decoding with a novel module and a self-supervised task. Specifically, we propose a syntax-aware aggregator module to model the syntax of a sentence based on cascade decoding framework such that it captures event dependencies as well as syntactic information. Moreover, we design a type discrimination task to learn better syntactic representations of different event types, which could further boost the performance of event extraction. Experimental results on two widely used event extraction datasets demonstrate that our method could improve the original cascade decoding framework by up to 2.2 percentage points of F1 score and outperform a number of competitive baseline methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://github.com/TimeBurningFish/FewFC.

  2. 2.

    http://ai.baidu.com/broad/download.

  3. 3.

    https://huggingface.co/.

  4. 4.

    https://nlp.stanford.edu/software/lex-parser.shtml.

  5. 5.

    https://www.dgl.ai/.

References

  1. Ahmad, W.U., Peng, N., Chang, K.W.: Gate: graph attention transformer encoder for cross-lingual relation and event extraction. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 12462–12470 (2021)

    Google Scholar 

  2. Chen, Y., Xu, L., Liu, K., Zeng, D., Zhao, J.: Event extraction via dynamic multi-pooling convolutional neural networks. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 167–176 (2015)

    Google Scholar 

  3. Chen, Y., Chen, T., Ebner, S., White, A.S., Van Durme, B.: Reading the manual: event extraction as definition comprehension. In: Proceedings of the Fourth Workshop on Structured Prediction for NLP (2020)

    Google Scholar 

  4. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

  5. Du, X., Cardie, C.: Event extraction by answering (almost) natural questions. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 671–683 (2020)

    Google Scholar 

  6. He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9726–9735 (2020)

    Google Scholar 

  7. Hendrycks, D., Gimpel, K.: Bridging nonlinearities and stochastic regularizers with gaussian error linear units. arXiv preprint arXiv:1606.08415 (2016)

  8. Hinton, G.E., Roweis, S.: Stochastic neighbor embedding. In: Becker, S., Thrun, S., Obermayer, K. (eds.) Advances in Neural Information Processing Systems, vol. 15 (2002)

    Google Scholar 

  9. Huang, P., Zhao, X., Takanobu, R., Tan, Z., Xiao, W.: Joint event extraction with hierarchical policy network. In: Proceedings of the 28th International Conference on Computational Linguistics, pp. 2653–2664 (2020)

    Google Scholar 

  10. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: ICLR (2015)

    Google Scholar 

  11. Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: International Conference on Learning Representations (ICLR) (2017)

    Google Scholar 

  12. Lai, V., Nguyen, M.V., Kaufman, H., Nguyen, T.H.: Event extraction from historical texts: a new dataset for black rebellions. In: Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 2390–2400 (2021)

    Google Scholar 

  13. Lai, V.D., Nguyen, T.N., Nguyen, T.H.: Event detection: gate diversity and syntactic importance scores for graph convolution neural networks. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 5405–5411 (2020)

    Google Scholar 

  14. Lee, D., Tian, Z., Xue, L., Zhang, N.L.: Enhancing content preservation in text style transfer using reverse attention and conditional layer normalization. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 93–102 (2021)

    Google Scholar 

  15. Li, F., et al.: Event extraction as multi-turn question answering. In: Findings of the Association for Computational Linguistics: EMNLP 2020, pp. 829–838 (2020)

    Google Scholar 

  16. Li, L., Liu, Y., Qin, M.: Extracting biomedical events with parallel multi-pooling convolutional neural networks. IEEE/ACM Trans. Comput. Biol. Bioinf. 17(2), 599–607 (2018)

    Google Scholar 

  17. Li, Q., Ji, H., Huang, L.: Joint event extraction via structured prediction with global features. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 73–82 (2013)

    Google Scholar 

  18. Li, Q., et al.: A survey on deep learning event extraction: approaches and applications. IEEE Trans. Neural Netw. Learn. Syst. (2022)

    Google Scholar 

  19. Li, Q., et al.: Reinforcement learning-based dialogue guided event extraction to exploit argument relations. IEEE/ACM Trans. Audio Speech Lang. Process. 30, 520–533 (2021)

    Article  Google Scholar 

  20. Li, X., et al.: DuEE: a large-scale dataset for Chinese event extraction in real-world scenarios. In: CCF International Conference on Natural Language Processing and Chinese Computing, pp. 534–545 (2020)

    Google Scholar 

  21. Liu, J., Chen, Y., Liu, K., Bi, W., Liu, X.: Event extraction as machine reading comprehension. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1641–1651 (2020)

    Google Scholar 

  22. Liu, X., Luo, Z., Huang, H.: Jointly multiple events extraction via attention-based graph information aggregation. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 1247–1256 (2018)

    Google Scholar 

  23. Lu, Y., et al.: Text2event: controllable sequence-to-structure generation for end-to-end event extraction. arXiv preprint arXiv:2106.09232 (2021)

  24. Ma, Y., et al.: Prompt for extraction? PAIE: prompting argument interaction for event argument extraction. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (2022)

    Google Scholar 

  25. Nguyen, T.H., Cho, K., Grishman, R.: Joint event extraction via recurrent neural networks. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 300–309 (2016)

    Google Scholar 

  26. Nguyen, T.H., Grishman, R.: Graph convolutional networks with argument-aware pooling for event detection. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence, pp. 5900–5907 (2018)

    Google Scholar 

  27. Nguyen, T.M., Nguyen, T.H.: One for all: neural joint modeling of entities and events. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 6851–6858 (2019)

    Google Scholar 

  28. Schlichtkrull, M., Kipf, T.N., Bloem, P., van den Berg, R., Titov, I., Welling, M.: Modeling relational data with graph convolutional networks. In: Gangemi, A., et al. (eds.) ESWC 2018. LNCS, vol. 10843, pp. 593–607. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93417-4_38

    Chapter  Google Scholar 

  29. Sha, L., Qian, F., Chang, B., Sui, Z.: Jointly extracting event triggers and arguments by dependency-bridge RNN and tensor-based argument interaction. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence (2018)

    Google Scholar 

  30. Sha, L., Qian, F., Chang, B., Sui, Z.: Jointly extracting event triggers and arguments by dependency-bridge RNN and tensor-based argument interaction. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)

    Google Scholar 

  31. Shen, S., Qi, G., Li, Z., Bi, S., Wang, L.: Hierarchical Chinese legal event extraction via pedal attention mechanism. In: Proceedings of the 28th International Conference on Computational Linguistics, pp. 100–113 (2020)

    Google Scholar 

  32. Sheng, J., et al.: CasEE: a joint learning framework with cascade decoding for overlapping event extraction. In: Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 164–174 (2021)

    Google Scholar 

  33. Wadden, D., Wennberg, U., Luan, Y., Hajishirzi, H.: Entity, relation, and event extraction with contextualized span representations. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 5784–5789 (2019)

    Google Scholar 

  34. Wang, Z., et al.: CLEVE: contrastive pre-training for event extraction. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 6283–6297 (2021)

    Google Scholar 

  35. Wei, Z., Su, J., Wang, Y., Tian, Y., Chang, Y.: A novel cascade binary tagging framework for relational triple extraction. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 1476–1488 (2020)

    Google Scholar 

  36. Xie, J., Sun, H., Zhou, J., Qu, W., Dai, X.: Event detection as graph parsing. In: Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 1630–1640 (2021)

    Google Scholar 

  37. Xu, N., Xie, H., Zhao, D.: A novel joint framework for multiple Chinese events extraction. In: Proceedings of the 19th Chinese National Conference on Computational Linguistics, pp. 950–961 (2020)

    Google Scholar 

  38. Yan, H., Jin, X., Meng, X., Guo, J., Cheng, X.: Event detection with multi-order graph convolution and aggregated attention. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 5766–5770 (2019)

    Google Scholar 

  39. Yang, H., Sui, D., Chen, Y., Liu, K., Zhao, J., Wang, T.: Document-level event extraction via parallel prediction networks. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 6298–6308 (2021)

    Google Scholar 

  40. Yang, S., Feng, D., Qiao, L., Kan, Z., Li, D.: Exploring pre-trained language models for event extraction and generation. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 5284–5294 (2019)

    Google Scholar 

  41. Zhang, W., Ding, X., Liu, T.: Learning target-dependent sentence representations for Chinese event detection. In: Zhang, S., Liu, T.-Y., Li, X., Guo, J., Li, C. (eds.) CCIR 2018. LNCS, vol. 11168, pp. 251–262. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01012-6_20

    Chapter  Google Scholar 

  42. Zhang, Z., Han, X., Liu, Z., Jiang, X., Sun, M., Liu, Q.: ERNIE: enhanced language representation with informative entities. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 1441–1451 (2019)

    Google Scholar 

  43. Zheng, S., Cao, W., Xu, W., Bian, J.: Doc2EDAG: an end-to-end document-level framework for Chinese financial event extraction. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 337–346 (2019)

    Google Scholar 

  44. Zheng, S., Wang, F., Bao, H., Hao, Y., Zhou, P., Xu, B.: Joint extraction of entities and relations based on a novel tagging scheme. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1227–1236 (2017)

    Google Scholar 

  45. Zhou, J., Zhang, Q., Chen, Q., Zhang, Q., He, L., Huang, X.: A multi-format transfer learning model for event argument extraction via variational information bottleneck. In: Proceedings of the 29th International Conference on Computational Linguistics (2022)

    Google Scholar 

  46. Zhou, Y., Chen, Y., Zhao, J., Wu, Y., Xu, J., Li, J.: What the role is vs. what plays the role: semi-supervised event argument extraction via dual question answering. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 14638–14646 (2021)

    Google Scholar 

Download references

Acknowledgements

This work was supported in part by ECNU Research Fund on Cultural Inheritance and Innovation (Grant No. 2022ECNU-WHCCYJ-31) and Shanghai Pujiang Talent Program (Project No. 22PJ1403000). We sincerely thank the anonymous reviewers for their valuable comments and feedback.

Author information

Authors and Affiliations

  1. School of Data Science and Engineering, East China Normal University, Shanghai, China

    Zeyu Sheng, Yuanyuan Liang & Yunshi Lan

Authors
  1. Zeyu Sheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuanyuan Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yunshi Lan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yunshi Lan .

Editor information

Editors and Affiliations

  1. Department of Computer Science and Technology, Tsinghua University, Beijing, China

    Maosong Sun

  2. Harbin Institute of Technology, Harbin, China

    Bing Qin

  3. Fudan University, Shanghai, China

    Xipeng Qiu

  4. School of Computing and Information, Singapore Management University, Singapore, Singapore

    Jiang Jing

  5. Institute of Software, Chinese Academy of Sciences, Beijing, China

    Xianpei Han

  6. Beijing Language and Culture University, Beijing, China

    Gaoqi Rao

  7. Chinese Academy of Sciences, Institute of Automation, Beijing, China

    Yubo Chen

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sheng, Z., Liang, Y., Lan, Y. (2023). Improving Cascade Decoding with Syntax-Aware Aggregator and Contrastive Learning for Event Extraction. In: Sun, M., et al. Chinese Computational Linguistics. CCL 2023. Lecture Notes in Computer Science(), vol 14232. Springer, Singapore. https://doi.org/10.1007/978-981-99-6207-5_11

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-6207-5_11

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-6206-8

  • Online ISBN: 978-981-99-6207-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation