Skip to main content
Springer Nature Link
Log in

Data Augmentation Based on Pre-trained Language Model for Event Detection

  • Conference paper
  • First Online:
CCKS 2021 - Evaluation Track (CCKS 2021)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1553))

Included in the following conference series:

  • 524 Accesses

Abstract

Event detection (ED) is an important task which needs to identify the event triggers in the sentence and classify the event types. For the general fine-grained event detection task, we propose an event detection scheme based on pre-trained model, combined with data augmentation and pseudo labelling method, which improves the event detection ability of the model. At the same time, we use voting for model ensemble, so as to effectively utilize the advantages of multiple models. Our model achieves F1 score of 69.86% on the test set of CCKS2021 general fine-grained event detection task and ranks the third place in the competition.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://www.biendata.xyz/competition/ccks_2021_maven/.

References

  1. Ahn, D.: The stages of event extraction. In: Proceedings of the Workshop on Annotating and Reasoning about Time and Events, pp. 1–8 (2006)

    Google Scholar 

  2. Chen, Y., Xu, L., Liu, K., Zeng, D., Zhao, J.: Event extraction via dynamic multi-pooling convolutional neural networks. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics, pp. 167–176. The Association for Computer Linguistics (2015). https://doi.org/10.3115/v1/p15-1017

  3. Chen, Y., Yang, H., Liu, K., Zhao, J., Jia, Y.: Collective event detection via a hierarchical and bias tagging networks with gated multi-level attention mechanisms. In: Riloff, E., Chiang, D., Hockenmaier, J., Tsujii, J. (eds.) Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 1267–1276. Association for Computational Linguistics (2018). https://doi.org/10.18653/v1/d18-1158

  4. Clark, K., Luong, M., Le, Q.V., Manning, C.D.: ELECTRA: pre-training text encoders as discriminators rather than generators. In: 8th International Conference on Learning Representations. OpenReview.net (2020)

    Google Scholar 

  5. Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Burstein, J., Doran, C., Solorio, T. (eds.) Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 4171–4186. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/n19-1423

  6. Ding, N., Li, Z., Liu, Z., Zheng, H., Lin, Z.: Event detection with trigger-aware lattice neural network. In: Inui, K., Jiang, J., Ng, V., Wan, X. (eds.) Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pp. 347–356. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/D19-1033

  7. Dong, L., et al.: Unified language model pre-training for natural language understanding and generation. In: Wallach, H.M., Larochelle, H., Beygelzimer, A., d’Alché-Buc, F., Fox, E.B., Garnett, R. (eds.) Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, pp. 13042–13054 (2019)

    Google Scholar 

  8. Dong, Z., Dong, Q., Hao, C.: Hownet and its computation of meaning. In: COLING 2010, 23rd International Conference on Computational Linguistics, pp. 53–56. Demonstrations Volume (2010)

    Google Scholar 

  9. Feng, S.Y., et al.: A survey of data augmentation approaches for NLP. In: Zong, C., Xia, F., Li, W., Navigli, R. (eds.) Findings of the Association for Computational Linguistics. Findings of ACL, vol. ACL/IJCNLP 2021, pp. 968–988. Association for Computational Linguistics (2021). https://doi.org/10.18653/v1/2021.findings-acl.84

  10. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997). https://doi.org/10.1162/neco.1997.9.8.1735

    Article  Google Scholar 

  11. Ji, H., Grishman, R.: Refining event extraction through cross-document inference. In: McKeown, K.R., Moore, J.D., Teufel, S., Allan, J., Furui, S. (eds.) ACL 2008, Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics, pp. 254–262. The Association for Computer Linguistics (2008)

    Google Scholar 

  12. Joshi, M., Chen, D., Liu, Y., Weld, D.S., Zettlemoyer, L., Levy, O.: Spanbert: Improving pre-training by representing and predicting spans. Trans. Assoc. Comput. Linguistics 8, 64–77 (2020)

    Article  Google Scholar 

  13. Liao, S., Grishman, R.: Using document level cross-event inference to improve event extraction. In: Hajic, J., Carberry, S., Clark, S. (eds.) ACL 2010, Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pp. 789–797. The Association for Computer Linguistics (2010)

    Google Scholar 

  14. Liu, Y., et al.: Roberta: a robustly optimized BERT pretraining approach. CoRR abs/1907.11692 (2019)

    Google Scholar 

  15. Miller, G.A.: Wordnet: a lexical database for English. Commun. ACM 38(11), 39–41 (1995). https://doi.org/10.1145/219717.219748

    Article  Google Scholar 

  16. Walker, C., Strassel, S., Medero, J., Maeda, K.: Ace 2005 multilingual training corpus. Prog. Theor. Phys. Suppl. 110(110), 261–276 (2006)

    Google Scholar 

  17. Wang, X., et al.: MAVEN: a massive general domain event detection dataset. In: Webber, B., Cohn, T., He, Y., Liu, Y. (eds.) Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pp. 1652–1671. Association for Computational Linguistics (2020). https://doi.org/10.18653/v1/2020.emnlp-main.129

  18. Yang, Z., Dai, Z., Yang, Y., Carbonell, J.G., Salakhutdinov, R., Le, Q.V.: Xlnet: generalized autoregressive pretraining for language understanding. In: Wallach, H.M., et al. (eds.) Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, pp. 5754–5764 (2019)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science, Wuhan University, Wuhan, 430072, China

    Meng Zhang, Zhiwen Xie & Jin Liu

Authors
  1. Meng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhiwen Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jin Liu .

Editor information

Editors and Affiliations

  1. Harbin Institute of Technology, Harbin, China

    Bing Qin

  2. Tongji University, Shanghai, China

    Haofen Wang

  3. Harbin Institute of Technology, Harbin, China

    Ming Liu

  4. Tsinghua University, Beijing, China

    Jiangtao Zhang

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhang, M., Xie, Z., Liu, J. (2022). Data Augmentation Based on Pre-trained Language Model for Event Detection. In: Qin, B., Wang, H., Liu, M., Zhang, J. (eds) CCKS 2021 - Evaluation Track. CCKS 2021. Communications in Computer and Information Science, vol 1553. Springer, Singapore. https://doi.org/10.1007/978-981-19-0713-5_8

Download citation

  • DOI: https://doi.org/10.1007/978-981-19-0713-5_8

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-19-0712-8

  • Online ISBN: 978-981-19-0713-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics