Skip to main content
SpringerLink
Log in

Simple but Effective: Keyword-Based Metric Learning for Event Sentence Coreference Identification

  • Conference paper
  • First Online:
Advanced Intelligent Computing Technology and Applications (ICIC 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14089))

Included in the following conference series:

  • 947 Accesses

Abstract

Event sentence coreference identification (ESCI) is a fundamental task of news event detection and tracking which aims to group sentences according to events they refer to. Most recent efforts address this task by means of identifying coreferential event sentence pairs. Currently, frameworks based on pre-trained language models like Sentence-BERT (SBERT) are widely used for sentence pair tasks. However, SBERT lacks keyword awareness, while the local features of sentences can demonstrate a strong correlation with the event topic. In addition, the strategy of encoding the whole sentence is less flexible and more time-consuming. After reconsidering the significance of keywords in ESCI task, we propose KeyML, a simple keyword-based metric learning approach which leverages both lexical and semantic features of keywords to capture subject patterns of events. Specifically, a Siamese network is adapted to optimize distance metrics of keyword embeddings, resulting in more separable similarity of event sentence pairs. Then, KeyML considers keywords of data with different granularity and exploits three training strategies, along with their corresponding sampling methods, to investigate co-occurrence relationships. Experimental results show that KeyML outperforms SBERT and SimCSE on three datasets and demonstrate the effectiveness and rationality of our method.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Kim, H.G., Lee, S., Kyeong, S.: Discovering hot topics using twitter streaming data social topic detection and geographic clustering. In: 2013 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2013), pp. 1215–1220 (2013). https://doi.org/10.1109/ASONAM.2013.6785858

  2. Wadden, D., Wennberg, U., Luan, Y., Hajishirzi, H.: Entity, relation, and event extraction with contextualized span representations. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China, pp. 5784–5789. Association for Computational Linguistics, November 2019

    Google Scholar 

  3. Blanco, E., Castell, N., Moldovan, D.: Causal relation extraction. In: Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC’08). European Language Resources Association (ELRA), Marrakech, Morocco, May 2008. http://www.lrec-conf.org/proceedings/lrec2008/pdf/87_paper.pdf

  4. Bouras, C., Tsogkas, V.: Improving news articles recommendations via user clustering. Int. J. Mach. Learn. Cybern. 8(1), 223–237 (2014). https://doi.org/10.1007/s13042-014-0316-3

    Article  Google Scholar 

  5. Ramos, J.E.: Using tf-idf to determine word relevance in document queries (2003)

    Google Scholar 

  6. Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent dirichlet allocation. J. Mach. Learn. Res. 3(null), 993–1022 (2003)

    Google Scholar 

  7. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9, 1735–1780 (1997)

    Article  Google Scholar 

  8. Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1724–1734, Doha, Qatar. Association for Computational Linguistics, October 2014

    Google Scholar 

  9. Bengio, Y., Boulanger-Lewandowski, N., Pascanu, R.: Advances in optimizing recurrent networks. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 8624–8628 (2012)

    Google Scholar 

  10. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, Minnesota, pp. 4171–4186. Association for Computational Linguistics, June 2019

    Google Scholar 

  11. Vaswani, A., et al.: Attention is all you need. ArXiv abs/1706.03762 (2017)

    Google Scholar 

  12. Reimers, N., Gurevych, I.: Sentence-BERT: sentence embeddings using Siamese BERT-networks. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China, pp. 3982–3992. Association for Computational Linguistics, November 2019

    Google Scholar 

  13. Schroff, F., Kalenichenko, D., Philbin, J.: Facenet: a unified embedding for face recognition and clustering. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 815–823 (2015)

    Google Scholar 

  14. Xu, J., Xu, B., Wang, P., Zheng, S., Tian, G., Zhao, J.: Self-taught convolutional neural networks for short text clustering. Neural Networks Off. J. Int. Neural Network Soc. 88, 22–31 (2017)

    Article  Google Scholar 

  15. Hadifar, A., Sterckx, L., Demeester, T., Develder, C.: A self-training approach for short text clustering. In: Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019). pp. 194–199. Association for Computational Linguistics, Florence, Italy (Aug 2019)

    Google Scholar 

  16. Rakib, M.R.H., Zeh, N., Jankowska, M., Milios, E.E.: Enhancement of short text clustering by iterative classification. Natural Lang. Process. Inf. Syst. 12089, 105–117 (2020)

    Google Scholar 

  17. Zhang, D., et al.: Supporting clustering with contrastive learning. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 5419–5430. Association for Computational Linguistics, Online, June 2021

    Google Scholar 

  18. Yin, J., Wang, J.: A model-based approach for text clustering with outlier detection. In: 2016 IEEE 32nd International Conference on Data Engineering (ICDE), pp. 625–636 (2016)

    Google Scholar 

  19. Hürriyetoğlu, A., Zavarella, V., Tanev, H., Yörük, E., Safaya, A., Mutlu, O.: Automated extraction of socio-political events from news (AESPEN): workshop and shared task report. In: Proceedings of the Workshop on Automated Extraction of Socio-political Events from News 2020, pp. 1–6, Marseille, France. European Language Resources Association (ELRA), May 2020. https://aclanthology.org/2020.aespen-1.1

  20. Bejan, C., Harabagiu, S.: Unsupervised event coreference resolution with rich linguistic features. In: Proceedings of the 48th Annual Meeting of the Association for computational Linguistics, pp. 1412–1422, Uppsala, Sweden. Association for Computational Linguistics, July 2010. https://aclanthology.org/P10-1143

  21. Lee, H., Recasens, M., Chang, A., Surdeanu, M., Jurafsky, D.: Joint entity and event coreference resolution across documents. In: Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 489–500, Jeju Island, Korea. Association for Computational Linguistics, July 2012. https://aclanthology.org/D12-1045

  22. Pennington, J., Socher, R., Manning, C.: GloVe: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543. Association for Computational Linguistics, Doha, Qatar, October 2014

    Google Scholar 

  23. Örs, F.K., Yeniterzi, S., Yeniterzi, R.: Event clustering within news articles. In: Proceedings of the Workshop on Automated Extraction of Socio-political Events from News 2020, Marseille, France, pp. 63–68. European Language Resources Association (ELRA), May 2020. https://aclanthology.org/2020.aespen-1.11

  24. Barhom, S., Shwartz, V., Eirew, A., Bugert, M., Reimers, N., Dagan, I.: Revisiting joint modeling of cross-document entity and event coreference resolution (2019)

    Google Scholar 

  25. Tan, F.A., Gollapalli, S.D., Ng, S.K.: NUS-IDS at CASE 2021 task 1: improving multilingual event sentence coreference identification with linguistic information. In: Proceedings of the 4th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE 2021), pp. 105–112. Association for Computational Linguistics, Online, Aug 2021

    Google Scholar 

  26. Lu, J., Ng, V.: Conundrums in event coreference resolution: Making sense of the state of the art. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 1368–1380, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics, November 2021

    Google Scholar 

  27. Joshi, M., Chen, D., Liu, Y., Weld, D.S., Zettlemoyer, L., Levy, O.: Span-BERT: improving pre-training by representing and predicting spans. Trans. Assoc. Comput. Linguist. 8, 64–77 (2020)

    Article  Google Scholar 

  28. Gao, T., Yao, X., Chen, D.: SimCSE: simple contrastive learning of sentence embeddings. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 6894–6910, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics, November 2021

    Google Scholar 

  29. Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: International Conference on Learning Representations (2017)

    Google Scholar 

  30. Laurensvan der Maaten, L., Hinton, G.E.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008)

    Google Scholar 

Download references

Acknowledgements

We thank the anonymous reviewers for providing insightful comments, suggestions and feedback. This research was supported by Sichuan Province Scientific and Technological Achievements Transfer and Transformation Demonstration Project, grant number 2022ZHCG0007.

Author information

Authors and Affiliations

  1. Chengdu Institute of Computer Applications, Chinese Academy of Sciences, Chengdu, 610041, China

    Tailai Peng, Rui Chen, Zhe Cui & Zheng Chen

  2. University of Chinese Academy of Sciences, Beijing, 101408, China

    Tailai Peng, Rui Chen, Zhe Cui & Zheng Chen

  3. School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, 610054, China

    Tailai Peng, Rui Chen, Zhe Cui & Zheng Chen

Authors
  1. Tailai Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rui Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhe Cui
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zheng Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tailai Peng .

Editor information

Editors and Affiliations

  1. Department of Computer Science, Eastern Institute of Technology, Zhejiang, China

    De-Shuang Huang

  2. University of Wollongong, North Wollongong, NSW, Australia

    Prashan Premaratne

  3. Zhengzhou University of Light Industry, Zhengzhou, China

    Baohua Jin

  4. Zhong Yuan University of Technology, Zhengzhou, China

    Boyang Qu

  5. University of Ulsan, Ulsan, Korea (Republic of)

    Kang-Hyun Jo

  6. Department of Computer Science, Liverpool John Moores University, Liverpool, UK

    Abir Hussain

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Peng, T., Chen, R., Cui, Z., Chen, Z. (2023). Simple but Effective: Keyword-Based Metric Learning for Event Sentence Coreference Identification. In: Huang, DS., Premaratne, P., Jin, B., Qu, B., Jo, KH., Hussain, A. (eds) Advanced Intelligent Computing Technology and Applications. ICIC 2023. Lecture Notes in Computer Science(), vol 14089. Springer, Singapore. https://doi.org/10.1007/978-981-99-4752-2_44

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-4752-2_44

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-4751-5

  • Online ISBN: 978-981-99-4752-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation