Skip to main content
SpringerLink
Log in

SMDM: Tackling zero-shot relation extraction with semantic max-divergence metric learning

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

In zero-shot relation extraction, existing methods usually learn semantic features from seen relations to infer unseen relations. However, because there is no instance of unseen relation that can be used for training, it is still a challenge for the existing models to learn the semantic gap between seen relations and unseen relations, resulting in poor generalization performance of the learned semantic features. Therefore, we propose a Semantic Max-Divergence Metric (SMDM) based method to measure the distances between relations from both direct and indirect semantic differences. For that, we learn multiple binary feature reference-spaces to extract the semantic divergences of each unseen relation instance relative to each seen relation, which can be converted to a relative-affinity (RA) matrix as indirect semantic metrics. Furthermore, we combine RA with direct semantic metrics based on BERT to maximum the divergences between unseen relation instances and get clearer unseen relation boundaries. Empirical results on benchmark datasets demonstrate SMDM can the superior improvement on F1-score and external indicators of SMDM compared to the state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Alt C, Gabryszak A, Hennig L (2020) Probing linguistic features of sentence-level representations in relation extraction. In: Jurafsky D, Chai J, Schluter N, et al (eds) Proceedings of the 58th annual meeting of the association for computational linguistics, ACL 2020, Online, July 5-10, 2020. association for computational linguistics. https://doi.org/10.18653/v1/2020.acl-main.140, pp 1534–1545

  2. Baldini Soares L, FitzGerald N, Ling J et al (2019) Matching the blanks: Distributional similarity for relation learning. In: Proceedings of the 57th Annual meeting of the association for computational linguistics. association for computational linguistics, Florence, Italy. https://doi.org/10.18653/v1/P19-1279. https://www.aclweb.org/anthology/P19-1279, pp 2895–2905

  3. Califf ME, Mooney RJ (1997) Relational learning of pattern-match rules for information extraction. In: CoNLL97: Computational natural language learning. https://www.aclweb.org/anthology/W97-1002

  4. Chen C, Li C (2021) ZS-BERT: towards zero-shot relation extraction with attribute representation learning. In: Toutanova K, Rumshisky A, Zettlemoyer L, et al (eds) Proceedings of the 2021 conference of the north american chapter of the association for computational linguistics: human language technologies, naacl-hlt 2021, online, June 6-11, 2021. association for computational linguistics. https://doi.org/10.18653/v1/2021.naacl-main.272, pp 3470–3479

  5. Chen M, Zhang W, Zhang W et al (2019) Meta relational learning for few-shot link prediction in knowledge graphs. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (emnlp-ijcnlp). association for computational linguistics, Hong Kong, China. https://doi.org/10.18653/v1/D19-1431. https://www.aclweb.org/anthology/D19-1431, pp 4217–4226

  6. Cong X, Yu B, Liu T et al (2020) Inductive unsupervised domain adaptation for few-shot classification via clustering. arXiv:200612816

  7. Devlin J, Chang MW, Lee K et al (2019) BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 conference of the north american chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers). association for computational linguistics, Minneapolis, Minnesota. https://doi.org/10.18653/v1/N19-1423. https://www.aclweb.org/anthology/N19-1423, pp 4171–4186

  8. Gao T, Han X, Zhu H et al (2019) FewRel 2.0: Towards more challenging few-shot relation classification. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Association for Computational Linguistics, Hong Kong, China. https://doi.org/10.18653/v1/D19-1649. https://www.aclweb.org/anthology/D19-1649, pp 6250–6255

  9. Gao T, Han X, Xie R et al (2020) Neural snowball for few-shot relation learning. In: The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020. AAAI Press. https://aaai.org/ojs/index.php/AAAI/article/view/6281, pp 7772–7779

  10. Gong J, Eldardiry H (2020) Zero-shot learning for relation extraction. arXiv:201107126

  11. Hadsell R, Chopra S, LeCun Y (2006) Dimensionality reduction by learning an invariant mapping. In: 2006 IEEE computer society conference on Computer Vision and Pattern Recognition (CVPR 2006), 17-22 June 2006, New York, NY, USA. IEEE Computer Society. https://doi.org/10.1109/CVPR.2006.100, pp 1735–1742

  12. Han X, Zhu H, Yu P et al (2018) FewRel: A large-scale supervised few-shot relation classification dataset with state-of-the-art evaluation. In: Proceedings of the 2018 conference on empirical methods in natural language processing. association for computational linguistics, Brussels, Belgium. https://doi.org/10.18653/v1/D18-1514. https://www.aclweb.org/anthology/D18-1514, pp 4803–4809

  13. Han X, Gao T, Lin Y et al (2020) More data, more relations, more context and more openness: A review and outlook for relation extraction. In: Proceedings of the 1st conference of the asia-pacific chapter of the association for computational linguistics and the 10th international joint conference on natural language processing. association for computational linguistics, Suzhou, China. https://www.aclweb.org/anthology/2020.aacl-main.75, pp 745–758

  14. Hendrickx I, Kim SN, Kozareva Z et al (2009) SemEval-2010 task 8: Multi-way classification of semantic relations between pairs of nominals. In: Proceedings of the workshop on semantic evaluations: recent achievements and future directions (SEW-2009). association for computational linguistics, Boulder, Colorado. https://www.aclweb.org/anthology/W09-2415, pp 94–99

  15. Hu X, Wen L, Xu Y et al (2020) SelfORE: Self-supervised relational feature learning for open relation extraction. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Online. https://doi.org/10.18653/v1/2020.emnlp-main.299. https://www.aclweb.org/anthology/2020.emnlp-main.299, pp 3673–3682

  16. Huffman SB (1995) Learning information extraction patterns from examples. In: Wermter S, Riloff E, Scheler G (eds) Connectionist, statistical, and symbolic approaches to learning for natural language processing, lecture notes in computer science, vol 1040. Springer. https://doi.org/10.1007/3-540-60925-3_51, pp 246–260

  17. Kambhatla N (2004) Combining lexical, syntactic, and semantic features with maximum entropy models for information extraction. In: Proceedings of the ACL interactive poster and demonstration sessions. Association for computational linguistics, Barcelona, Spain. https://www.aclweb.org/anthology/P04-3022, pp 178–181

  18. Lampert CH, Nickisch H, Harmeling S (2009) Learning to detect unseen object classes by between-class attribute transfer. In: 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2009), 20-25 June 2009, Miami, Florida, USA. IEEE Computer Society. https://doi.org/10.1109/CVPR.2009.5206594, pp 951–958

  19. Lan Z, Chen M, Goodman S et al (2020) ALBERT: A lite BERT for self-supervised learning of language representations. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net. https://openreview.net/forum?id=H1eA7AEtvS

  20. Levy O, Seo M, Choi E et al (2017) Zero-shot relation extraction via reading comprehension. In: Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017). Association for Computational Linguistics, Vancouver, Canada. https://doi.org/10.18653/v1/K17-1034. https://www.aclweb.org/anthology/K17-1034, pp 333–342

  21. Liu Y, Ott M, Goyal N et al (2019) Roberta: A robustly optimized bert pretraining approach. arXiv:190711692

  22. Mintz M, Bills S, Snow R et al (2009) Distant supervision for relation extraction without labeled data. In: Proceedings of the joint conference of the 47th annual meeting of the ACL and the 4th international joint conference on natural language processing of the AFNLP. Association for computational linguistics, Suntec, Singapore. https://www.aclweb.org/anthology/P09-1113, pp 1003–1011

  23. Peng H, Gao T, Han X et al (2020) Learning from Context or Names? An Empirical Study on Neural Relation Extraction. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Online. https://doi.org/10.18653/v1/2020.emnlp-main.298. https://www.aclweb.org/anthology/2020.emnlp-main.298, pp 3661–3672

  24. Reimers N, Gurevych I (2019) Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Association for Computational Linguistics, Hong Kong, China. https://doi.org/10.18653/v1/D19-1410. https://www.aclweb.org/anthology/D19-1410, pp 3982–3992

  25. Rosenman S, Jacovi A, Goldberg Y (2020) Exposing Shallow Heuristics of Relation Extraction Models with Challenge Data. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Online. https://doi.org/10.18653/v1/2020.emnlp-main.302. https://www.aclweb.org/anthology/2020.emnlp-main.302, pp 3702–3710

  26. Roth D, Yih Wt (2002) Probabilistic reasoning for entity & relation recognition. In: COLING 2002: The 19th International Conference on Computational Linguistics. https://www.aclweb.org/anthology/C02-1151

  27. Snell J, Swersky K, Zemel RS (2017) Prototypical networks for few-shot learning. In: Guyon I, von Luxburg U, Bengio S, et al (eds) Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA. https://proceedings.neurips.cc/paper/2017/hash/cb8da6767461f2812ae4290eac7cbc42-Abstract.html, pp 4077–4087

  28. Tran TT, Le P, Ananiadou S (2020) Revisiting unsupervised relation extraction. In: Proceedings of the 58th Annual meeting of the association for computational linguistics. Association for computational linguistics, Online. https://doi.org/10.18653/v1/2020.acl-main.669. https://www.aclweb.org/anthology/2020.acl-main.669, pp 7498–7505

  29. Trisedya BD, Weikum G, Qi J et al (2019) Neural relation extraction for knowledge base enrichment. In: Proceedings of the 57th annual meeting of the association for computational linguistics. Association for computational linguistics, Florence, Italy. https://doi.org/10.18653/v1/P19-1023. https://www.aclweb.org/anthology/P19-1023, pp 229–240

  30. Van D, Hinton G (2008) 2008. visualizing high-dimensional data using t-sne. J Mach Learn Res 9(2):2579–2605

    MATH  Google Scholar 

  31. Wu R, Yao Y, Han X et al (2019) Open relation extraction: Relational knowledge transfer from supervised data to unsupervised data. In: https://doi.org/10.18653/v1/D19-1021. https://www.aclweb.org/anthology/D19-1021, pp 219–228

  32. Wu S, He Y (2019) Enriching pre-trained language model with entity information for relation classification. In: Zhu W, Tao D, Cheng X, et al (eds) Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM 2019, Beijing, China, November 3-7, 2019. ACM. https://doi.org/10.1145/3357384.3358119, pp 2361–2364

  33. Xiong W, Yu M, Chang S et al (2018) One-shot relational learning for knowledge graphs. In: Proceedings of the 2018 conference on empirical methods in natural language processing. Association for computational linguistics, Brussels, Belgium. https://doi.org/10.18653/v1/D18-1223. https://www.aclweb.org/anthology/D18-1223, pp 1980–1990

  34. Yu D, Huang L, Ji H (2017) Open relation extraction and grounding. In: Proceedings of the eighth international joint conference on natural language processing (vol 1: Long Papers). Asian federation of natural language processing, Taipei, Taiwan. https://www.aclweb.org/anthology/I17-1086, pp 854–864

  35. Zeng D, Liu K, Lai S et al (2014) Relation classification via convolutional deep neural network. In: Proceedings of COLING 2014, the 25th international conference on computational linguistics: technical papers. Dublin City University and Association for Computational Linguistics, Dublin, Ireland. https://www.aclweb.org/anthology/C14-1220, pp 2335–2344

  36. Zhang D, Wang D (2015) Relation classification via recurrent neural network. arXiv:150801006

Download references

Author information

Authors and Affiliations

  1. College of Artificial Intelligence, Beijing University of Posts and Telecommunications, Haidian District, 10 Xitucheng Road, Beijing, 100876, Beijing, China

    Bosen Zhang, Yajing Xu, Jinglei Li, Shusen Wang & Sheng Gao

  2. The National Computer Network Emergency Response Technical Team, Coordination Center of China, Haidian District, Beijing Zhongguancun South Street on the 4th, Beijing, 100190, Beijing, China

    Boya Ren

Authors
  1. Bosen Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yajing Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jinglei Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shusen Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Boya Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Sheng Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yajing Xu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, B., Xu, Y., Li, J. et al. SMDM: Tackling zero-shot relation extraction with semantic max-divergence metric learning. Appl Intell 53, 6569–6584 (2023). https://doi.org/10.1007/s10489-022-03596-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03596-z

Keywords

Search

Navigation