Skip to main content
SpringerLink
Log in
Book cover

International Conference of the Pacific Association for Computational Linguistics

PACLING 2019: Computational Linguistics pp 67–78Cite as

Context-Guided Self-supervised Relation Embeddings

  • Conference paper
  • First Online:

  • 677 Accesses

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1215))

Abstract

A semantic relation between two given words a and b can be represented using two complementary sources of information: (a) the semantic representations of a and b (expressed as word embeddings) and, (b) the contextual information obtained from the co-occurrence contexts of the two words (expressed in the form of lexico-syntactic patterns). Pattern-based approach suffers from sparsity while methods rely only on word embeddings for the related pairs lack of relational information. Prior works on relation embeddings have pre-dominantly focused on either one type of those two resources exclusively, except for a notable few exceptions. In this paper, we proposed a self-supervised context-guided Relation Embedding method (CGRE) using the two sources of information. We evaluate the learnt method to create relation representations for word-pairs that do not co-occur. Experimental results on SemEval-2012 task2 dataset show that the proposed operator outperforms other methods in representing relations for unobserved word-pairs.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    https://github.com/Huda-Hakami/Context-Guided-Relation-Embeddings.

  2. 2.

    http://nlp.stanford.edu/data/glove.6B.zip.

  3. 3.

    The accuracy of NLRA when its trained on pattern extracted using word pairs in the entire SemEval dataset is 45.28%, which is similar to the result reported in the original paper.

References

  1. Bollegala, D.T., Matsuo, Y., Ishizuka, M.: Relational duality: Unsupervised extraction of semantic relations between entities on the web. In: Proceedings of the 19th International Conference on World Wide Web, pp. 151–160. ACM (2010)

    Google Scholar 

  2. Cafarella, M.J., Banko, M., Etzioni, O.: Relational web search. In: WWW Conference (2006)

    Google Scholar 

  3. Nakov, P.: Improved statistical machine translation using monolingual paraphrases. In: Proceedings of ECAI 2008: 18th European Conference on Artificial Intelligence, 21–25 July 2008, Patras, Greece: Including Prestigious Applications of Intelligent Systems (PAIS 2008), vol. 178, p. 338. IOS Press (2008)

    Google Scholar 

  4. Yang, S., Zou, L., Wang, Z., Yan, J., Wen, J.-R.: Efficiently answering technical questions-a knowledge graph approach, pp. 3111–3118. In: AAAI (2017)

    Google Scholar 

  5. Joshi, M., Choi, E., Levy, O., Weld, D.S., Zettlemoyer, L.: pair2vec: compositional word-pair embeddings for cross-sentence inference. arXiv preprint arXiv:1810.08854 (2018)

  6. Hakami, H., Hayashi, K., Bollegala, D.: Why does pairdiff work? - A mathematical analysis of bilinear relational compositional operators for analogy detection. In: Proceedings of the 27th International Conference on Computational Linguistics (COLING) (2018)

    Google Scholar 

  7. Hakami, H., Bollegala, D.: Learning relation representations from word representations (2018)

    Google Scholar 

  8. Mikolov, T., Yih, W.-T., Zweig, G.: Linguistic regularities in continuous space word representations. In: Proceedings of HLT-NAACL, pp. 746–751 (2013)

    Google Scholar 

  9. Washio, K., Kato, T.: Filling missing paths: modeling co-occurrences of word pairs and dependency paths for recognizing lexical semantic relations. arXiv preprint arXiv:1809.03411 (2018)

  10. Turney, P.D.: Measuring semantic similarity by latent relational analysis. arXiv preprint cs/0508053 (2005)

    Google Scholar 

  11. Snow, R., Jurafsky, D., Ng, A.Y.: Learning syntactic patterns for automatic hypernym discovery. In: Advances in Neural Information Processing Systems, pp. 1297–1304 (2005)

    Google Scholar 

  12. Riedel, S., Yao, L., McCallum, A., Marlin, B.M.: Relation extraction with matrix factorization and universal schemas. In: Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 74–84 (2013)

    Google Scholar 

  13. Hearst, M.A.: Automatic acquisition of hyponyms from large text corpora. In: Proceedings of the 14th Conference on Computational Linguistics, vol. 2, pp. 539–545. Association for Computational Linguistics (1992)

    Google Scholar 

  14. Girju, R., Badulescu, A., Moldovan, D.: Learning semantic constraints for the automatic discovery of part-whole relations. In: Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology, vol. 1, pp. 1–8. Association for Computational Linguistics (2003)

    Google Scholar 

  15. Marshman, E.: The cause-effect relation in a biopharmaceutical corpus: English knowledge patterns. In: Terminology and Knowledge Engineering, pp. 89–94 (2002)

    Google Scholar 

  16. Bigham, J., Littman, M.L., Shnayder, V., Turney, P.D.: Combining independent modules to solve multiple-choice synonym and analogy problems. In: Proceedings of the International Conference on Recent Advances in Natural Language Processing, pp. 482–489 (2003)

    Google Scholar 

  17. Jameel, S., Bouraoui, Z., Schockaert, S.: Unsupervised learning of distributional relation vectors. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), vol. 1, pp. 23–33 (2018)

    Google Scholar 

  18. Pennington, J., Socher, R., Manning, C.: Glove: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)

    Google Scholar 

  19. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)

  20. Hakami, H., Bollegala, D.: Compositional approaches for representing relations between words: a comparative study. Knowl.-Based Syst. 136, 172–182 (2017)

    Article  Google Scholar 

  21. Gábor, K., Zargayouna, H., Tellier, I., Buscaldi, D., Charnois, T.: Exploring vector spaces for semantic relations. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 1814–1823 (2017)

    Google Scholar 

  22. Levy, O., Remus, S., Biemann, C., Dagan, I.: Do supervised distributional methods really learn lexical inference relations? In: Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 970–976 (2015)

    Google Scholar 

  23. Linzen, T.: Issues in evaluating semantic spaces using word analogies. arXiv preprint arXiv:1606.07736 (2016)

  24. Rogers, A., Drozd, A., Li, B.: The (too many) problems of analogical reasoning with word vectors. In: Proceedings of the 6th Joint Conference on Lexical and Computational Semantics (* SEM 2017), pp. 135–148 (2017)

    Google Scholar 

  25. Roller, S., Kiela, D., Nickel, M.: Hearst patterns revisited: automatic hypernym detection from large text corpora. arXiv preprint arXiv:1806.03191 (2018)

  26. Vylomova, E., Rimell, L., Cohn, T., Baldwin, T.: Take and took, gaggle and goose, book and read: evaluating the utility of vector differences for lexical relation learning. arXiv preprint arXiv:1509.01692 (2015)

  27. Zhila, A., Yih, W.-T., Meek, C., Zweig, G., Mikolov, T.: Combining heterogeneous models for measuring relational similarity. In: Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1000–1009 (2013)

    Google Scholar 

  28. Washio, K., Kato, T.: Neural latent relational analysis to capture lexical semantic relations in a vector space. arXiv preprint arXiv:1809.03401 (2018)

  29. Gladkova, A., Drozd, A., Matsuoka, S.: Analogy-based detection of morphological and semantic relations with word embeddings: what works and what doesn’t. In: Proceedings of the NAACL Student Research Workshop, pp. 8–15 (2016)

    Google Scholar 

  30. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  31. Patel, P., Davey, D., Panchal, V., Pathak, P.: Annotation of a large clinical entity corpus. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2033–2042 (2018)

    Google Scholar 

  32. Rosenberg, A., Hirschberg, J.: V-measure: a conditional entropy-based external cluster evaluation measure. In: Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL) (2007)

    Google Scholar 

  33. Turney, P.D.: The latent relation mapping engine: algorithm and experiments. J. Artif. Intell. Res. 33, 615–655 (2008)

    Article  Google Scholar 

  34. Duchi, J., Hazan, E., Singer, Y.: Adaptive subgradient methods for online learning and stochastic optimization. J. Mach. Learn. Res. 12(Jul), 2121–2159 (2011)

    MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Liverpool, Liverpool, UK

    Huda Hakami & Danushka Bollegala

  2. Department of Computer Science, Taif Univeristy, Taif, Saudi Arabia

    Huda Hakami

Authors
  1. Huda Hakami
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Danushka Bollegala
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huda Hakami .

Editor information

Editors and Affiliations

  1. Japan Advanced Institute of Science and Technology, Ishikawa, Japan

    Le-Minh Nguyen

  2. University of Engineering and Technology, Hanoi, Vietnam

    Xuan-Hieu Phan

  3. Graduate School of Information Science and Technology, The University of Tokyo, Tokyo, Japan

    Kôiti Hasida

  4. Japan Advanced Institute of Science and Technology, Ishikawa, Japan

    Satoshi Tojo

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Hakami, H., Bollegala, D. (2020). Context-Guided Self-supervised Relation Embeddings. In: Nguyen, LM., Phan, XH., Hasida, K., Tojo, S. (eds) Computational Linguistics. PACLING 2019. Communications in Computer and Information Science, vol 1215. Springer, Singapore. https://doi.org/10.1007/978-981-15-6168-9_6

Download citation

  • DOI: https://doi.org/10.1007/978-981-15-6168-9_6

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-15-6167-2

  • Online ISBN: 978-981-15-6168-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation