Abstract
Knowledge graph embedding has been proposed to embed entities and relations into continuous vector spaces, which can benefit various downstream tasks, such as question answering and recommender systems, etc. A common assumption of existing knowledge graph embedding models is that the relation is a translation vector connecting the embedded head entity and tail entity. However, based on this assumption, the same relation connecting multiple entities may form a circle and lead to mistakes during computing process. To solve this so-called circular relation problem that has been ignored previously, we propose a novel method called ContE (Contextualized Embedding) for knowledge graphs by exploring collaborative relations. Specifically, each collaborative relation combines an explicit relation and a latent relation, where the explicit one is the original relation between two entities, and the latent one is introduced to capture the implicit interactions obtained via the context information of the two entities. With the collaborative relations, the same relations will be embedded varying across different contexts, and the sharing of similar semantics will be guaranteed as well. We conduct extensive experiments in link prediction and triple classification tasks on five benchmark datasets. The experimental results demonstrate that the proposed ContE achieves improvements over several baselines.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Balazevic I, Allen C, Hospedales TM (2019) Tucker: Tensor factorization for knowledge graph completion. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP, pp 5184–5193
Bollacker K, Evans C, Paritosh P, et al (2008) Freebase: A collaboratively created graph database for structuring human knowledge. In: Proc. the 2008 ACM SIGMOD International Conference on Management of Data, ICMD, pp 1247–1250
Bordes A, Glorot X, Weston J, et al (2012) A semantic matching energy function for learning with multi-relational data. Machine Learning pp 233–259
Bordes A, Usunier N, Garcia-Duran A et al (2013) Translating embeddings for modeling multi-relational data. Proc. Advances in Neural Information Processing Systems, NIPS, pp 2787–2795
Carlson A, Betteridge J, Kisiel B, et al (2010) Toward an architecture for never-ending language learning. In: Proc. the 24th AAAI Conference on Artificial Intelligence, AAAI, pp 1306–1313
Chami I, Wolf A, Juan DC, et al (2020) Low-dimensional hyperbolic knowledge graph embeddings. In: Proc. the 58th Annual Meeting of the Association for Computational Linguistics, ACL, pp 6901–6914
Cui W, Xiao Y, Wang H, et al (2017) Kbqa: Learning question answering over qa corpora and knowledge bases. In: Proc. PVLDB, pp 565–576
Deng Y, Xie Y, Li Y, et al (2019) Multi-task learning with multi-view attention for answer selection and knowledge base question answering. In: Proc. the 33rd AAAI Conference on Artificial Intelligence, AAAI, pp 6318–6325
Dettmers T, Minervini P, Stenetorp P, et al (2018) Convolutional 2d knowledge graph embeddings. In: Proc. the 32th AAAI Conference on Artificial Intelligence, AAAI
Ebisu T, Ichise R (2018) Toruse: Knowledge graph embedding on a lie group. In: Proc. the 32nd AAAI Conference on Artificial Intelligence, AAAI
Han X, Cao S, Lv X, et al (2018) Openke: An open toolkit for knowledge embedding. In: Proc. the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP, pp 139–144
Ji G, He S, Xu L, et al (2015) Knowledge graph embedding via dynamic mapping matrix. In: Proc. the 53rd Annual Meeting of the Association for Computational Linguistics, ACL, pp 687–696
Ji G, Liu K, He S, et al (2016) Knowledge graph completion with adaptive sparse transfer matrix. In: Proc. the 30th AAAI Conference on Artificial Intelligence, AAAI, pp 985–991
Kazemi SM, Poole D (2018) Simple embedding for link prediction in knowledge graphs. Proc. Advances in Neural Information Processing Systems, NIPS, pp 4284–4295
Kingma, Diederik, Jimmy B (2014) Adam: A method for stochastic optimization. Computer Science
Kolyvakis, Prodromos, Kalousis A, et al (2019) Hyperkg: Hyperbolic knowledge graph embeddings for knowledge base completion. In: arXiv
Lin Y, Liu Z, Sun M, et al (2015) Learning entity and relation embeddings for knowledge graph completion. In: Proc. the 29th AAAI Conference on Artificial Intelligence Learning, AAAI, pp 2181–2187
Liu H, Wu Y, Yang Y (2017) Analogical inference for multi-relational embeddings. In: Proc. the 34th International Conference on Machine Learning, ICML, pp 2068–2078
Nathani D, Chauhan J, Sharma C, et al (2019) Learning attention-based embeddings for relation prediction in knowledge graphs. In: Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL, pp 4710–4723
Nayyeri M, Xu C, Yaghoobzadeh Y, et al (2019) Toward understanding the effect of loss function on then performance of knowledge graph embedding. In: arXiv
Nguyen DQ, Nguyen TD, Nguyen DQ, et al (2018) A novel embedding model for knowledge base completion based on convolutional neural network. In: Proc. the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL, pp 327–333
Nickel, Tresp, Kriegel (2011) A three-way model for collective learning on multi-relational data. In: Proc. the 28th International Conference on Machine Learning, ICML, pp 809–816
Nickel M, Rosasco L, Poggio T (2016) Holographic embeddings of knowledge graphs. In: Proc. the 30th AAAI Conference on Artificial Intelligence, AAAI, pp 1955–1961
Socher R, Chen D, Manning CD et al (2013) Reasoning with neural tensor networks for knowledge base completion. Proc. Advances in Neural Information Processing Systems, NIPS, pp 926–934
Sun Z, Deng ZH, Nie JY, et al (2019a) Rotate: Knowledge graph embedding by relational rotation in complex space. In: Proc. the 7th International Conference on Learning Representations, ICLR
Sun Z, Huang J, Hu W, et al (2019b) Transedge: Translating relation-contextualized embeddings for knowledge graphs. In: Proc. the 18th International Semantic Web Conference, ISWC, pp 612–629
Toutanova K, Chen D (2015) Observed versus latent features for knowledge base and text inference. In: Proc. Workshop on Continuous Vector Space MODELS and Their Compositionality, pp 57–66
Trouillon, Welb J, Riedel S, et al (2016) Complex embeddings for simple link prediction. In: Proc. the 33th International Conference on Machine Learning, ICML, pp 2071–2080
Vashishth S, Sanyal S, Nitin V, et al (2020) Composition-based multi-relational graph convolutional networks. In: 8th International Conference on Learning Representations, ICLR
Vaswani A, Shazeer N, Parmar N, et al (2017) Attention is all you need. In: Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems, NIPS, pp 5998–6008
Velickovic P, Cucurull G, Casanova A, et al (2018) Graph attention networks. In: 6th International Conference on Learning Representations, ICLR
Wang H, Zhang F, Xie X, et al (2018a) Dkn: Deep knowledge-aware network for news recommendation. In: Proc. the 2018 World Wide Web Conference on World Wide Web, WWW, pp 1835–1844
Wang H, Zhang F, Xie X, et al (2018b) Dkn: Deep knowledge-aware network for news recommendation. In: Proc. International World Wide Web Conferences, WWW, pp 1835–1844
Wang H, Zhang F, Zhao M, et al (2019a) Multi-task feature learning for knowledge graph enhanced recommendation. In: Proc. the 2019 World Wide Web Conference on World Wide Web, WWW, pp 2000–2010
Wang Q, Mao Z, Wang B, et al (2017) Knowledge graph embedding: A survey of approaches and applications. IEEE Transactions on Knowledge & Data Engineering pp 2724–2743
Wang X, He X, Cao Y, et al (2019b) Kgat: Knowledge graph attention network for recommendation. In: Proc. the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD, pp 950–958
Wang Z, Zhang J, Feng J, et al (2014) Knowledge graph embedding by translating on hyperplanes. In: Proc. the 28th AAAI Conference on Artificial Intelligence, AAAI, pp 1112–1119
Xiao H, Huang M, Zhu X (2016) Transg :a generative model for knowledge graph embedding. In: Proc. the 54th Annual Meeting of the Association for Computational Linguistics, ACL, pp 2316–2325
Xiong W, Hoang T, Wang WY (2017) Deeppath: A reinforcement learning method for knowledge graph reasoning. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP, pp 564–573
Yang B, tau Yih W, He X, et al (2015) Embedding entities and relations for learning and inference in knowledge bases. In: Proc. the International Conference on Learning Representations, ICLR
Zhang W, Paudel B, Wang L (2019) Iteratively learning embeddings and rules for knowledge graph reasoning. In: Proc. International World Wide Web Conferences, WWW, pp 2366–2377
Zhang Z, Zhuang F, Qu M, et al (2018) Knowledge graph embedding with hierarchical relation structure. In: Proc. the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP, pp 3198–3207
Zhou X, Zhu Q, Liu P, et al (2017) Learning knowledge embeddings by combining limit-based scoring loss. In: Proc. the 2017 Conference on Information and Knowledge Management, pp 1009–1018
Funding
No funding was received for conducting this study.
Author information
Authors and Affiliations
Corresponding author
Additional information
Responsible editor: Tim Weninger.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Ma, T., Li, M., Lv, S. et al. ContE: contextualized knowledge graph embedding for circular relations. Data Min Knowl Disc 37, 110–135 (2023). https://doi.org/10.1007/s10618-022-00851-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10618-022-00851-2