research-article

DiriE: Knowledge Graph Embedding with Dirichlet Distribution

Authors:

Zhongbao Zhang,

Yang YanAuthors Info & Claims

WWW '22: Proceedings of the ACM Web Conference 2022

Pages 3082 - 3091

https://doi.org/10.1145/3485447.3512028

Published: 25 April 2022 Publication History

Abstract

Knowledge graph embedding aims to learn representations of entities and relations in low-dimensional space. Recently, extensive studies combine the characteristics of knowledge graphs with different geometric spaces, including Euclidean space, complex space, hyperbolic space and others, which achieves significant progress in representation learning. However, existing methods are subject to at least one of the following limitations: 1) ignoring the uncertainty, 2) incapability of complex relation patterns. To address the above issues simultaneously, we propose a novel model named DiriE, which embeds entities as Dirichlet distributions and relations as multinomial distributions. DiriE employs Bayesian inference to measure the relations between entities and learns binary embeddings of knowledge graphs for modeling complex relation patterns. Additionally, we propose a two-step negative triple generation method that generates negative triples of both entities and relations. We conduct a solid theoretical analysis to demonstrate the effectiveness and robustness of our method, including the expressiveness of complex relation patterns and the ability to model uncertainty. Furthermore, extensive experiments show that our method outperforms state-of-the-art methods in link prediction on benchmark datasets.

References

[1]

Ralph Abboud, Ismail Ceylan, Thomas Lukasiewicz, and Tommaso Salvatori. 2020. BoxE: A Box Embedding Model for Knowledge Base Completion. Advances in Neural Information Processing Systems 33 (2020).

[2]

Saadullah Amin, Stalin Varanasi, Katherine Ann Dunfield, and Günter Neumann. 2020. LowFER: Low-rank bilinear pooling for link prediction. In International Conference on Machine Learning. 257–268.

[3]

Sören Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary Ives. 2007. Dbpedia: A nucleus for a web of open data. In The semantic web. Springer, 722–735.

Digital Library

[4]

Ivana Balazevic, Carl Allen, and Timothy Hospedales. 2019. Multi-relational poincaré graph embeddings. Advances in Neural Information Processing Systems 32 (2019), 4463–4473.

[5]

Ivana Balažević, Carl Allen, and Timothy Hospedales. 2019. TuckER: Tensor Factorization for Knowledge Graph Completion. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 5185–5194.

[6]

Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: a collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD international conference on Management of data. 1247–1250.

Digital Library

[7]

Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating Embeddings for Modeling Multi-relational Data. Advances in Neural Information Processing Systems 26 (2013), 2787–2795.

[8]

Antoine Bordes, Jason Weston, Ronan Collobert, and Yoshua Bengio. 2011. Learning structured embeddings of knowledge bases. In Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence. 301–306.

Digital Library

[9]

Yixin Cao, Xiang Ji, Xin Lv, Juanzi Li, Yonggang Wen, and Hanwang Zhang. 2021. Are Missing Links Predictable? An Inferential Benchmark for Knowledge Graph Completion. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 6855–6865.

[10]

Zongsheng Cao, Qianqian Xu, Zhiyong Yang, Xiaochun Cao, and Qingming Huang. 2021. Dual Quaternion Knowledge Graph Embeddings. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 6894–6902.

[11]

Ines Chami, Adva Wolf, Da-Cheng Juan, Frederic Sala, Sujith Ravi, and Christopher Ré. 2020. Low-Dimensional Hyperbolic Knowledge Graph Embeddings. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 6901–6914.

[12]

Linlin Chao, Jianshan He, Taifeng Wang, and Wei Chu. 2021. PairRE: Knowledge Graph Embeddings via Paired Relation Vectors. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics. 4360–4369.

[13]

Xuelu Chen, Michael Boratko, Muhao Chen, Shib Sankar Dasgupta, Xiang Lorraine Li, and Andrew McCallum. 2021. Probabilistic Box Embeddings for Uncertain Knowledge Graph Reasoning. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 882–893.

[14]

Xuelu Chen, Muhao Chen, Weijia Shi, Yizhou Sun, and Carlo Zaniolo. 2019. Embedding uncertain knowledge graphs. In Proceedings of the AAAI Conference on Artificial Intelligence. 3363–3370.

Digital Library

[15]

Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. 2018. Convolutional 2d knowledge graph embeddings. In Thirty-second AAAI conference on artificial intelligence, Vol. 32.

[16]

Xin Dong, Evgeniy Gabrilovich, Geremy Heitz, Wilko Horn, Ni Lao, Kevin Murphy, Thomas Strohmann, Shaohua Sun, and Wei Zhang. 2014. Knowledge vault: A web-scale approach to probabilistic knowledge fusion. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. 601–610.

Digital Library

[17]

Jun Feng, Minlie Huang, Mingdong Wang, Mantong Zhou, Yu Hao, and Xiaoyan Zhu. 2016. Knowledge graph embedding by flexible translation. In Proceedings of the Fifteenth International Conference on Principles of Knowledge Representation and Reasoning. 557–560.

Digital Library

[18]

Xingcheng Fu, Jianxin Li, Jia Wu, Qingyun Sun, Cheng Ji, Senzhang Wang, Jiajun Tan, Hao Peng, and S Yu Philip. 2021. ACE-HGNN: Adaptive Curvature Exploration Hyperbolic Graph Neural Network. In 2021 IEEE International Conference on Data Mining (ICDM). IEEE, 111–120.

[19]

Shizhu He, Kang Liu, Guoliang Ji, and Jun Zhao. 2015. Learning to represent knowledge graphs with gaussian embedding. In Proceedings of the 24th ACM international on conference on information and knowledge management. 623–632.

Digital Library

[20]

Guoliang Ji, Shizhu He, Liheng Xu, Kang Liu, and Jun Zhao. 2015. Knowledge graph embedding via dynamic mapping matrix. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 687–696.

[21]

Shaoxiong Ji, Shirui Pan, Erik Cambria, Pekka Marttinen, and S Yu Philip. 2021. A survey on knowledge graphs: Representation, acquisition, and applications. IEEE Transactions on Neural Networks and Learning Systems (2021).

[22]

Seyed Mehran Kazemi and David Poole. 2018. SimplE embedding for link prediction in knowledge graphs. In Proceedings of the 32nd International Conference on Neural Information Processing Systems. 4289–4300.

[23]

Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. 2015. Learning entity and relation embeddings for knowledge graph completion. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, Vol. 29. 2181–2187.

Digital Library

[24]

Robert Logan, Nelson F Liu, Matthew E Peters, Matt Gardner, and Sameer Singh. 2019. Barack’s Wife Hillary: Using Knowledge Graphs for Fact-Aware Language Modeling. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 5962–5971.

[25]

Farzaneh Mahdisoltani, Joanna Biega, and Fabian Suchanek. 2014. Yago3: A knowledge base from multilingual wikipedias. In 7th biennial conference on innovative data systems research. CIDR Conference.

[26]

George A Miller. 1995. WordNet: a lexical database for English. Commun. ACM 38, 11 (1995), 39–41.

Digital Library

[27]

Deepak Nathani, Jatin Chauhan, Charu Sharma, and Manohar Kaul. 2019. Learning Attention-based Embeddings for Relation Prediction in Knowledge Graphs. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 4710–4723.

[28]

Mojtaba Nayyeri, Gökce Müge Cil, Sahar Vahdati, Francesco Osborne, Andrey Kravchenko, Simone Angioni, Angelo Salatino, Diego Reforgiato Recupero, Enrico Motta, and Jens Lehmann. 2021. Link prediction of weighted triples for knowledge graph completion within the scholarly domain. IEEE Access 9(2021), 116002–116014.

[29]

Tu Dinh Nguyen, Dat Quoc Nguyen, Dinh Phung, 2018. A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers). 327–333.

[30]

Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. 2011. A three-way model for collective learning on multi-relational data. In Proceedings of the 28th International Conference on International Conference on Machine Learning. 809–816.

[31]

H Ren, W Hu, and J Leskovec. 2020. Query2box: Reasoning Over Knowledge Graphs In Vector Space Using Box Embeddings. In International Conference on Learning Representations (ICLR).

[32]

Hongyu Ren and Jure Leskovec. 2020. Beta Embeddings for Multi-Hop Logical Reasoning in Knowledge Graphs. Advances in Neural Information Processing Systems 33 (2020).

[33]

Fabian M Suchanek, Gjergji Kasneci, and Gerhard Weikum. 2007. Yago: a core of semantic knowledge. In Proceedings of the 16th international conference on World Wide Web. 697–706.

Digital Library

[34]

Li Sun, Zhongbao Zhang, Junda Ye, Hao Peng, Jiawei Zhang, Sen Su, and Philip S Yu. 2022. A Self-supervised Mixed-curvature Graph Neural Network. In Proceedings of the AAAI Conference on Artificial Intelligence.

[35]

Li Sun, Zhongbao Zhang, Jiawei Zhang, Feiyang Wang, Yang Du, Sen Su, and S Yu Philip. 2020. PERFECT: A Hyperbolic Embedding for Joint User and Community Alignment. In 2020 IEEE International Conference on Data Mining (ICDM). IEEE, 501–510.

[36]

Li Sun, Zhongbao Zhang, Jiawei Zhang, Feiyang Wang, Hao Peng, Sen Su, and Philip S Yu. 2021. Hyperbolic variational graph neural network for modeling dynamic graphs. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 4375–4383.

[37]

Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. 2019. RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space. In International Conference on Learning Representations.

[38]

Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. 2016. Complex embeddings for simple link prediction. In Proceedings of the 33rd International Conference on International Conference on Machine Learning-Volume 48. 2071–2080.

Digital Library

[39]

Stephen Tu. 2014. The dirichlet-multinomial and dirichlet-categorical models for bayesian inference. Computer Science Division, UC Berkeley 2 (2014).

[40]

Guojia Wan and Bo Du. 2021. GaussianPath: A Bayesian Multi-Hop Reasoning Framework for Knowledge Graph Reasoning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 4393–4401.

[41]

Feiyang Wang, Li Sun, and Zhongbao Zhang. 2020. Hyperbolic User Identity Linkage across Social Networks. In GLOBECOM 2020-2020 IEEE Global Communications Conference. IEEE, 1–6.

Digital Library

[42]

Xiang Wang, Xiangnan He, Yixin Cao, Meng Liu, and Tat-Seng Chua. 2019. Kgat: Knowledge graph attention network for recommendation. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 950–958.

Digital Library

[43]

Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014. Knowledge graph embedding by translating on hyperplanes. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 28. 1112–1119.

[44]

Han Xiao, Minlie Huang, and Xiaoyan Zhu. 2016. TransG: A Generative Model for Knowledge Graph Embedding. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2316–2325.

[45]

Qizhe Xie, Xuezhe Ma, Zihang Dai, and Eduard Hovy. 2017. An Interpretable Knowledge Transfer Model for Knowledge Base Completion. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 950–962.

[46]

Wenhan Xiong, Thien Hoang, and William Yang Wang. 2017. DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 564–573.

[47]

Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. 2015. Embedding entities and relations for learning and inference in knowledge bases. In International Conference on Learning Representations.

[48]

ShuAi Zhang, Yi Tay, Lina Yao, and Qi Liu. 2019. Quaternion Knowledge Graph Embeddings. Advances in Neural Information Processing Systems 32 (2019), 2735–2745.

[49]

Zhanqiu Zhang, Jianyu Cai, Yongdong Zhang, and Jie Wang. 2020. Learning hierarchy-aware knowledge graph embeddings for link prediction. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 3065–3072.

Cited By

Liu CKwong SWang X(2025)Uncertainty modeling for inductive knowledge graph embeddingNeural Networks10.1016/j.neunet.2024.107103184(107103)Online publication date: Apr-2025
https://doi.org/10.1016/j.neunet.2024.107103
Yang YChen JXiang Y(2025)A review on the reliability of knowledge graph: from a knowledge representation learning perspectiveWorld Wide Web10.1007/s11280-024-01316-w28:1Online publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1007/s11280-024-01316-w
Islam MLiu F(2024)Domain-Specific Dictionary between Human and Machine LanguagesInformation10.3390/info1503014415:3(144)Online publication date: 5-Mar-2024
https://doi.org/10.3390/info15030144
Show More Cited By

Index Terms

DiriE: Knowledge Graph Embedding with Dirichlet Distribution

Index terms have been assigned to the content through auto-classification.

Recommendations

Knowledge graph embedding by relational and entity rotation▪
Abstract
Knowledge graphs are typical large-scale multi-relational structures and useful for many artificial intelligence tasks. However, knowledge graphs often have missing facts, which limits the development of downstream tasks. To refine the knowledge ...
An introduction to the imprecise Dirichlet model for multinomial data

The imprecise Dirichlet model (IDM) was recently proposed by Walley as a model for objective statistical inference from multinomial data with chances @q. In the IDM, prior or posterior uncertainty about @q is described by a set of Dirichlet ...
Knowledge Graph Embedding: A Locally and Temporally Adaptive Translation-Based Approach

A knowledge graph is a graph with entities of different types as nodes and various relations among them as edges. The construction of knowledge graphs in the past decades facilitates many applications, such as link prediction, web search analysis, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '22: Proceedings of the ACM Web Conference 2022

April 2022

3764 pages

ISBN:9781450390965

DOI:10.1145/3485447

Editors:
Frédérique Laforest
INSA Lyon, France
,
Raphaël Troncy
EURECOM, France
,
Elena Simperl
King’s College London, UK
,
Deepak Agarwal
Pinterest, USA
,
Aristides Gionis
KTH Royal Institute of Technology, Sweden
,
Ivan Herman
W3C / retired
,
Lionel Médini
Université Lyon 1, France

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 April 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

National Natural Science Foundation of China

Conference

WWW '22

Sponsor:

SIGWEB

WWW '22: The ACM Web Conference 2022

April 25 - 29, 2022

Virtual Event, Lyon, France

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
644
Total Downloads

Downloads (Last 12 months)76
Downloads (Last 6 weeks)4

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liu CKwong SWang X(2025)Uncertainty modeling for inductive knowledge graph embeddingNeural Networks10.1016/j.neunet.2024.107103184(107103)Online publication date: Apr-2025
https://doi.org/10.1016/j.neunet.2024.107103
Yang YChen JXiang Y(2025)A review on the reliability of knowledge graph: from a knowledge representation learning perspectiveWorld Wide Web10.1007/s11280-024-01316-w28:1Online publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1007/s11280-024-01316-w
Islam MLiu F(2024)Domain-Specific Dictionary between Human and Machine LanguagesInformation10.3390/info1503014415:3(144)Online publication date: 5-Mar-2024
https://doi.org/10.3390/info15030144
Cao JFang JMeng ZLiang S(2024)Knowledge Graph Embedding: A Survey from the Perspective of Representation SpacesACM Computing Surveys10.1145/364380656:6(1-42)Online publication date: 13-Mar-2024
https://dl.acm.org/doi/10.1145/3643806
Zhang WXu YYe PHuang ZXu ZChen JPan JChen H(2024)Start From Zero: Triple Set Prediction for Automatic Knowledge Graph CompletionIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.339983236:11(7087-7101)Online publication date: Nov-2024
https://doi.org/10.1109/TKDE.2024.3399832
Zheng MLiu ZChen BHu Z(2024)PFedSKD: Personalized Federated Learning via Self-Knowledge Distillation2024 IEEE International Symposium on Parallel and Distributed Processing with Applications (ISPA)10.1109/ISPA63168.2024.00216(1591-1598)Online publication date: 30-Oct-2024
https://doi.org/10.1109/ISPA63168.2024.00216
Xin HChen L(2024)KartGPS: Knowledge Base Update with Temporal Graph Pattern-based Semantic Rules2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00105(5075-5087)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00105
Che FTao J(2024) M ixKG: Mixing for harder negative samples in knowledge graph Neural Networks10.1016/j.neunet.2024.106358177(106358)Online publication date: Sep-2024
https://doi.org/10.1016/j.neunet.2024.106358
Zhang DLiu JLiu DLi G(2023)HPREJournal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology10.3233/JIFS-23098245:4(5907-5926)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.3233/JIFS-230982
Yu MZhang QYu JZhao MLi XJin DYang MYu R(2023)Knowledge graph completion using topological correlation and multi-perspective independenceKnowledge-Based Systems10.1016/j.knosys.2022.110031259:COnline publication date: 10-Jan-2023
https://dl.acm.org/doi/10.1016/j.knosys.2022.110031
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten