research-article

Beyond Triplets: Hyper-Relational Knowledge Graph Embedding for Link Prediction

Authors:

Philippe Cudré-MaurouxAuthors Info & Claims

WWW '20: Proceedings of The Web Conference 2020

Pages 1885 - 1896

https://doi.org/10.1145/3366423.3380257

Published: 20 April 2020 Publication History

Abstract

Knowledge Graph (KG) embeddings are a powerful tool for predicting missing links in KGs. Existing techniques typically represent a KG as a set of triplets, where each triplet (h, r, t) links two entities h and t through a relation r, and learn entity/relation embeddings from such triplets while preserving such a structure. However, this triplet representation oversimplifies the complex nature of the data stored in the KG, in particular for hyper-relational facts, where each fact contains not only a base triplet (h, r, t), but also the associated key-value pairs (k, v). Even though a few recent techniques tried to learn from such data by transforming a hyper-relational fact into an n-ary representation (i.e., a set of key-value pairs only without triplets), they result in suboptimal models as they are unaware of the triplet structure, which serves as the fundamental data structure in modern KGs and preserves the essential information for link prediction. To address this issue, we propose HINGE, a hyper-relational KG embedding model, which directly learns from hyper-relational facts in a KG. HINGE captures not only the primary structural information of the KG encoded in the triplets, but also the correlation between each triplet and its associated key-value pairs. Our extensive evaluation shows the superiority of HINGE on various link prediction tasks over KGs. In particular, HINGE consistently outperforms not only the KG embedding methods learning from triplets only (by 0.81-41.45% depending on the link prediction tasks and settings), but also the methods learning from hyper-relational facts using the n-ary representation (by 13.2-84.1%).

References

[1]

Ivana Balazevic, Carl Allen, and Timothy M Hospedales. 2018. Hypernetwork Knowledge Graph Embeddings. arXiv preprint arXiv:1808.07018(2018).

[2]

Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: a collaboratively created graph database for structuring human knowledge. In ACM SIGMOD/PODS. ACM, 1247–1250.

[3]

Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In NIPS. 2787–2795.

[4]

Dan Brickley, Ramanathan V Guha, and Brian McBride. 2014. RDF Schema 1.1. W3C recommendation 25(2014), 2004–2014.

[5]

Hongyun Cai, Vincent W Zheng, and Kevin Chen-Chuan Chang. 2018. A comprehensive survey of graph embedding: Problems, techniques, and applications. IEEE Transactions on Knowledge and Data Engineering 30, 9(2018), 1616–1637.

Digital Library

[6]

Jianpeng Cheng, Zhongyuan Wang, Ji-Rong Wen, Jun Yan, and Zheng Chen. 2015. Contextual text understanding in distributional semantic space. In CIKM. ACM, 133–142.

[7]

Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. 2017. Convolutional 2d knowledge graph embeddings. In AAAI. 1811–1818.

[8]

John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research 12, Jul (2011), 2121–2159.

Digital Library

[9]

Takuma Ebisu and Ryutaro Ichise. 2018. Toruse: Knowledge graph embedding on a lie group. In AAAI.

[10]

Wei Fang, Jianwen Zhang, Dilin Wang, Zheng Chen, and Ming Li. 2016. Entity disambiguation by knowledge and text jointly embedding. In CoNLL. 260–269.

[11]

Alberto Garcia-Duran and Mathias Niepert. 2017. Kblrn: End-to-end learning of knowledge base representations with latent, relational, and numerical features. arXiv preprint arXiv:1709.04676(2017).

[12]

[12] Google.2014. https://www.google.com/intl/bn/insidesearch/features/search/knowledge.html.

[13]

Jens Graupmann, Ralf Schenkel, and Gerhard Weikum. 2005. The SphereSearch engine for unified ranked retrieval of heterogeneous XML and web documents. In VLDB. VLDB Endowment, 529–540.

[14]

Saiping Guan, Xiaolong Jin, Yuanzhuo Wang, and Xueqi Cheng. 2019. Link Prediction on N-ary Relational Data. In The World Wide Web Conference. ACM, 583–593.

[15]

Xu Han, Zhiyuan Liu, and Maosong Sun. 2018. Neural knowledge acquisition via mutual attention between knowledge graph and text. In AAAI.

[16]

Rana Hussein, Dingqi Yang, and Philippe Cudré-Mauroux. 2018. Are Meta-Paths Necessary? Revisiting Heterogeneous Graph Embeddings. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management. 437–446.

[17]

Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167(2015).

[18]

Guoliang Ji, Shizhu He, Liheng Xu, Kang Liu, and Jun Zhao. 2015. Knowledge graph embedding via dynamic mapping matrix. In ACL and IJCNLP, Vol. 1. 687–696.

[19]

Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980(2014).

[20]

Agustinus Kristiadi, Mohammad Asif Khan, Denis Lukovnikov, Jens Lehmann, and Asja Fischer. 2018. Incorporating literals into knowledge graph embeddings. arXiv preprint arXiv:1802.00934(2018).

[21]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems. 1097–1105.

[22]

Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. 2015. Learning entity and relation embeddings for knowledge graph completion. In AAAI, Vol. 15. 2181–2187.

[23]

Hanxiao Liu, Yuexin Wu, and Yiming Yang. 2017. Analogical inference for multi-relational embeddings. In Proceedings of the 34th International Conference on Machine Learning-Volume 70. 2168–2178.

Digital Library

[24]

Ye Liu, Hui Li, Alberto Garcia-Duran, Mathias Niepert, Daniel Onoro-Rubio, and David S Rosenblum. 2019. MMKG: Multi-modal Knowledge Graphs. In European Semantic Web Conference. Springer, 459–474.

[25]

Dai Quoc Nguyen, Thanh Vu, Tu Dinh Nguyen, Dat Quoc Nguyen, and Dinh Phung. 2018. A Capsule Network-based Embedding Model for Knowledge Graph Completion and Search Personalization. arXiv preprint arXiv:1808.04122(2018).

[26]

Maximilian Nickel, Kevin Murphy, Volker Tresp, and Evgeniy Gabrilovich. 2015. A review of relational machine learning for knowledge graphs. Proc. IEEE 104, 1 (2015), 11–33.

[27]

Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. 2011. A Three-Way Model for Collective Learning on Multi-Relational Data. In ICML, Vol. 11. 809–816.

[28]

Paolo Rosso, Dingqi Yang, and Philippe Cudré-Mauroux. 2019. Knowledge Graph Embeddings. In Encyclopedia of Big Data Technologies.https://doi.org/10.1007/978-3-319-63962-8_284-1

[29]

Paolo Rosso, Dingqi Yang, and Philippe Cudre-Mauroux. 2019. Revisiting Text and Knowledge Graph Joint Embeddings: The Amount of Shared Information Matters!. In Proceedings of the 2018 IEEE International Conference on Big Data (Big Data).

[30]

Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. 2018. Modeling relational data with graph convolutional networks. In European Semantic Web Conference. Springer, 593–607.

Digital Library

[31]

Richard Socher, Danqi Chen, Christopher D Manning, and Andrew Ng. 2013. Reasoning with neural tensor networks for knowledge base completion. In NIPS. 926–934.

[32]

Yi Tay, Luu Anh Tuan, Minh C Phan, and Siu Cheung Hui. 2017. Multi-Task Neural Network for Non-discrete Attribute Prediction in Knowledge Graphs. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. ACM, 1029–1038.

Digital Library

[33]

Kristina Toutanova, Danqi Chen, Patrick Pantel, Hoifung Poon, Pallavi Choudhury, and Michael Gamon. 2015. Representing text for joint embedding of text and knowledge bases. In EMNLP. 1499–1509.

[34]

Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. 2016. Complex embeddings for simple link prediction. In ICML. 2071–2080.

[35]

Quan Wang, Zhendong Mao, Bin Wang, and Li Guo. 2017. Knowledge graph embedding: A survey of approaches and applications. TKDE 29, 12 (2017), 2724–2743.

[36]

Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014. Knowledge graph and text jointly embedding. In EMNLP. 1591–1601.

[37]

Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014. Knowledge Graph Embedding by Translating on Hyperplanes. In AAAI, Vol. 14. 1112–1119.

[38]

Jianfeng Wen, Jianxin Li, Yongyi Mao, Shini Chen, and Richong Zhang. 2016. On the representation and embedding of knowledge bases beyond binary relations. In IJCAI. AAAI Press, 1300–1307.

[39]

Robert West, Evgeniy Gabrilovich, Kevin Murphy, Shaohua Sun, Rahul Gupta, and Dekang Lin. 2014. Knowledge base completion via search-based question answering. In Proceedings of the 23rd international conference on World wide web. ACM, 515–526.

Digital Library

[40]

[40] Wikidata.2012. http://wikidata.org/.

[41]

Chenyan Xiong, Russell Power, and Jamie Callan. 2017. Explicit semantic ranking for academic search via knowledge graph embedding. In Proceedings of the 26th international conference on world wide web. International World Wide Web Conferences Steering Committee, 1271–1279.

Digital Library

[42]

Ikuya Yamada, Hiroyuki Shindo, Hideaki Takeda, and Yoshiyasu Takefuji. 2016. Joint learning of the embedding of words and entities for named entity disambiguation. In CoNLL. 250–259.

[43]

Ikuya Yamada, Hiroyuki Shindo, Hideaki Takeda, and Yoshiyasu Takefuji. 2017. Learning Distributed Representations of Texts and Entities from Knowledge Base. TACL 5(2017), 397–411. https://www.transacl.org/ojs/index.php/tacl/article/view/1065

[44]

Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. 2015. Embedding entities and relations for learning and inference in knowledge bases. In ICLR.

[45]

Dingqi Yang, Paolo Rosso, Bin Li, and Philippe Cudre-Mauroux. 2019. NodeSketch: Highly-Efficient Graph Embeddings via Recursive Sketching. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1162–1172.

Digital Library

[46]

Scott Wen-tau Yih, Ming-Wei Chang, Xiaodong He, and Jianfeng Gao. 2015. Semantic parsing via staged query graph generation: Question answering with knowledge base. In ACL and IJCNLP. 1321–1331.

[47]

Mo Yu and Mark Dredze. 2014. Improving lexical embeddings with semantic knowledge. In ACL, Vol. 2. 545–550.

[48]

Fuzheng Zhang, Nicholas Jing Yuan, Defu Lian, Xing Xie, and Wei-Ying Ma. 2016. Collaborative knowledge base embedding for recommender systems. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. ACM, 353–362.

Digital Library

[49]

Richong Zhang, Junpeng Li, Jiajie Mei, and Yongyi Mao. 2018. Scalable instance reconstruction in knowledge bases via relatedness affiliated embedding. In Proceedings of the 2018 World Wide Web Conference. International World Wide Web Conferences Steering Committee, 1185–1194.

Digital Library

[50]

Huaping Zhong, Jianwen Zhang, Zhen Wang, Hai Wan, and Zheng Chen. 2015. Aligning knowledge and text embeddings by entity descriptions. In EMNLP. 267–272.

Cited By

Zang YRen LWu JXiao YHu R(2025)Power on graph: Mining power relationship via user interaction correlationExpert Systems with Applications10.1016/j.eswa.2024.126348270(126348)Online publication date: Apr-2025
https://doi.org/10.1016/j.eswa.2024.126348
Chi HLu YXie CKe WChen B(2025)Spatio-temporal attention based collaborative local–global learning for traffic flow predictionEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.109575139:PBOnline publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1016/j.engappai.2024.109575
Yang YChen JXiang Y(2025)A review on the reliability of knowledge graph: from a knowledge representation learning perspectiveWorld Wide Web10.1007/s11280-024-01316-w28:1Online publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1007/s11280-024-01316-w
Show More Cited By

Index Terms

Beyond Triplets: Hyper-Relational Knowledge Graph Embedding for Link Prediction
1. Computing methodologies
  1. Machine learning
2. Information systems
  1. Information systems applications

Index terms have been assigned to the content through auto-classification.

Recommendations

Embedding based Link Prediction for Knowledge Graph Completion
CIKM '20: Proceedings of the 29th ACM International Conference on Information & Knowledge Management

Knowledge Graphs (KGs) have recently gained attention for representing knowledge about a particular domain. Since its advent, the Linked Open Data (LOD) cloud has constantly been growing containing many KGs about many different domains such as ...
Hierarchical-aware relation rotational knowledge graph embedding for link prediction
Abstract
Knowledge graph embedding, as the upstream task of link prediction which aims to predict new links between entities under the premise of known relations, its reliability greatly affects the performance of link prediction. However, previous ...
Schema-Aware Hyper-Relational Knowledge Graph Embeddings for Link Prediction
Knowledge Graph (KG) embeddings have become a powerful paradigm to resolve link prediction tasks for KG completion. The widely adopted triple-based representation, where each triplet <inline-formula><tex-math notation="LaTeX">$(h,r,t)$</tex-math><...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '20: Proceedings of The Web Conference 2020

April 2020

3143 pages

ISBN:9781450370233

DOI:10.1145/3366423

Editors:
Yennun Huang
Acadmica sinica, Taiwan
,
Irwin King
The Chinese University of Hong Kong, Hong Kong
,
Tie-Yan Liu
Microsoft Research Asia, China
,
Maarten van Steen
University of Twente, Netherlands

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 April 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

WWW '20

Sponsor:

SIGWEB

WWW '20: The Web Conference 2020

April 20 - 24, 2020

Taipei, Taiwan

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

71
Total Citations
View Citations
2,153
Total Downloads

Downloads (Last 12 months)310
Downloads (Last 6 weeks)22

Reflects downloads up to 18 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zang YRen LWu JXiao YHu R(2025)Power on graph: Mining power relationship via user interaction correlationExpert Systems with Applications10.1016/j.eswa.2024.126348270(126348)Online publication date: Apr-2025
https://doi.org/10.1016/j.eswa.2024.126348
Chi HLu YXie CKe WChen B(2025)Spatio-temporal attention based collaborative local–global learning for traffic flow predictionEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.109575139:PBOnline publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1016/j.engappai.2024.109575
Yang YChen JXiang Y(2025)A review on the reliability of knowledge graph: from a knowledge representation learning perspectiveWorld Wide Web10.1007/s11280-024-01316-w28:1Online publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1007/s11280-024-01316-w
Dang XShu XLi FDong X(2024)Research on Predicting Super-Relational Data Links for Mine Hoists Within Hyper-Relational Knowledge GraphsInformation10.3390/info1601000316:1(3)Online publication date: 25-Dec-2024
https://doi.org/10.3390/info16010003
Xiong BNayyeri MLuo LWang ZPan SStaab SWooldridge MDy JNatarajan S(2024)NestEProceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence10.1609/aaai.v38i8.28772(9205-9213)Online publication date: 20-Feb-2024
https://dl.acm.org/doi/10.1609/aaai.v38i8.28772
Wang YHe JWang HCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)RHKH: Relational Hypergraph Neural Network for Link Prediction on N-ary Knowledge HypergraphProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681706(8759-8767)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681706
Li LWang HLi JXu XWang YRen TSerra ESpezzano F(2024)Integrating Structure and Text for Enhancing Hyper-relational Knowledge Graph Representation via Structure Soft Prompt TuningProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679698(1226-1234)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679698
Kim JKim EYeo KJeon YKim CLee SLee JHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Content-based Graph Reconstruction for Cold-start Item RecommendationProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657801(1263-1273)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657801
Li ZWang CWang XChen ZLi J(2024)HJE: Joint Convolutional Representation Learning for Knowledge Hypergraph CompletionIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.336572736:8(3879-3892)Online publication date: Aug-2024
https://doi.org/10.1109/TKDE.2024.3365727
Lu YYang DWang PRosso PCudre-Mauroux P(2024)Schema-Aware Hyper-Relational Knowledge Graph Embeddings for Link PredictionIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.3323499(1-15)Online publication date: 2024
https://doi.org/10.1109/TKDE.2023.3323499
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten