research-article

Probabilistic Logic Graph Attention Networks for Reasoning

Authors:

L Vivek Harsha Vardhan,

Stanley KokAuthors Info & Claims

WWW '20: Companion Proceedings of the Web Conference 2020

Pages 669 - 673

https://doi.org/10.1145/3366424.3391265

Published: 20 April 2020 Publication History

Abstract

Knowledge base completion, which involves the prediction of missing relations between entities in a knowledge graph, has been an active area of research. Markov logic networks, which combine probabilistic graphical models and first order logic, have proven to be effective on knowledge graph tasks like link prediction and question answering. However, their intractable inference limits their scalability and wider applicability across various tasks. In recent times, graph attention neural networks, which capture features of neighbouring entities, have achieved superior results on highly complex graph problems like node classification and link prediction. Combining the best of both worlds, we propose Probabilistic Logic Graph Attention Network (pGAT) for reasoning. In the proposed model, the joint distribution of all possible triplets defined by a Markov logic network is optimized with a variational EM algorithm. This helps us to efficiently combine first-order logic and graph attention networks. With the goal of establishing strong baselines for future research on link prediction, we evaluate our model on various standard link prediction benchmarks, and obtain competitive results.

References

[1]

Jonathan Berant, Ido Dagan, and Jacob Goldberger. 2010. Global Learning of Focused Entailment Graphs. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Uppsala, Sweden, 1220–1229. https://www.aclweb.org/anthology/P10-1124

Digital Library

[2]

Jonathan Berant and Percy Liang. 2014. Semantic Parsing via Paraphrasing. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Baltimore, Maryland, 1415–1425. https://doi.org/10.3115/v1/P14-1133

[3]

Julian Besag. 1975. Statistical Analysis of Non-Lattice Data. Journal of the Royal Statistical Society. Series D (The Statistician) 24, 3(1975), pp. 179–195.

[4]

Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: A Collaboratively Created Graph Database for Structuring Human Knowledge(SIGMOD ’08). ACM, New York, NY, USA, 1247–1250. https://doi.org/10.1145/1376616.1376746

Digital Library

[5]

Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating Embeddings for Modeling Multi-relational Data. In Advances in Neural Information Processing Systems 26, C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger(Eds.). Curran Associates, Inc., 2787–2795. http://papers.nips.cc/paper/5071-translating-embeddings-for-modeling-multi-relational-data.pdf

Digital Library

[6]

Robin Burke. 2000. Knowledge-based recommender systems. In Encyclopedia of Library and Information Systems. Vol. 69. Marcel Dekker, New York, NY, USA, 180–200. http://www.cs.pitt.edu/~mrotaru/comp/rs/Burke%20ELIS%202000.pdf

[7]

Kersting K. De Raedt L.2008. Probabilistic Inductive Logic Programming.In Probabilistic Inductive Logic Programming., Kersting K. Muggleton S. De Raedt L., Frasconi P. (Ed.). Springer, Berlin, Chapter 1, 1–27.

[8]

Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. 2018. Convolutional 2D Knowledge Graph Embeddings. https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/17366

[9]

Thomas N. Kipf and Max Welling. 2016. Semi-Supervised Classification with Graph Convolutional Networks. CoRR abs/1609.02907(2016). arxiv:1609.02907http://arxiv.org/abs/1609.02907

[10]

Solomon Kullback and Richard A Leibler. 1951. On information and sufficiency. The annals of mathematical statistics 22, 1 (1951), 79–86.

[11]

Deepak Nathani, Jatin Chauhan, Charu Sharma, and Manohar Kaul. 2019. Learning Attention-based Embeddings for Relation Prediction in Knowledge Graphs. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Florence, Italy, 4710–4723. https://doi.org/10.18653/v1/P19-1466

[12]

Radford M. Neal and Geoffrey E. Hinton. 1998. A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants. Springer Netherlands, Dordrecht, 355–368. https://doi.org/10.1007/978-94-011-5014-9_12

[13]

Dai Quoc Nguyen, Tu Dinh Nguyen, Dat Quoc Nguyen, and Dinh Phung. 2018. A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network. In Proceedings of the 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT). 327–333.

[14]

Manfred Opper and David Saad. 2001. Advanced mean field methods: Theory and practice. MIT press.

[15]

Meng Qu, Yoshua Bengio, and Jian Tang. 2019. GMNN: Graph Markov Neural Networks. arxiv:cs.LG/1905.06214

[16]

Meng Qu and Jian Tang. 2019. Probabilistic Logic Neural Networks for Reasoning. arxiv:cs.LG/1906.08495

[17]

Matthew Richardson and Pedro Domingos. 2006. Markov Logic Networks. Mach. Learn. 62, 1-2 (Feb. 2006), 107–136. https://doi.org/10.1007/s10994-006-5833-1

Digital Library

[18]

Matthew Richardson and Pedro Domingos. 2006. Markov logic networks. Machine Learning 62, 1 (01 Feb 2006), 107–136. https://doi.org/10.1007/s10994-006-5833-1

[19]

Michael Schlichtkrull, Thomas N. Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, and Max Welling. 2017. Modeling Relational Data with Graph Convolutional Networks. arxiv:stat.ML/1703.06103

[20]

Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. 2019. RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space. In International Conference on Learning Representations. https://openreview.net/forum?id=HkgEQnRqYQ

[21]

Ben Taskar, Pieter Abbeel, and Daphne Koller. 2012. Discriminative Probabilistic Models for Relational Data. arxiv:cs.LG/1301.0604

[22]

Komal K. Teru and William L. Hamilton. 2019. Inductive Relation Prediction on Knowledge Graphs. arxiv:cs.LG/1911.06962

[23]

Kristina Toutanova and Danqi Chen. 2015. Observed versus latent features for knowledge base and text inference. In Proceedings of the 3rd Workshop on Continuous Vector Space Models and their Compositionality. Association for Computational Linguistics, Beijing, China, 57–66. https://doi.org/10.18653/v1/W15-4007

[24]

Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. 2016. Complex embeddings for simple link prediction. In International Conference on Machine Learning (ICML), Vol. 48. 2071–2080.

[25]

Xiaobo Wang, Shifeng Zhang, Zhen Lei, Si Liu, Xiaojie Guo, and Stan Z. Li. 2018. Ensemble Soft-Margin Softmax Loss for Image Classification. arxiv:cs.CV/1805.03922

[26]

Michael P. Wellman, John S. Breese, and Robert P. Goldman. 1992. From knowledge bases to decision models. The Knowledge Engineering Review 7, 1 (1992), 35–53. https://doi.org/10.1017/S0269888900006147

[27]

Bishan Yang, Scott Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. 2015. Embedding Entities and Relations for Learning and Inference in Knowledge Bases. In Proceedings of the International Conference on Learning Representations (ICLR) 2015(proceedings of the international conference on learning representations (iclr) 2015 ed.). https://www.microsoft.com/en-us/research/publication/embedding-entities-and-relations-for-learning-and-inference-in-knowledge-bases/

[28]

Xuchen Yao and Benjamin Van Durme. 2014. Information Extraction over Structured Data: Question Answering with Freebase. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Baltimore, Maryland, 956–966. https://doi.org/10.3115/v1/P14-1090

Cited By

Cheng KAhmed NRossi RWillke TSun Y(2024)Neural-Symbolic Methods for Knowledge Graph Reasoning: A SurveyACM Transactions on Knowledge Discovery from Data10.1145/368680618:9(1-44)Online publication date: 12-Nov-2024
https://dl.acm.org/doi/10.1145/3686806
Liang KMeng LLiu MLiu YTu WWang SZhou SLiu XSun FHe K(2024)A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multi-ModalIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.341745146:12(9456-9478)Online publication date: Dec-2024
https://doi.org/10.1109/TPAMI.2024.3417451
Li AYang BHuo HHussain FXu G(2024)Structure- and Logic-Aware Heterogeneous Graph Learning for Recommendation2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00048(544-556)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00048
Show More Cited By

Index Terms

Probabilistic Logic Graph Attention Networks for Reasoning

Index terms have been assigned to the content through auto-classification.

Recommendations

AM-DGCNN: Leveraging Graph Attention Networks and Edge Attributes for Link Classification in Knowledge Graphs
SC-W '24: Proceedings of the SC '24 Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis

Graph-based representations are increasingly popular for storing and managing information through knowledge graphs, which capture entities and their relationships. However, these knowledge graphs often suffer from incomplete link information. To address ...
Automated Reasoning in Higher-order Logic: Set Comprehension and Extensionality in Church's Type Theory
On graph reasoning

In this paper, we study the (positive) graph relational calculus. The basis for this calculus was introduced by Curtis and Lowe in 1996 and some variants, motivated by their applications to semantics of programs and foundations of mathematics, appear ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '20: Companion Proceedings of the Web Conference 2020

April 2020

854 pages

ISBN:9781450370240

DOI:10.1145/3366424

Editors:
Amal El Fallah Seghrouchni
Sorbonne University, France
,
Gita Sukthankar
University of Central Florida, United States
,
Tie-Yan Liu
Microsoft Research Asia, China
,
Maarten van Steen
University of Twente, Netherlands

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 April 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

WWW '20

Sponsor:

SIGWEB

WWW '20: The Web Conference 2020

April 20 - 24, 2020

Taipei, Taiwan

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
924
Total Downloads

Downloads (Last 12 months)37
Downloads (Last 6 weeks)1

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Cheng KAhmed NRossi RWillke TSun Y(2024)Neural-Symbolic Methods for Knowledge Graph Reasoning: A SurveyACM Transactions on Knowledge Discovery from Data10.1145/368680618:9(1-44)Online publication date: 12-Nov-2024
https://dl.acm.org/doi/10.1145/3686806
Liang KMeng LLiu MLiu YTu WWang SZhou SLiu XSun FHe K(2024)A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multi-ModalIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.341745146:12(9456-9478)Online publication date: Dec-2024
https://doi.org/10.1109/TPAMI.2024.3417451
Li AYang BHuo HHussain FXu G(2024)Structure- and Logic-Aware Heterogeneous Graph Learning for Recommendation2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00048(544-556)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00048
Sheng YLi LWang YZeng D(2024)Integrating Language Models with Symbolic Formulas for First-Order Logic ReasoningICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP48485.2024.10446308(11586-11590)Online publication date: 14-Apr-2024
https://doi.org/10.1109/ICASSP48485.2024.10446308
Ma JLiu BLi KLi CZhang FLuo XQiao Y(2024)A review of graph neural networks and pretrained language models for knowledge graph reasoningNeurocomputing10.1016/j.neucom.2024.128490609:COnline publication date: 7-Dec-2024
https://dl.acm.org/doi/10.1016/j.neucom.2024.128490
Zeng DHuang TZhang ZJiang L(2024)Entity neighborhood awareness and hierarchical message aggregation for inductive relation predictionInformation Processing & Management10.1016/j.ipm.2024.10373761:4(103737)Online publication date: Jul-2024
https://doi.org/10.1016/j.ipm.2024.103737
Lu YYang STang B(2024)A Comprehensive Review of Relation Prediction Techniques in Knowledge GraphWeb and Big Data. APWeb-WAIM 2023 International Workshops10.1007/978-981-97-2991-3_2(11-24)Online publication date: 9-May-2024
https://doi.org/10.1007/978-981-97-2991-3_2
Cheng KSun YCheng KSun Y(2024)Knowledge Graph CompletionKnowledge Graph Reasoning10.1007/978-3-031-72008-6_3(23-72)Online publication date: 22-Nov-2024
https://doi.org/10.1007/978-3-031-72008-6_3
Chabin JHalfeld-Ferrari MWang L(2024)A Preliminary Investigation: Strategies for Incorporating Logical Rules Into Knowledge Graph EmbeddingsNew Trends in Database and Information Systems10.1007/978-3-031-70421-5_10(104-116)Online publication date: 14-Nov-2024
https://doi.org/10.1007/978-3-031-70421-5_10
Zeng ZCheng QSi Y(2023)Logical Rule-Based Knowledge Graph Reasoning: A Comprehensive SurveyMathematics10.3390/math1121448611:21(4486)Online publication date: 30-Oct-2023
https://doi.org/10.3390/math11214486
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten