skip to main content
10.1145/3477495.3531996acmconferencesArticle/Chapter ViewAbstractPublication PagesirConference Proceedingsconference-collections
research-article

Incorporating Context Graph with Logical Reasoning for Inductive Relation Prediction

Published: 07 July 2022 Publication History

Abstract

Relation prediction on knowledge graphs (KGs) aims to infer missing valid triples from observed ones. Although this task has been deeply studied, most previous studies are limited to the transductive setting and cannot handle emerging entities. Actually, the inductive setting is closer to real-life scenarios because it allows entities in the testing phase to be unseen during training. However, it is challenging to precisely conduct inductive relation prediction as there exists requirements of entity-independent relation modeling and discrete logical reasoning for interoperability. To this end, we propose a novel model ConGLR to incorporate context graph with logical reasoning. Firstly, the enclosing subgraph w.r.t. target head and tail entities are extracted and initialized by the double radius labeling. And then the context graph involving relational paths, relations and entities is introduced. Secondly, two graph convolutional networks (GCNs) with the information interaction of entities and relations are carried out to process the subgraph and context graph respectively. Considering the influence of different edges and target relations, we introduce edge-aware and relation-aware attention mechanisms for the subgraph GCN. Finally, by treating the relational path as rule body and target relation as rule head, we integrate neural calculating and logical reasoning to obtain inductive scores. And to focus on the specific modeling goals of each module, the stop-gradient is utilized in the information interaction between context graph and subgraph GCNs in the training process. In this way, ConGLR satisfies two inductive requirements at the same time. Extensive experiments demonstrate that ConGLR obtains outstanding performance against state-of-the-art baselines on twelve inductive dataset versions of three common KGs.

Supplementary Material

MP4 File (SIGIR22-modfp0461.mp4)
In this viedo, we talk about the model ConGLR that incorporates context graph with logical reasoning for inductive relation prediction. ConGLR satisfies both modeling level and reasoning level requirements of inductive relation prediction by introducing the context graph.

References

[1]
Ibrahim Abdelaziz, Srinivas Ravishankar, Pavan Kapanipathi, Salim Roukos, and Alexander G. Gray. 2021. A Semantic Parsing and Reasoning-Based Approach to Knowledge Base Question Answering. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI). 15985--15987.
[2]
Sören Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary G. Ives. 2007. DBpedia: A Nucleus for a Web of Open Data. In International Semantic Web Conference (ISWC) and Asian Semantic Web Conference (ASWC), Vol. 4825. 722--735.
[3]
Jinheon Baek, Dong Bok Lee, and Sung Ju Hwang. 2020. Learning to Extrapolate Knowledge: Transductive Few-shot Out-of-Graph Link Prediction. In Advances in Neural Information Processing Systems (NeurIPS).
[4]
Andrea Banino, Adrià Puigdomè nech Badia, Jacob C. Walker, Tim Scholtes, Jovana Mitrovic, and Charles Blundell. 2021. CoBERL: Contrastive BERT for Reinforcement Learning. CoRR, Vol. abs/2107.05431 (2021).
[5]
Kurt D. Bollacker, Colin Evans, Praveen K. Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: A Collaboratively Created Graph Database for Structuring Human Knowledge. In Proceedings of the International Conference on Management of Data (SIGMOD). 1247--1250.
[6]
Antoine Bordes, Nicolas Usunier, Alberto García-Durán, Jason Weston, and Oksana Yakhnenko. 2013. Translating Embeddings for Modeling Multi-relational Data. In Advances in Neural Information Processing Systems (NeurIPS). 2787--2795.
[7]
Jiajun Chen, Huarui He, Feng Wu, and Jie Wang. 2021. Topology-Aware Correlations Between Relations for Inductive Link Prediction in Knowledge Graphs. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI). 6271--6278.
[8]
Xinlei Chen and Kaiming He. 2021. Exploring Simple Siamese Representation Learning. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Computer Vision Foundation / IEEE, 15750--15758.
[9]
Kyunghyun Cho, Bart van Merrienboer, Çaglar Gülçehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning PRepresentations using RNN Encoder-Decoder for Statistical Machine Translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). 1724--1734.
[10]
Junyoung Chung, Çaglar Gülçehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling. CoRR, Vol. abs/1412.3555 (2014).
[11]
William W. Cohen. 2016. TensorLog: A Differentiable Deductive Database. CoRR, Vol. abs/1605.06523 (2016).
[12]
Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. 2018. Convolutional 2D Knowledge Graph Embeddings. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), the innovative Applications of Artificial Intelligence (IAAI) and the AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI). 1811--1818.
[13]
Herbert B Enderton. 2001. A Mathematical Introduction to Logic. Elsevier.
[14]
Luis Galárraga, Christina Teflioudi, Katja Hose, and Fabian M. Suchanek. 2015. Fast Rule Mining in Ontological Knowledge Bases with AMIE. The VLDB Journal, Vol. 24, 6 (2015), 707--730.
[15]
Luis Antonio Galárraga, Christina Teflioudi, Katja Hose, and Fabian M. Suchanek. 2013. AMIE: Association Rule Mining under Incomplete Evidence in Ontological Knowledge Bases. In The World Wide Web Conference (WWW). 413--422.
[16]
Zikun Hu, Yixin Cao, Lifu Huang, and Tat-Seng Chua. 2021. How Knowledge Graph and Attention Help? A Qualitative Analysis into Bag-level Relation Extraction. In Proceedings of the Annual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (ACL/IJCNLP). 4662--4671.
[17]
Longlong Jing and Yingli Tian. 2021. Self-Supervised Visual Feature Learning With Deep Neural Networks: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), Vol. 43, 11 (2021), 4037--4058.
[18]
Pei Ke, Haozhe Ji, Yu Ran, Xin Cui, Liwei Wang, Linfeng Song, Xiaoyan Zhu, and Minlie Huang. 2021. JointGT: Graph-Text Joint Representation Learning for Text Generation from Knowledge Graphs. In Proceedings of the Annual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (ACL/IJCNLP) (Findings of ACL). 2526--2538.
[19]
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In International Conference on Learning Representations (ICLR).
[20]
Shuang Liang, Jie Shao, Dongyang Zhang, Jiasheng Zhang, and Bin Cui. 2021. DRGI: Deep Relational Graph Infomax for Knowledge Graph Completion. IEEE Transactions on Knowledge and Data Engineering (TKDE) (2021).
[21]
Qika Lin, Jun Liu, Yudai Pan, Lingling Zhang, Xin Hu, and Jie Ma. 2021 a. Rule-enhanced Iterative Complementation for Knowledge Graph Reasoning. Information Sciences, Vol. 575 (2021), 66--79.
[22]
Qika Lin, Jun Liu, Lingling Zhang, Yudai Pan, Xin Hu, Fangzhi Xu, and Hongwei Zeng. 2021 b. Contrastive Graph Representations for Logical Formulas Embedding. IEEE Transactions on Knowledge and Data Engineering (2021).
[23]
Justin Lovelace, Denis Newman-Griffis, Shikhar Vashishth, Jill Fain Lehman, and Carolyn P. Rosé. 2021. Robust Knowledge Graph Completion with Stacked Convolutions and a Student Re-Ranking Network. In Proceedings of the Annual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (ACL/IJCNLP). 1016--1029.
[24]
Sijie Mai, Shuangjia Zheng, Yuedong Yang, and Haifeng Hu. 2021. Communicative Message Passing for Inductive Relation Reasoning. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI). 4294--4302.
[25]
Christian Meilicke, Manuel Fink, Yanjie Wang, Daniel Ruffinelli, Rainer Gemulla, and Heiner Stuckenschmidt. 2018. Fine-Grained Evaluation of Rule- and Embedding-Based Systems for Knowledge Graph Completion. In Proceedings of the International Semantic Web Conference (ISWC), Vol. 11136. Springer, 3--20.
[26]
Radford M. Neal and Geoffrey E. Hinton. 1998. A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants. In Learning in Graphical Models. NATO ASI Series, Vol. 89. Springer Netherlands, 355--368.
[27]
Pouya Ghiasnezhad Omran, Kewen Wang, and Zhe Wang. 2018. Scalable Rule Learning via Learning Representation. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI). 2149--2155.
[28]
Yudai Pan, Jun Liu, Lingling Zhang, Xin Hu, Tianzhe Zhao, and Qika Lin. 2021. Learning First-Order Rules with Relational Path Contrast for Inductive Relation Reasoning. CoRR, Vol. abs/2110.08810 (2021).
[29]
Heiko Paulheim. 2018. How much is a Triple? Estimating the Cost of Knowledge Graph Creation. In Proceedings of the International Semantic Web Conference (ISWC) (CEUR Workshop Proceedings, Vol. 2180).
[30]
David Poole. 1993. Probabilistic Horn Abduction and Bayesian Networks. Artificial Intelligence, Vol. 64, 1 (1993), 81--129.
[31]
Meng Qu, Junkun Chen, Louis-Pascal A. C. Xhonneux, Yoshua Bengio, and Jian Tang. 2021. RNNLogic: Learning Logic Rules for Reasoning on Knowledge Graphs. In International Conference on Learning Representations (ICLR).
[32]
Meng Qu and Jian Tang. 2019. Probabilistic Logic Neural Networks for Reasoning. In Advances in Neural Information Processing Systems (NeurIPS). 7710--7720.
[33]
Leonardo F. R. Ribeiro, Yue Zhang, Claire Gardent, and Iryna Gurevych. 2020. Modeling Global and Local Node Contexts for Text Generation from Knowledge Graphs. Transactions of the Association for Computational Linguistics (TACL), Vol. 8 (2020), 589--604.
[34]
Matthew Richardson and Pedro Domingos. 2006. Markov Logic Networks. Machine learning, Vol. 62, 1--2 (2006), 107--136.
[35]
Ali Sadeghian, Mohammadreza Armandpour, Patrick Ding, and Daisy Zhe Wang. 2019. DRUM: End-To-End Differentiable Rule Mining On Knowledge Graphs. In Advances in Neural Information Processing Systems (NeurIPS). 15321--15331.
[36]
Michael Sejr Schlichtkrull, Thomas N. Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, and Max Welling. 2018. Modeling Relational Data with Graph Convolutional Networks. In European Semantic Web Conference (ESWC). 593--607.
[37]
David A Schum. 2001. The Evidential Foundations of Probabilistic Reasoning. Northwestern University Press.
[38]
Chao Shang, Yun Tang, Jing Huang, Jinbo Bi, Xiaodong He, and Bowen Zhou. 2019. End-to-End Structure-Aware Convolutional Networks for Knowledge Base Completion. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI). 3060--3067.
[39]
Baoxu Shi and Tim Weninger. 2018. Open-World Knowledge Graph Completion. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI). 1957--1964.
[40]
Laurent Sikló ssy, A. Rich, and Vesko Marinov. 1973. Breadth-First Search: Some Surprising Results. Artificial Intelligence, Vol. 4, 1 (1973), 1--27.
[41]
Ying Song, Shuangjia Zheng, Zhangming Niu, Zhang-Hua Fu, Yutong Lu, and Yuedong Yang. 2020. Communicative Representation Learning on Attributed Molecular Graphs. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI). 2831--2838.
[42]
Fabian M. Suchanek, Gjergji Kasneci, and Gerhard Weikum. 2007. Yago: A Core of Semantic Knowledge. In Proceedings of the International Conference on World Wide Web (WWW). 697--706.
[43]
Komal K. Teru, Etienne Denis, and Will Hamilton. 2020. Inductive Relation Prediction by Subgraph Reasoning. In International Conference on Machine Learning (ICML). 9448--9457.
[44]
Kristina Toutanova, Danqi Chen, Patrick Pantel, Hoifung Poon, Pallavi Choudhury, and Michael Gamon. 2015. Representing Text for Joint Embedding of Text and Knowledge Bases. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP). 1499--1509.
[45]
Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. 2016. Complex Embeddings for Simple Link Prediction. In International Conference on Machine Learning (ICML), Vol. 48. 2071--2080.
[46]
Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing Data Using t-SNE. Journal of Machine Learning Research (JMLR), Vol. 9, 11 (2008).
[47]
L. Vivek Harsha Vardhan, Guo Jia, and Stanley Kok. 2020. Probabilistic Logic Graph Attention Networks for Reasoning. In The World Wide Web Conference (WWW). 669--673.
[48]
Shikhar Vashishth, Soumya Sanyal, Vikram Nitin, and Partha P. Talukdar. 2020. Composition-based Multi-Relational Graph Convolutional Networks. In International Conference on Learning Representations (ICLR).
[49]
Severine Verlinden, Klim Zaporojets, Johannes Deleu, Thomas Demeester, and Chris Develder. 2021. Injecting Knowledge Base Information into End-to-End Joint Entity and Relation Extraction and Coreference Resolution. In Proceedings of the Annual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (ACL/IJCNLP) (Findings of ACL ). 1952--1957.
[50]
Denny Vrandecic. 2012. Wikidata: A New Platform for Collaborative Data Collection. In Proceedings of the 21st World Wide Web Conference (WWW). 1063--1064.
[51]
Hongwei Wang, Miao Zhao, Xing Xie, Wenjie Li, and Minyi Guo. 2019 b. Knowledge Graph Convolutional Networks for Recommender Systems. In The World Wide Web Conference (WWW). ACM, 3307--3313.
[52]
Minjie Wang, Lingfan Yu, Da Zheng, Quan Gan, Yu Gai, Zihao Ye, Mufei Li, Jinjing Zhou, Qi Huang, Chao Ma, Ziyue Huang, Qipeng Guo, Hao Zhang, Haibin Lin, Junbo Zhao, Jinyang Li, Alexander J. Smola, and Zheng Zhang. 2019 a. Deep Graph Library: Towards Efficient and Scalable Deep Learning on Graphs. CoRR, Vol. abs/1909.01315 (2019).
[53]
Xiang Wang, Tinglin Huang, Dingxian Wang, Yancheng Yuan, Zhenguang Liu, Xiangnan He, and Tat-Seng Chua. 2021. Learning Intents behind Interactions with Knowledge Graph for Recommendation. In The World Wide Web Conference (WWW). 878--887.
[54]
Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014. Knowledge Graph Embedding by Translating on Hyperplanes. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI). 1112--1119.
[55]
Wenhan Xiong, Thien Hoang, and William Yang Wang. 2017. DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP). 564--573.
[56]
Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. 2015. Embedding Entities and Relations for Learning and Inference in Knowledge Bases. In International Conference on Learning Representations (ICLR).
[57]
Fan Yang, Zhilin Yang, and William W. Cohen. 2017. Differentiable Learning of Logical Rules for Knowledge Base Reasoning. In Advances in Neural Information Processing Systems (NeurIPS). 2319--2328.
[58]
Rui Ye, Xin Li, Yujie Fang, Hongyu Zang, and Mingzhong Wang. 2019. A Vectorized Relational Graph Convolutional Network for Multi-Relational Network Alignment. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI). 4135--4141.
[59]
Muhan Zhang and Yixin Chen. 2018. Link Prediction Based on Graph Neural Networks. In Advances in Neural Information Processing Systems (NeurIPS). 5171--5181.
[60]
Yuyu Zhang, Xinshi Chen, Yuan Yang, Arun Ramamurthy, Bo Li, Yuan Qi, and Le Song. 2020. Efficient Probabilistic Logic Reasoning with Graph Neural Networks. In International Conference on Learning Representations (ICLR).
[61]
Yuyu Zhang, Hanjun Dai, Zornitsa Kozareva, Alexander J. Smola, and Le Song. 2018. Variational Reasoning for Question Answering With Knowledge Graph. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI). 6069--6076.
[62]
Yifan Zhu, Qika Lin, Hao Lu, Kaize Shi, Donglei Liu, James Chambua, Shanshan Wan, and Zhendong Niu. 2021. Recommending Learning Objects through Attentive Heterogeneous Graph Convolution and Operation-Aware Neural Network. IEEE Transactions on Knowledge and Data Engineering (2021).

Cited By

View all

Index Terms

  1. Incorporating Context Graph with Logical Reasoning for Inductive Relation Prediction

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      SIGIR '22: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval
      July 2022
      3569 pages
      ISBN:9781450387323
      DOI:10.1145/3477495
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 07 July 2022

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. context graph
      2. first-order logic
      3. inductive relation prediction
      4. knowledge graph
      5. logical reasoning

      Qualifiers

      • Research-article

      Funding Sources

      Conference

      SIGIR '22
      Sponsor:

      Acceptance Rates

      Overall Acceptance Rate 792 of 3,983 submissions, 20%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)293
      • Downloads (Last 6 weeks)13
      Reflects downloads up to 28 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2025)A rule- and query-guided reinforcement learning for extrapolation reasoning in temporal knowledge graphsNeural Networks10.1016/j.neunet.2025.107186185(107186)Online publication date: May-2025
      • (2025)Uncertainty modeling for inductive knowledge graph embeddingNeural Networks10.1016/j.neunet.2024.107103184(107103)Online publication date: Apr-2025
      • (2025)Inductive link prediction via global relational semantic learningInformation Systems10.1016/j.is.2024.102514130(102514)Online publication date: Apr-2025
      • (2024)Commonsense-Guided Inductive Relation Prediction with Dual Attention MechanismApplied Sciences10.3390/app1405204414:5(2044)Online publication date: 29-Feb-2024
      • (2024)Logical Reasoning with Relation Network for Inductive Knowledge Graph CompletionProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671911(4268-4277)Online publication date: 25-Aug-2024
      • (2024)Inductive Knowledge Graph Embedding via Exploring Interaction Patterns of RelationsProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679667(1722-1731)Online publication date: 21-Oct-2024
      • (2024)Untargeted Adversarial Attack on Knowledge Graph EmbeddingsProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657702(1701-1711)Online publication date: 10-Jul-2024
      • (2024)A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multi-ModalIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.341745146:12(9456-9478)Online publication date: Dec-2024
      • (2024)One Subgraph for All: Efficient Reasoning on Opening Subgraphs for Inductive Knowledge Graph CompletionIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.343276736:12(8914-8927)Online publication date: Dec-2024
      • (2024)A New Perspective for Computational Social Systems: Fuzzy Modeling and Reasoning for Social Computing in CPSSIEEE Transactions on Computational Social Systems10.1109/TCSS.2022.319742111:1(101-116)Online publication date: Feb-2024
      • Show More Cited By

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media