Kai Sun
Skip Abstract Section
Abstract
In recent years, Graph Neural Networks (GNNs) have achieved unprecedented success in handling graph-structured data, thereby driving the development of numerous GNN-oriented techniques for inductive knowledge graph completion (KGC). A key limitation of existing methods, however, is their dependence on pre-defined aggregation functions, which lack the adaptability to diverse data, resulting in suboptimal performance on established benchmarks. Another challenge arises from the exponential increase in irrelated entities as the reasoning path lengthens, introducing unwarranted noise and consequently diminishing the model’s generalization capabilities. To surmount these obstacles, we design an innovative framework that synergizes Multi-Level Sampling with an Adaptive Aggregation mechanism (MLSAA). Distinctively, our model couples GNNs with enhanced set transformers, enabling dynamic selection of the most appropriate aggregation function tailored to specific datasets and tasks. This adaptability significantly boosts both the model’s flexibility and its expressive capacity. Additionally, we unveil a unique sampling strategy designed to selectively filter irrelevant entities, while retaining potentially beneficial targets throughout the reasoning process. We undertake an exhaustive evaluation of our novel inductive KGC method across three pivotal benchmark datasets and the experimental results corroborate the efficacy of MLSAA.
- [1] . 2016. Layer normalization. arXiv:1607.06450. Retrieved from https://arxiv.org/abs/1607.06450Google Scholar
- [2] . 2021. Accurate learning of graph representations with graph multiset pooling. In Proceedings of the 9th International Conference on Learning Representations.Google Scholar
- [3] . 2022. (LA)yer-neigh(BOR) sampling: Defusing neighborhood explosion in GNNs. abs/2210.13339. Retrieved from https://arxiv.org/abs/2210.13339Google Scholar
- [4] . 2018. Relational inductive biases, deep learning, and graph networks. arXiv:1806.01261. Retrieved from https://arxiv.org/abs/1806.01261Google Scholar
- [5] . 2013. Translating embeddings for modeling multi-relational data. In Proceedings of the Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. 2787–2795.Google Scholar
- [6] . 2018. FastGCN: Fast learning with graph convolutional networks via importance sampling. In Proceedings of the 6th International Conference on Learning Representations.Google Scholar
- [7] . 2022. DACHA: A dual graph convolution based temporal knowledge graph representation learning method using historical relation. ACM Transactions on Knowledge Discovery from Data 16, 3 (2022), 46:1–46:18.Google ScholarDigital Library
- [8] . 2019. Cluster-GCN: An efficient algorithm for training deep and large graph convolutional networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 257–266.Google ScholarDigital Library
- [9] . 2022. You are AllSet: A multiset function framework for hypergraph neural networks. In Proceedings of the 10th International Conference on Learning Representations..Google Scholar
- [10] . 2018. Convolutional 2D knowledge graph embeddings. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 1811–1818.Google ScholarCross Ref
- [11] . 2015. Fast rule mining in ontological knowledge bases with AMIE+. The VLDB Journal 24, 6 (2015), 707–730.Google ScholarDigital Library
- [12] . 2017. Kern: Kernel embeddings of categorical data. In Proceedings of the International Semantic Web Conference. Springer, 210–218.Google Scholar
- [13] . 2018. Conditional neural processes. In Proceedings of the 35th International Conference on Machine Learning. 1690–1699.Google Scholar
- [14] . 2017. Neural message passing for quantum chemistry. In Proceedings of the 34th International Conference on Machine Learning. 1263–1272.Google ScholarDigital Library
- [15] . 2017. Inductive representation learning on large graphs. In Proceedings of the Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017. 1024–1034.Google Scholar
- [16] . 2018. Rule learning from knowledge graphs guided by embedding models. In Proceedings of the Semantic Web - ISWC 2018-17th International Semantic Web Conference. 72–90.Google ScholarDigital Library
- [17] . 2020. Open graph benchmark: Datasets for machine learning on graphs. In Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6–12, 2020, virtual.Google Scholar
- [18] . 2018. Adaptive sampling towards fast graph representation learning. In Proceedings of the Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3–8, 2018, Montréal, Canada. 4563–4572.Google Scholar
- [19] . 2015. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7–9, 2015, Conference Track Proceedings.Google Scholar
- [20] Thomas N. Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In Proceedings of the 5th International Conference on Learning Representations.Google Scholar
- [21] . 2018. Canonical tensor decomposition for knowledge base completion. In Proceedings of the 35th International Conference on Machine Learning. 2869–2878.Google Scholar
- [22] . 2019. Set transformer: A framework for attention-based permutation-invariant neural networks. In Proceedings of the 36th International Conference on Machine Learning. 3744–3753.Google Scholar
- [23] . 2017. Discovering causal signals in images. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. 58–66.Google ScholarCross Ref
- [24] . 2018. Fine-grained evaluation of rule- and embedding-based systems for knowledge graph completion. In Proceedings of the Semantic Web - ISWC 2018-17th International Semantic Web Conference. 3–20.Google ScholarDigital Library
- [25] . 2019. Janossy pooling: Learning deep permutation-invariant functions for variable-size inputs. In Proceedings of the 7th International Conference on Learning Representations.Google Scholar
- [26] . 2019. Learning attention-based embeddings for relation prediction in knowledge graphs. In Proceedings of the 57th Conference of the Association for Computational Linguistics. 4710–4723.Google ScholarCross Ref
- [27] . 2015. Compositional vector space models for knowledge base completion. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing. 156–166.Google ScholarCross Ref
- [28] . 2017. Automatic differentiation in pytorch. In Proceedings of the Neural Information Processing Systems.Google Scholar
- [29] . 2017. PointNet: Deep learning on point sets for 3D classification and segmentation. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. 77–85.Google Scholar
- [30] . 2019. DRUM: End-to-end differentiable rule mining on knowledge graphs. In Proceedings of the Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019. 15321–15331.Google Scholar
- [31] . 2018. Modeling relational data with graph convolutional networks. In Proceedings of the Semantic Web - 15th International Conference. 593–607.Google ScholarDigital Library
- [32] . 2023. LMC: Fast training of GNNs via subgraph sampling with provable convergence. In Proceedings of the 11th International Conference on Learning Representations.Google Scholar
- [33] . 2017. Prototypical networks for few-shot learning. In Proceedings of the Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017. 4077–4087.Google Scholar
- [34] . 2013. Reasoning with neural tensor networks for knowledge base completion. In Proceedings of the Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. 926–934.Google Scholar
- [35] . 2019. RotatE: Knowledge graph embedding by relational rotation in complex space. In Proceedings of the 7th International Conference on Learning Representations.Google Scholar
- [36] . 2020. Inductive relation prediction by subgraph reasoning. In Proceedings of the 37th International Conference on Machine Learning. 9448–9457.Google ScholarDigital Library
- [37] . 2015. Observed versus latent features for knowledge base and text inference. In Proceedings of the 3rd Workshop on Continuous Vector Space Models and their Compositionality. 57–66.Google ScholarCross Ref
- [38] . 2015. Representing text for joint embedding of text and knowledge bases. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 1499–1509.Google ScholarCross Ref
- [39] . 2020. Composition-based multi-relational graph convolutional networks. In Proceedings of the 8th International Conference on Learning Representations.Google Scholar
- [40] . 2019. On the limitations of representing functions on sets. In Proceedings of the 36th International Conference on Machine Learning. 6487–6494.Google Scholar
- [41] . 2018. RippleNet: Propagating user preferences on the knowledge graph for recommender systems. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management. 417–426.Google ScholarDigital Library
- [42] . 2024. Multi-level interaction based knowledge graph completion. IEEE/ACM Transactions on Audio, Speech, and Language Processing 32 (2024), 386–396.Google ScholarDigital Library
- [43] . 2023. Multi-concept representation learning for knowledge graph completion. ACM Transactions on Knowledge Discovery from Data 17, 1 (2023), 11:1–11:19.Google ScholarDigital Library
- [44] . 2023. TDN: Triplet distributor network for knowledge graph completion. IEEE Transactions on Knowledge and Data Engineering 35, 12 (2023), 13002–13014.Google ScholarDigital Library
- [45] . 2023. A survey on temporal knowledge graph completion: Taxonomy, progress, and prospects. arXiv:2308.02457. Retrieved from https://arxiv.org/abs/2308.02457Google Scholar
- [46] . 2019. KGAT: Knowledge graph attention network for recommendation. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 950–958.Google ScholarDigital Library
- [47] . 2024. MEGA: Meta-graph augmented pre-training model for knowledge graph completion. ACM Transactions on Knowledge Discovery from Data 18, 1 (2024), 30:1–30:24.Google ScholarDigital Library
- [48] . 2017. DeepPath: A reinforcement learning method for knowledge graph reasoning. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 564–573.Google ScholarCross Ref
- [49] . 2015. Embedding entities and relations for learning and inference in knowledge bases. In Proceedings of the 3rd International Conference on Learning Representations.Google Scholar
- [50] . 2017. Differentiable learning of logical rules for knowledge base reasoning. In Proceedings of the Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017. 2319–2328.Google Scholar
- [51] . 2021. Performance-adaptive sampling strategy towards fast and accurate graph neural networks. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2021. 2046–2056.Google ScholarDigital Library
- [52] . 2017. Deep sets. In Proceedings of the Advances in Neural Information Processing Systems. 3391–3401.Google Scholar
- [53] . 2021. Decoupling the depth and scope of graph neural networks. In Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021. 19665–19679.Google Scholar
- [54] . 2020. GraphSAINT: Graph sampling based inductive learning method. In Proceedings of the 8th International Conference on Learning Representations.Google Scholar
- [55] . 2018. Variational reasoning for question answering with knowledge graph. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 6069–6076.Google ScholarCross Ref
- [56] . 2023. Missing-edge aware knowledge graph inductive inference through dual graph learning and traversing. Expert Systems with Applications 213, Part B (2023), 118969.Google ScholarDigital Library
- [57] . 2022. Knowledge graph reasoning with relational digraph. In Proceedings of the ACM Web Conference 2022. 912–924.Google ScholarDigital Library
- [58] . 2019. ERNIE: Enhanced language representation with informative entities. In Proceedings of the 57th Conference of the Association for Computational Linguistics. 1441–1451.Google ScholarCross Ref
- [59] . 2021. Neural bellman-ford networks: A general graph neural network framework for link prediction. In Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021. 29476–29490.Google Scholar
- [60] . 2019. Layer-dependent importance sampling for training deep and large graph convolutional networks. In Proceedings of the Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019. 11247–11256.Google Scholar
Index Terms
- Incorporating Multi-Level Sampling with Adaptive Aggregation for Inductive Knowledge Graph Completion
Recommendations
A versatile adaptive aggregation framework for spatially large discrete location-allocation problems
An adaptive aggregation framework for discrete location-allocation problems is proposed.The mechanism relies on the adjustment of data aggregation to high quality solutions.Adaptive aggregation is more beneficial than the elimination of aggregation ...
Multilevel Adaptive Aggregation for Markov Chains, with Application to Web Ranking
A multilevel adaptive aggregation method for calculating the stationary probability vector of an irreducible stochastic matrix is described. The method is a special case of the adaptive smoothed aggregation and adaptive algebraic multigrid methods for ...
Logic attention based neighborhood aggregation for inductive knowledge graph embedding
AAAI'19/IAAI'19/EAAI'19: Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial IntelligenceKnowledge graph embedding aims at modeling entities and relations with low-dimensional vectors. Most previous methods require that all entities should be seen during training, which is unpractical for real-world knowledge graphs with new entities emerging ...
Comments