Abstract
Link prediction on knowledge graphs has become increasingly important in various fields, including recommender systems, question answering, and social networks. To address this challenge, ConvKB, a state-of-the-art model in deep learning approaches, has been proposed. However, ConvKB is limited in its ability to exploit general information about relations because it uses the same filters for all relations. Additionally, ConvKB's design, consisting of a convolution layer and a linear layer, may not provide enough parameters to store necessary information during feature learning. To address these limitations, this paper proposes the ConvHyper model, which combines the ConvKB model with a HyperNetwork architecture to create relation-specific weights for convolutional neural network layers. Specifically, the embedding of relations is linearly combined through a HyperNetwork to generate weights for the base neural network. This HyperNetwork architecture helps reduce the complexity of the search space to identify optimal weights and create a relation-specific weight structure for neural network layers. Experimental results show that ConvHyper significantly improves on all metrics in four well-known datasets. ConvHyper improves on H@1 up to 5.5% and achieves better results on other metrics, ranging from 0.8% to 1%. We also compared ConvHyper with other CNN-based models and found that many metrics have better values, particularly from 1.1% to 4.7% on the H@10 metric. Furthermore, we analyzed the influence of hyperparameters and found that the learning rate and negative samples significantly affect the model's results.














Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availability
The datasets generated and/or analysed during the current study are available in the GitHub repository, https://github.com/lnthanhhcmus/ConvHyper.
Code Availability
The code implemented during the current study are available in the GitHub repository, https://github.com/lnthanhhcmus/ConvHyper.
References
Schneider EW (1973) Course modularization applied: the interface system and its implications for sequence control and data analysis. ERIC, Chicago, Illinois, p 21. https://eric.ed.gov/?id=ED088424
Kejriwal M (2022) Knowledge Graphs. Applied Data Science in Tourism: Interdisciplinary Approaches, Methodologies, and Applications 423–449. https://doi.org/10.1007/978-3-030-88389-8_20
Paulheim H (2017) Knowledge graph refinement: A survey of approaches and evaluation methods. Semantic Web 8:489–508. https://doi.org/10.3233/SW-160218
Noy N, Gao Y, Jain A et al (2019) Industry-scale knowledge graphs: lessons and challenges. Commun ACM 62:36–43. https://doi.org/10.1145/3331166
Curtiss M, Becker I, Bosman T et al (2013) Unicorn: a system for searching the social graph. Proc VLDB Endow 6:1150–1161. https://doi.org/10.14778/2536222.2536239
Bollacker K, Evans C, Paritosh P et al (2008) Freebase: a collaboratively created graph database for structuring human knowledge. In: ACM SIGMOD international conference on Management of data. pp 1247–1250
Vrandečić D, Krötzsch M (2014) Wikidata: a free collaborative knowledgebase. Commun ACM 57:78–85
Auer S, Bizer C, Kobilarov G, et al (2007) Dbpedia: a nucleus for a web of open data. In: The Semantic Web. ISWC ASWC 2007 2007. Lecture Notes in Computer Science. Springer, Berlin, Heidelberg, pp 722–735. https://doi.org/10.1007/978-3-540-76298-0_52
Miller GA (1995) WordNet: a lexical database for English. Commun ACM 38:39–41. https://doi.org/10.1145/219717.219748
Mahdisoltani F, Biega J, Suchanek F (2015) YAGO3: A knowledge base from multilingual wikipedias. In: Proceedings of the 7th Biennial Conference on Innovative Data Systems Research
Ji S, Pan S, Cambria E et al (2021) A survey on knowledge graphs: Representation, acquisition and applications. IEEE Trans Neural Netw Learning Syst 33:494–514. https://doi.org/10.1109/TNNLS.2021.3070843
Dong X, Gabrilovich E, Heitz G et al (2014) Knowledge vault: a web-scale approach to probabilistic knowledge fusion. In: Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. Association for Computing Machinery, New York, NY, USA, pp 601–610
Ben Seghier MEA, Corriea JAFO, Jafari-Asl J et al (2021) On the modeling of the annual corrosion rate in main cables of suspension bridges using combined soft computing model and a novel nature-inspired algorithm. Neural Comput Appl 33:15969–15985. https://doi.org/10.1007/s00521-021-06199-w
Mai SH, Ben Seghier MEA, Nguyen PL et al (2022) A hybrid model for predicting the axial compression capacity of square concrete-filled steel tubular columns. Eng Comput 38:1205–1222. https://doi.org/10.1007/s00366-020-01104-w
Ohadi S, HashemiMonfared SA, AzhdaryMoghaddam M, Givehchi M (2023) Feasibility of a novel predictive model based on multilayer perceptron optimized with Harris hawk optimization for estimating of the longitudinal dispersion coefficient in rivers. Neural Comput Appl 35:7081–7105. https://doi.org/10.1007/s00521-022-08074-8
Dettmers T, Minervini P, Stenetorp P, Riedel S (2018) Convolutional 2d knowledge graph embeddings. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence. pp 1811–1818
Jiang X, Wang Q, Wang B (2019) Adaptive convolution for multi-relational learning. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, pp 978–987
Nguyen DQ, Nguyen TD, Nguyen DQ, Phung D (2018) A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network. In: Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. pp 327–333
Le T, Nguyen D, Le B (2021) Learning Embedding for Knowledge Graph Completion with Hypernetwork. In: Proceedings of the 13th International Conference on Computational Collective Intelligence. Springer International Publishing, Cham, pp 16–28
Bordes A, Usunier N, Garcia-Duran A et al (2013) Translating embeddings for modeling multi-relational data. In: Proceedings of the 26th International Conference on Neural Information Processing Systems. Curran Associates Inc. pp 2787–2795
Wang Z, Zhang J, Feng J, Chen Z (2014) Knowledge graph embedding by translating on hyperplanes. In: Proceedings of the 28th AAAI Conference on Artificial Intelligence. p 8
Sun Z, Deng Z-H, Nie J-Y, Tang J (2019) RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space. In: Proceedings of 7th International Conference on Learning Representations. OpenReview.net
Peng Y, Zhang J (2020) LineaRE: Simple but Powerful Knowledge Graph Embedding for Link Prediction. In: Proceedings of 2020 IEEE International Conference on Data Mining. IEEE, pp 422–431
Chao L, He J, Wang T, Chu W (2021) PairRE: Knowledge Graph Embeddings via Paired Relation Vectors. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. Association for Computational Linguistics, pp 4360–4369
Balažević I, Allen C, Hospedales TM (2019) Hypernetwork knowledge graph embeddings. In: Proceedings of the 2019 International Conference on Artificial Neural Networks. Springer, pp 553–565
Guo L, Sun Z, Hu W (2019) Learning to exploit long-term relational dependencies in knowledge graphs. In: Proceedings of the 36th International Conference on Machine Learning. pp 2505–2514
Nguyen DQ, Vu T, Nguyen TD, et al (2019) A Capsule Network-based Embedding Model for Knowledge Graph Completion and Search Personalization. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Association for Computational Linguistics, Minneapolis, Minnesota, pp 2180–2189
Schlichtkrull M, Kipf TN, Bloem P et al (2018) Modeling Relational Data with Graph Convolutional Networks. In: Gangemi A, Navigli R, Vidal M-E et al (eds) The Semantic Web. Springer International Publishing, Cham, pp 593–607
Ye R, Li X, Fang Y et al (2019) A Vectorized Relational Graph Convolutional Network for Multi-Relational Network Alignment. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence. pp 4135–4141
Ha D, Dai A, Le QV (2017) Hypernetworks. In: Proceedings of the 5th International Conference on Learning Representations
Lin Y, Liu Z, Sun M et al (2015) Learning Entity and Relation Embeddings for Knowledge Graph Completion. In: Proceedings of the AAAI Conference on Artificial Intelligence
Toutanova K, Chen D (2015) Observed versus latent features for knowledge base and text inference. In: Proceedings of the 3rd Workshop on Continuous Vector Space Models and their Compositionality. Association for Computational Linguistics, Beijing, China, pp 57–66
Rossi A, Firmani D, Matinata A et al (2021) Knowledge Graph Embedding for Link Prediction: A Comparative Analysis. ACM Trans Knowl Discov Data 15:1–49. https://doi.org/10.1145/3424672
Shen T, Zhang F, Cheng J (2022) A comprehensive overview of knowledge graph completion. Knowl-Based Syst 255:109597. https://doi.org/10.1016/j.knosys.2022.109597
Duchi J, Hazan E, Singer Y (2011) Adaptive subgradient methods for online learning and stochastic optimization. J Mach Learn Res 12:2121–2159
Kingma DP, Ba J (2015) Adam: A method for stochastic optimization. In: Proceedings of the 3rd International Conference on Learning Representations
Kazemi SM, Poole D (2018) SimplE embedding for link prediction in knowledge graphs. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems. Curran Associates Inc., Red Hook, NY, USA, pp 4289–4300
Nathani D, Chauhan J, Sharma C, Kaul M (2019) Learning attention-based embeddings for relation prediction in knowledge graphs. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, pp 4710–4723
Xie X, Zhang N, Li Z, et al (2022) From Discrimination to Generation: Knowledge Graph Completion with Generative Transformer. In: Companion Proceedings of the Web Conference 2022. Association for Computing Machinery, New York, NY, USA, pp 162–165
Kamigaito H, Hayashi K (2021) Unified Interpretation of Softmax Cross-Entropy and Negative Sampling: With Case Study for Knowledge Graph Embedding. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Association for Computational Linguistics, Online, pp 5517–5531
Huang J, Zhang T, Zhu J et al (2021) A deep embedding model for knowledge graph completion based on attention mechanism. Neural Comput & Applic 33:9751–9760. https://doi.org/10.1007/s00521-021-05742-z
Junhua D, Yucheng H, Yi-an Z, Dong Z (2022) Attention-Based Relational Graph Convolutional Network for Knowledge Graph Reasoning. In: 2022 21st International Symposium on Communications and Information Technologies (ISCIT). pp 216–221
Das R, Godbole A, Monath N et al (2020) Probabilistic Case-based Reasoning for Open-World Knowledge Graph Completion. In: Findings of the Association for Computational Linguistics: EMNLP 2020. Association for Computational Linguistics, pp 4752–4765
Le T, Le N, Le B (2023) Knowledge graph embedding by relational rotation and complex convolution for link prediction. Expert Syst Appl 214:119122. https://doi.org/10.1016/j.eswa.2022.119122
Author information
Authors and Affiliations
Contributions
Thanh Le: Conceptualization, Methodology, Validation, Supervision, Writing – original draft. Duy Nguyen: Methodology, Software, Formal analysis, Visualization, Writing – original draft, Writing – review & editing. Bac Le: Conceptualization, Methodology, Supervision, Review, ValidationDeclarations.
Corresponding author
Ethics declarations
Informed consent
Informed consent was obtained from all of the subjects involved in this study.
Competing interests
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix
Appendix
Table 12
Table 13
Table 14
Table 15
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Le, T., Nguyen, D. & Le, B. Generating relation-specific weights for ConvKB using a HyperNetwork architecture. Appl Intell 53, 21092–21115 (2023). https://doi.org/10.1007/s10489-023-04670-w
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-023-04670-w