Abstract
Knowledge graphs (KGs) are being widely utilised in many fields of artificial intelligence. However, the lack of relations has become a huge obstacle to their real-life application. Therefore, recent researches in this field tend to focus on link/relation prediction techniques on KGs, but most of them merely learn KG embeddings from the central nodes’ neighbourhood in an absolute or unconditional way, fusing a great deal of weak or even useless information. To tackle this issue, we propose a novel end-to-end KG embedding model named Relational Gated Graph Attention neTwork (R-GGAT), which learns embeddings by supervising and controlling the node aggregation process by selecting and filtering the aggregated information through its multi-attention mechanism (MAM), in an attempt to fully and effectively mine the underlying semantic information of KGs. Evaluation experiments on several datasets have been performed, where our proposed R-GGAT achieved the elevated performance in link prediction tasks compared to several state-of-the-art baselines. We also carried out ablation study and analysed the changing of node embeddings filtered by the MAM to demonstrate the effectiveness of our model.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bollacker, K., Evans, C., Paritosh, P., Sturge, T., Taylor, J.: Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, pp. 1247–1250 (2008)
Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: Neural Information Processing Systems (NIPS), pp. 1–9 (2013)
Chen, W., Chang, L., Bin, C., Gu, T., Jia, Z.: Jointing knowledge graph and neural network for Top-N recommendation. In: PRICAI 2019: Trends in Artificial Intelligence (2019)
Cho, K., et al.: Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014)
Dettmers, T., Minervini, P., Stenetorp, P., Riedel, S.: Convolutional 2D knowledge graph embeddings. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)
Kok, S., Domingos, P.: Statistical predicate invention. In: Proceedings of the 24th International Conference on Machine Learning, pp. 433–440 (2007)
Lehmann, J., et al.: Dbpedia-a large-scale, multilingual knowledge base extracted from wikipedia. Semant. Web 6(2), 167–195 (2015)
Li, S., Huang, Z., Cheng, G., Kharlamov, E., Gunaratna, K.: Enriching documents with compact, representative, relevant knowledge graphs. In: Twenty-Ninth International Joint Conference on Artificial Intelligence and Seventeenth Pacific Rim International Conference on Artificial Intelligence IJCAI-PRICAI-20 (2020)
Lin, X.V., Socher, R., Xiong, C.: Multi-hop knowledge graph reasoning with reward shaping. arXiv preprint arXiv:1808.10568 (2018)
Ma, L., Sun, P., Lin, Z., Wang, H.: Composing knowledge graph embeddings via word embeddings. arXiv preprint arXiv:1909.03794 (2019)
Nathani, D., Chauhan, J., Sharma, C., Kaul, M.: Learning attention-based embeddings for relation prediction in knowledge graphs. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics (2019)
Nguyen, D.Q., Nguyen, T.D., Nguyen, D.Q., Phung, D.: A novel embedding model for knowledge base completion based on convolutional neural network. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 2 (Short Papers), pp. 327–333. Association for Computational Linguistics, New Orleans (2018). https://doi.org/10.18653/v1/N18-2053, https://www.aclweb.org/anthology/N18-2053
Schlichtkrull, M., Kipf, T.N., Bloem, P., van den Berg, R., Titov, I., Welling, M.: Modeling relational data with graph convolutional networks. In: Gangemi, A., et al. (eds.) ESWC 2018. LNCS, vol. 10843, pp. 593–607. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93417-4_38
Shang, C., Tang, Y., Huang, J., Bi, J., He, X., Zhou, B.: End-to-end structure-aware convolutional networks for knowledge base completion. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 3060–3067 (2019)
Stewart, M., Liu, W.: Seq2kg: an end-to-end neural model for domain agnostic knowledge graph (not text graph) construction from text. In: Proceedings of the International Conference on Principles of Knowledge Representation and Reasoning, vol. 17, pp. 748–757 (2020)
Toutanova, K., Chen, D., Pantel, P., Poon, H., Choudhury, P., Gamon, M.: Representing text for joint embedding of text and knowledge bases. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1499–1509 (2015)
Trouillon, T., Welbl, J., Riedel, S., Gaussier, É., Bouchard, G.: Complex embeddings for simple link prediction. In: International Conference on Machine Learning, pp. 2071–2080. PMLR (2016)
Vashishth, S., Sanyal, S., Nitin, V., Talukdar, P.: Composition-based multi-relational graph convolutional networks. arXiv preprint arXiv:1911.03082 (2019)
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)
Wang, Y., Liu, Y., Zhang, H., Xie, H.: Leveraging lexical semantic information for learning concept-based multiple embedding representations for knowledge graph completion. In: Shao, J., Yiu, M.L., Toyoda, M., Zhang, D., Wang, W., Cui, B. (eds.) APWeb-WAIM 2019. LNCS, vol. 11641, pp. 382–397. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-26072-9_28
Yang, B., Yih, W.T., He, X., Gao, J., Deng, L.: Embedding entities and relations for learning and inference in knowledge bases. arXiv preprint arXiv:1412.6575 (2014)
Zhu, G., Bin, C., Gu, T., Chang, L., Jia, Z.: A neural user preference modeling framework for recommendation based on knowledge graph. In: PRICAI 2019: Trends in Artificial Intelligence (2019)
Zhu, Z., Yu, J., Wang, Y., Sun, Y., Wu, Q.: MUCKO: multi-layer cross-modal knowledge reasoning for fact-based visual question answering. In: Twenty-Ninth International Joint Conference on Artificial Intelligence and Seventeenth Pacific Rim International Conference on Artificial Intelligence IJCAI-PRICAI-20 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, M., Li, H., Qiu, L. (2022). Learning Knowledge Graph Embeddings by Multi-Attention Mechanism for Link Prediction. In: Lai, Y., Wang, T., Jiang, M., Xu, G., Liang, W., Castiglione, A. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2021. Lecture Notes in Computer Science(), vol 13155. Springer, Cham. https://doi.org/10.1007/978-3-030-95384-3_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-95384-3_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-95383-6
Online ISBN: 978-3-030-95384-3
eBook Packages: Computer ScienceComputer Science (R0)