Abstract
The knowledge graphs embedding performance of the classic graph convolutional network has been limited due to the large-scale knowledge information. The complex knowledge information requires the model for better learnability rather than linearly weighted qualitative constraints. By studying the structural characteristics of the knowledge graph and investigation the imbalance of knowledge information, the end-to-end relation-enhanced learnable graph self-attention network for knowledge graphs embedding is proposed in this work. A relation-enhanced adjacency matrix is constructed to take into account the incompleteness of the knowledge graph. The convolutional knowledge sub-graph is introduced by using a graph self-attention network to obtain the entity node information’s global encoding and relevance ranking. The training effect of the Convolutional Knowledge Base (convKB) model is improved by changing the construction of negative samples, and a better reliability score in the decoder can be obtained. The experimental results on the data set FB15k-237 and WN18RR show that the proposed method achieves better scores than the compared methods in terms of Hits@10 and MRR.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Wang, H., Zhang, F., Xie, X., Guo, M.: DKN: deep knowledge-aware network for news recommendation. In: Proceedings of WWW, pp. 1835–1844 (2018)
Huang, X., Zhang, J., Li, D., Li, P.: Knowledge graph embedding based question answering. In: Proceedings of WSDM, pp. 105–113 (2019)
Mancini, M., Camacho-Collados, J., Iacobacci, I., Navigli, R.: Embedding words and senses together via joint knowledgeenhanced training. CoRR abs/1612.02703 (2016)
García-Durán, A., Bordes, A., Usunier, N., Grandvalet, Y.: Combining two and three-way embedding models for link prediction in knowledge bases. J. Artif. Intell. Res. 55, 715–742 (2016)
Nickel, M., Murphy, K., Tresp, V., Gabrilovich, E.: A review of relational machine learning for knowledge graphs. Proc. IEEE 104(1), 11–33 (2015)
Rossi, A., Firmani, D., Matinata, A., Merialdo, P., Barbosa, D.: Knowledge graph embedding for link prediction: a comparative analysis. ACM Trans. Knowl. Discovery Data 15(2), 1–49 (2021)
Nguyen, D.Q., Nguyen, T.D., Nguyen, D.Q., Phung, D.: A novel embedding model for knowledge base completion based on convolutional neural network. arXiv preprint arXiv:1712.02121 (2017)
Schlichtkrull, M., Kipf, T.N., Bloem, P., van den Berg, R., Titov, I., Welling, M.: Modeling relational data with graph convolutional networks. In: Gangemi, A., et al. (eds.) ESWC 2018. LNCS, vol. 10843, pp. 593–607. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93417-4_38
Kalchbrenner, N., Grefenstette, E., Blunsom, P.: A convolutional neural network for modelling sentences. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, vol. 1, pp. 655–665 (2014)
Velicković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., Bengio, Y.: Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)
Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: Advances in Neural Information Processing Systems, pp. 2787–2795 (2013)
Wang, Z., Zhang, J., Feng, J., Chen, Z.: Knowledge graph embedding by translating on hyperplanes. In: Twenty-Eighth AAAI Conference on Artificial Intelligence, pp. 1532–1543 (2014)
Lin, Y., Liu, Z., Sun, M., Liu, Y., Zhu, X.: Learning entity and relation embeddings for knowledge graph completion. In: Twenty-ninth AAAI Conference on Artificial Intelligence, pp. 1–7 (2015)
Ji, G., He, S., Xu, L., Liu, K., Zhao, J.: Knowledge graph embedding via dynamic mapping matrix. In: 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pp. 687–696 (2015)
He, S., Liu, K., Ji, G., Zhao, J.: Learning to represent knowledge graphs with gaussian embedding. In: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, pp. 623–632. ACM (2015)
Ji, G., Liu, K., He, S., Zhao, J.: Knowledge graph completion with adaptive sparse transfer matrix. In: Thirtieth AAAI Conference on Artificial Intelligence, pp. 985–991 (2016)
Ebisu, T., Ichise, R.: TorusE: knowledge graph embedding on a lie group. In: Thirty-Second AAAI Conference on Artificial Intelligence, pp. 1–8 (2018)
Zhiqing, S., Zhi-Hong, D., Jian-Yun, N., Jian, T.: RotatE: knowledge graph embedding by relational rotation in complex space. arXiv preprint arXiv:1902.10197 (2019)
Dettmers, T., Minervini, P., Stenetorp, P., Riedel, S.: Convolutional 2D knowledge graphembeddings. In: Thirty-Second AAAI Conference on Artificial Intelligence, pp. 1–9 (2018)
Vashishth, S., Sanyal, S., Nitin, V., Agrawal, N., Talukdar, P.: Interacte: improving convolution-based knowledge graph embeddings by increasing feature interactions. arXiv preprint arXiv:1911.00219 (2019)
Trouillon, T., Welbl, J., Riedel, S., Gaussier, é., Bouchard, G.: Complex embeddings for simple link prediction. In: International Conference on Machine Learning, pp. 2071–2080 (2016)
Cai, L., Wang, W.Y.: KBGAN: adversarial learning for knowledge graph embeddings. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 1, pp. 1–11 (2018)
Defferrard, M., Bresson, X., Vandergheynst, P.: Convolutional neural networks on graphs with fast localized spectral filtering. In: Proceedings of NIPS, pp. 3837–3845 (2016)
Shang, C., Tang, Y., Huang, J., Bi, J., He, X., Zhou, B.: End-to-end structure-aware convolutional networks for knowledge base completion. arXiv preprint arXiv:1811.04441 (2018)
Nathani, D., Chauhan, J., Sharma, C., Kaul, M.: Learning attention-based embeddings for relation prediction in knowledge graphs. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 4710–4723 (2019)
Ding, B., Wang, Q., Wang, B., Guo, L.: Improving knowledge graph embedding using simple constraints. In Proceedings of ACL, pp. 110–121 (2018)
Kazemi, S.M., Poole, D.: Simple embedding for link prediction in knowledge graphs. In: Proceedings of NIPS, pp. 4289–4300 (2018)
Xu, C., Li, R.: Relation embedding with dihedral group in knowledge graph. In: Proceedings of ACL, pp. 263–272 (2019)
Chen, H., Sun, X., Tian, Y., Perozzi, B., Chen, M., Skiena, S.: Enhanced network embeddings via exploiting edge labels. In: Proceedings of the 27th ACM International Conference on Information and Knowledge Management, pp. 1579–1582 (2018)
Zhang, W., Paudel, B., Zhang, W., Bernstein, A., Chen, H.: Interaction embeddings for prediction and explanation in knowledge graphs. In: Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, pp. 96–104 (2019)
Hamilton William L., Payal, B., Marinka, Z., Dan, J., Jure, L.: Embedding logical queries on knowledge graphs. In: Advances in Neural Information Processing Systems, pp. 2026–2037 (2018)
Vaswani, A., et al.: Attention is all you need. In: Advances in neural information processing systems, pp. 5998–6008 (2017)
Yang, B., Yih, W.-T., He, X., Gao, J., Deng, L.: Embedding entities and relations for learning and inference in knowledge bases. arXiv preprint arXiv:1412.6575 (2014)
Acknowledgements
This work is partly supported by National Key R &D Program of China (Grant No. 2018YFB1402900), National Natural Science Foundation of China (Grant No. 61966020) and Chongqing Social Science Foundation (Grant No. 2020YBTQ130).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Jiang, S., Wang, H., Hou, X. (2024). End-to-end Relation-Enhanced Learnable Graph Self-attention Network for Knowledge Graphs Embedding. In: Sun, Y., Lu, T., Wang, T., Fan, H., Liu, D., Du, B. (eds) Computer Supported Cooperative Work and Social Computing. ChineseCSCW 2023. Communications in Computer and Information Science, vol 2012. Springer, Singapore. https://doi.org/10.1007/978-981-99-9637-7_38
Download citation
DOI: https://doi.org/10.1007/978-981-99-9637-7_38
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-9636-0
Online ISBN: 978-981-99-9637-7
eBook Packages: Computer ScienceComputer Science (R0)