Skip to main content

End-to-end Relation-Enhanced Learnable Graph Self-attention Network for Knowledge Graphs Embedding

  • Conference paper
  • First Online:
Computer Supported Cooperative Work and Social Computing (ChineseCSCW 2023)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 2012))

  • 468 Accesses

Abstract

The knowledge graphs embedding performance of the classic graph convolutional network has been limited due to the large-scale knowledge information. The complex knowledge information requires the model for better learnability rather than linearly weighted qualitative constraints. By studying the structural characteristics of the knowledge graph and investigation the imbalance of knowledge information, the end-to-end relation-enhanced learnable graph self-attention network for knowledge graphs embedding is proposed in this work. A relation-enhanced adjacency matrix is constructed to take into account the incompleteness of the knowledge graph. The convolutional knowledge sub-graph is introduced by using a graph self-attention network to obtain the entity node information’s global encoding and relevance ranking. The training effect of the Convolutional Knowledge Base (convKB) model is improved by changing the construction of negative samples, and a better reliability score in the decoder can be obtained. The experimental results on the data set FB15k-237 and WN18RR show that the proposed method achieves better scores than the compared methods in terms of Hits@10 and MRR.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Wang, H., Zhang, F., Xie, X., Guo, M.: DKN: deep knowledge-aware network for news recommendation. In: Proceedings of WWW, pp. 1835–1844 (2018)

    Google Scholar 

  2. Huang, X., Zhang, J., Li, D., Li, P.: Knowledge graph embedding based question answering. In: Proceedings of WSDM, pp. 105–113 (2019)

    Google Scholar 

  3. Mancini, M., Camacho-Collados, J., Iacobacci, I., Navigli, R.: Embedding words and senses together via joint knowledgeenhanced training. CoRR abs/1612.02703 (2016)

    Google Scholar 

  4. García-Durán, A., Bordes, A., Usunier, N., Grandvalet, Y.: Combining two and three-way embedding models for link prediction in knowledge bases. J. Artif. Intell. Res. 55, 715–742 (2016)

    Article  MathSciNet  Google Scholar 

  5. Nickel, M., Murphy, K., Tresp, V., Gabrilovich, E.: A review of relational machine learning for knowledge graphs. Proc. IEEE 104(1), 11–33 (2015)

    Article  Google Scholar 

  6. Rossi, A., Firmani, D., Matinata, A., Merialdo, P., Barbosa, D.: Knowledge graph embedding for link prediction: a comparative analysis. ACM Trans. Knowl. Discovery Data 15(2), 1–49 (2021)

    Article  Google Scholar 

  7. Nguyen, D.Q., Nguyen, T.D., Nguyen, D.Q., Phung, D.: A novel embedding model for knowledge base completion based on convolutional neural network. arXiv preprint arXiv:1712.02121 (2017)

  8. Schlichtkrull, M., Kipf, T.N., Bloem, P., van den Berg, R., Titov, I., Welling, M.: Modeling relational data with graph convolutional networks. In: Gangemi, A., et al. (eds.) ESWC 2018. LNCS, vol. 10843, pp. 593–607. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93417-4_38

    Chapter  Google Scholar 

  9. Kalchbrenner, N., Grefenstette, E., Blunsom, P.: A convolutional neural network for modelling sentences. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, vol. 1, pp. 655–665 (2014)

    Google Scholar 

  10. Velicković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., Bengio, Y.: Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)

  11. Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: Advances in Neural Information Processing Systems, pp. 2787–2795 (2013)

    Google Scholar 

  12. Wang, Z., Zhang, J., Feng, J., Chen, Z.: Knowledge graph embedding by translating on hyperplanes. In: Twenty-Eighth AAAI Conference on Artificial Intelligence, pp. 1532–1543 (2014)

    Google Scholar 

  13. Lin, Y., Liu, Z., Sun, M., Liu, Y., Zhu, X.: Learning entity and relation embeddings for knowledge graph completion. In: Twenty-ninth AAAI Conference on Artificial Intelligence, pp. 1–7 (2015)

    Google Scholar 

  14. Ji, G., He, S., Xu, L., Liu, K., Zhao, J.: Knowledge graph embedding via dynamic mapping matrix. In: 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pp. 687–696 (2015)

    Google Scholar 

  15. He, S., Liu, K., Ji, G., Zhao, J.: Learning to represent knowledge graphs with gaussian embedding. In: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, pp. 623–632. ACM (2015)

    Google Scholar 

  16. Ji, G., Liu, K., He, S., Zhao, J.: Knowledge graph completion with adaptive sparse transfer matrix. In: Thirtieth AAAI Conference on Artificial Intelligence, pp. 985–991 (2016)

    Google Scholar 

  17. Ebisu, T., Ichise, R.: TorusE: knowledge graph embedding on a lie group. In: Thirty-Second AAAI Conference on Artificial Intelligence, pp. 1–8 (2018)

    Google Scholar 

  18. Zhiqing, S., Zhi-Hong, D., Jian-Yun, N., Jian, T.: RotatE: knowledge graph embedding by relational rotation in complex space. arXiv preprint arXiv:1902.10197 (2019)

  19. Dettmers, T., Minervini, P., Stenetorp, P., Riedel, S.: Convolutional 2D knowledge graphembeddings. In: Thirty-Second AAAI Conference on Artificial Intelligence, pp. 1–9 (2018)

    Google Scholar 

  20. Vashishth, S., Sanyal, S., Nitin, V., Agrawal, N., Talukdar, P.: Interacte: improving convolution-based knowledge graph embeddings by increasing feature interactions. arXiv preprint arXiv:1911.00219 (2019)

  21. Trouillon, T., Welbl, J., Riedel, S., Gaussier, é., Bouchard, G.: Complex embeddings for simple link prediction. In: International Conference on Machine Learning, pp. 2071–2080 (2016)

    Google Scholar 

  22. Cai, L., Wang, W.Y.: KBGAN: adversarial learning for knowledge graph embeddings. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 1, pp. 1–11 (2018)

    Google Scholar 

  23. Defferrard, M., Bresson, X., Vandergheynst, P.: Convolutional neural networks on graphs with fast localized spectral filtering. In: Proceedings of NIPS, pp. 3837–3845 (2016)

    Google Scholar 

  24. Shang, C., Tang, Y., Huang, J., Bi, J., He, X., Zhou, B.: End-to-end structure-aware convolutional networks for knowledge base completion. arXiv preprint arXiv:1811.04441 (2018)

  25. Nathani, D., Chauhan, J., Sharma, C., Kaul, M.: Learning attention-based embeddings for relation prediction in knowledge graphs. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 4710–4723 (2019)

    Google Scholar 

  26. Ding, B., Wang, Q., Wang, B., Guo, L.: Improving knowledge graph embedding using simple constraints. In Proceedings of ACL, pp. 110–121 (2018)

    Google Scholar 

  27. Kazemi, S.M., Poole, D.: Simple embedding for link prediction in knowledge graphs. In: Proceedings of NIPS, pp. 4289–4300 (2018)

    Google Scholar 

  28. Xu, C., Li, R.: Relation embedding with dihedral group in knowledge graph. In: Proceedings of ACL, pp. 263–272 (2019)

    Google Scholar 

  29. Chen, H., Sun, X., Tian, Y., Perozzi, B., Chen, M., Skiena, S.: Enhanced network embeddings via exploiting edge labels. In: Proceedings of the 27th ACM International Conference on Information and Knowledge Management, pp. 1579–1582 (2018)

    Google Scholar 

  30. Zhang, W., Paudel, B., Zhang, W., Bernstein, A., Chen, H.: Interaction embeddings for prediction and explanation in knowledge graphs. In: Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, pp. 96–104 (2019)

    Google Scholar 

  31. Hamilton William L., Payal, B., Marinka, Z., Dan, J., Jure, L.: Embedding logical queries on knowledge graphs. In: Advances in Neural Information Processing Systems, pp. 2026–2037 (2018)

    Google Scholar 

  32. Vaswani, A., et al.: Attention is all you need. In: Advances in neural information processing systems, pp. 5998–6008 (2017)

    Google Scholar 

  33. Yang, B., Yih, W.-T., He, X., Gao, J., Deng, L.: Embedding entities and relations for learning and inference in knowledge bases. arXiv preprint arXiv:1412.6575 (2014)

Download references

Acknowledgements

This work is partly supported by National Key R &D Program of China (Grant No. 2018YFB1402900), National Natural Science Foundation of China (Grant No. 61966020) and Chongqing Social Science Foundation (Grant No. 2020YBTQ130).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Hongbin Wang .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Jiang, S., Wang, H., Hou, X. (2024). End-to-end Relation-Enhanced Learnable Graph Self-attention Network for Knowledge Graphs Embedding. In: Sun, Y., Lu, T., Wang, T., Fan, H., Liu, D., Du, B. (eds) Computer Supported Cooperative Work and Social Computing. ChineseCSCW 2023. Communications in Computer and Information Science, vol 2012. Springer, Singapore. https://doi.org/10.1007/978-981-99-9637-7_38

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-9637-7_38

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-9636-0

  • Online ISBN: 978-981-99-9637-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics