Abstract
Knowledge graph completion has become a well-studied problem and a non-trivial task with the broad application of the knowledge graphs. Previously, a lot of works have been proposed to solve the knowledge graph completion problem, for example, a series of Trans model, semantic matching models, convolutional neural networks based methods and so on. However, a series of Trans models and semantic matching models only focused on the shadow information of the knowledge graph, thus failed to capture the implicit fine-grained feature in the triple of knowledge graphs; convolutional neural networks based methods learned more expressive feature for knowledge graph completion, and it also ignored the directional relation characteristic and implicit fine-grained feature in the triple. In this paper, we propose a novel knowledge graph completion model named directional multi-dimensional attention convolution model that explores directional information and an inherent deep expressive characteristic of the triple. At last, we evaluate our directional multi-dimensional attention convolution model based on three standard evaluation criteria in two robust datasets, and the experiment shows that our model achieves state-of-the-art MeanRank.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bollacker K, Evans C, Paritosh P, Sturge T, Taylor J (2008) Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of the 2008 ACM SIGMOD international conference on management of data, SIGMOD ’08, pp 1247–1250. ACM, New York, NY, USA. https://doi.org/10.1145/1376616.1376746. URL http://doi.acm.org/10.1145/1376616.1376746
Dong X, Gabrilovich E, Heitz G, Horn W, Lao N, Murphy K, Strohmann T, Sun S, Zhang W (2014) Knowledge vault: a web-scale approach to probabilistic knowledge fusion. In: Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, pp 601–610
Huang CQ, Han ZM, Li MX, yung Jong MS, Tsai CC, (2019) Investigating students’ interaction patterns and dynamic learning sentiments in online discussions. Computers and Education 140:103589. https://doi.org/10.1016/j.compedu.2019.05.015. URL http://www.sciencedirect.com/science/article/pii/S0360131519301307
Shen T, Zhou T, Long G, Jiang J, Pan S, Zhang C (2018) Disan: Directional self-attention network for rnn/cnn-free language understanding. In: McIlraith SA, Weinberger KQ (eds) Proceedings of the 32nd AAAI conference on artificial intelligence, (AAAI-18), the 30th innovative applications of artificial intelligence (IAAI-18), and the 8th AAAI symposium on educational advances in artificial intelligence (EAAI-18), New Orleans, Louisiana, USA, Feb 2–7, 2018, pp 5446–5455. AAAI Press. URL https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16126
Bordes A, Usunier N, Garcia-Duran A, Weston J, Yakhnenko O (2013) Translating embeddings for modeling multi-relational data. In: Advances in neural information processing systems, pp 2787–2795
Wang Z, Zhang J, Feng J, Chen Z (2014) Knowledge graph embedding by translating on hyperplanes. In: Brodley CE, Stone P (eds) Proceedings of the 28th AAAI conference on artificial intelligence, July 27–31, 2014, Québec City, Québec, Canada, pp 1112–1119. AAAI Press. URL http://www.aaai.org/ocs/index.php/AAAI/AAAI14/paper/view/8531
Lin Y, Liu Z, Sun M, Liu Y, Zhu X (2015) Learning entity and relation embeddings for knowledge graph completion. In: Bonet B, Koenig S (eds) Proceedings of the 29th AAAI conference on artificial intelligence, Jan 25–30, 2015, Austin, Texas, USA, pp 2181–2187. AAAI Press. URL http://www.aaai.org/ocs/index.php/AAAI/AAAI15/paper/view/9571
Ji G, He S, Xu L, Liu K, Zhao J (2015) Knowledge graph embedding via dynamic mapping matrix. In: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing (vol 1: Long Papers), pp 687–696
Ji G, Liu K, He S, Zhao J (2016) Knowledge graph completion with adaptive sparse transfer matrix. In: Schuurmans D, Wellman MP (eds) Proceedings of the 30th AAAI conference on artificial intelligence, Feb 12–17, 2016, Phoenix, Arizona, USA, pp 985–991. AAAI Press. URL http://www.aaai.org/ocs/index.php/AAAI/AAAI16/paper/view/11982
Nickel M, Tresp V, Kriegel H (2011) A three-way model for collective learning on multi-relational data. In: Getoor L, Scheffer T (eds) Proceedings of the 28th international conference on machine learning, ICML 2011, Bellevue, Washington, USA, June 28–July 2, 2011, pp 809–816. Omnipress. URL https://icml.cc/2011/papers/438_icmlpaper.pdf
Yang B, Yih W, He X, Gao J, Deng L (2015) Embedding entities and relations for learning and inference in knowledge bases. In: Bengio Y, LeCun Y (eds) 3rd international conference on learning representations, ICLR 2015, San Diego, CA, USA, May 7–9, 2015, Conference Track Proceedings. URL http://arxiv.org/abs/1412.6575
Trouillon T, Welbl J, Riedel S, Gaussier É, Bouchard G (2016) Complex embeddings for simple link prediction. In: Balcan M, Weinberger KQ (eds) Proceedings of the 33nd international conference on machine learning, ICML 2016, New York City, NY, USA, June 19–24, 2016, JMLR Workshop and Conference Proceedings, vol. 48, pp 2071–2080. JMLR.org. URL http://proceedings.mlr.press/v48/trouillon16.html
Sun Z, Deng Z, Nie J, Tang J (2019) Rotate: knowledge graph embedding by relational rotation in complex space. In: 7th international conference on learning representations, ICLR 2019, New Orleans, LA, USA, May 6–9, 2019. OpenReview.net. URL https://openreview.net/forum?id=HkgEQnRqYQ
Dettmers T, Minervini P, Stenetorp P, Riedel S (2018) Convolutional 2d knowledge graph embeddings. In: McIlraith SA, Weinberger KQ (eds) Proceedings of the 32nd AAAI conference on artificial intelligence, (AAAI-18), the 30th innovative applications of artificial intelligence (IAAI-18), and the 8th AAAI Symposium on educational advances in artificial intelligence (EAAI-18), New Orleans, Louisiana, USA, Feb 2–7, 2018, pp 1811–1818. AAAI Press. URL https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/17366
Nguyen DQ, Nguyen TD, Nguyen DQ, Phung D (2018) A novel embedding model for knowledge base completion based on convolutional neural network. In: Proceedings of the 2018 conference of the north american chapter of the association for computational linguistics: human language technologies, Volu 2 (Short Papers), pp 327–333. Association for Computational Linguistics, New Orleans, Louisiana. https://doi.org/10.18653/v1/N18-2053. URL https://www.aclweb.org/anthology/N18-2053
Kipf TN, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: 5th international conference on learning representations, ICLR 2017, Toulon, France, Apr 24–26, 2017, Conference Track Proceedings. OpenReview.net. URL https://openreview.net/forum?id=SJU4ayYgl
Schlichtkrull MS, Kipf TN, Bloem P, van den Berg R, Titov I, Welling M (2018) Modeling relational data with graph convolutional networks. In: Gangemi A, Navigli R, Vidal M, Hitzler P, Troncy R, Hollink L, Tordai A, Alam M (eds) The semantic web—15th international conference, ESWC 2018, Heraklion, Crete, Greece, June 3-7, 2018, Proceedings, Lecture Notes in Computer Science, vol 10843, pp 593–607. Springer. https://doi.org/10.1007/978-3-319-93417-4_38. URL https://doi.org/10.1007/978-3-319-93417-4_38
Xie R, Liu Z, Jia J, Luan H, Sun M (2016) Representation learning of knowledge graphs with entity descriptions. In: Schuurmans D, Wellman MP (eds) Proceedings of the 30th AAAI conference on artificial intelligence, Feb 12–17, 2016, Phoenix, Arizona, USA, pp 2659–2665. AAAI Press. URL http://www.aaai.org/ocs/index.php/AAAI/AAAI16/paper/view/12216
Wang Z, Li J (2016) Text-enhanced representation learning for knowledge graph. In: Kambhampati S (ed) Proceedings of the 35th international joint conference on artificial intelligence, IJCAI 2016, New York, NY, USA, 9–15 July 2016, pp 1293–1299. IJCAI/AAAI Press. URL http://www.ijcai.org/Abstract/16/187
Shi B, Weninger T (2018) Open-world knowledge graph completion. In: McIlraith SA, Weinberger KQ (eds) Proceedings of the 32nd AAAI conference on artificial intelligence, (AAAI-18), the 30th innovative applications of artificial intelligence (IAAI-18), and the 8th AAAI symposium on educational advances in artificial intelligence (EAAI-18), New Orleans, Louisiana, USA, Feb 2–7, 2018, pp 1957–1964. AAAI Press. URL https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16055
Cho K, van Merrienboer B, Gülçehre Ç, Bahdanau D, Bougares F, Schwenk H, Bengio Y (2014) Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: Moschitti A, Pang B, Daelemans W (eds) Proceedings of the 2014 conference on empirical methods in natural language processing, EMNLP 2014, Oct 25–29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL, pp 1724–1734. ACL. https://doi.org/10.3115/v1/d14-1179. URL https://doi.org/10.3115/v1/d14-1179
Kingma DP, Ba J (2015) Adam: A method for stochastic optimization. In: Bengio Y, LeCun Y (eds) 3rd international conference on learning representations, ICLR 2015, San Diego, CA, USA, May 7–9, 2015, Conference Track Proceedings. URL http://arxiv.org/abs/1412.6980
Toutanova K, Chen D (2015) Observed versus latent features for knowledge base and text inference. In: Proceedings of the 3rd workshop on continuous vector space models and their compositionality, pp 57–66
Balažević I, Allen C, Hospedales TM (2019) Hypernetwork knowledge graph embeddings. In: International conference on artificial neural networks. Springer, pp 553–565
Jiang X, Wang Q, Wang B (2019) Adaptive convolution for multi-relational learning. In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, Vol 1 (Long and Short Papers), pp 978–987
Cai L, Wang WY (2018) KBGAN: Adversarial learning for knowledge graph embeddings. In: Proceedings of the 2018 conference of the North American Chapter of the association for computational linguistics: human language technologies, Vol 1 (Long Papers), pp 1470–1480. Association for computational linguistics, New Orleans, Louisiana. https://doi.org/10.18653/v1/N18-1133. URL https://www.aclweb.org/anthology/N18-1133
Pinter Y, Eisenstein J (2018) Predicting semantic relations using global graph properties. In: Proceedings of the 2018 conference on empirical methods in natural language processing, pp 1741–1751. Association for computational linguistics, Brussels, Belgium. https://doi.org/10.18653/v1/D18-1201. URL https://www.aclweb.org/anthology/D18-1201
Shang C, Tang Y, Huang J, Bi J, He X, Zhou B (2019) End-to-end structure-aware convolutional networks for knowledge base completion. In: The 33rd AAAI conference on artificial intelligence, AAAI 2019, The 31st innovative applications of artificial intelligence conference, IAAI 2019, The 9th AAAI symposium on educational advances in artificial intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27 - February 1, 2019, pp 3060–3067. AAAI Press. https://doi.org/10.1609/aaai.v33i01.33013060. URL https://doi.org/10.1609/aaai.v33i01.33013060
Acknowledgement
This work was supported by the National Natural Science Foundation of China under Grant 61877020 and 62077015.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declared that they have no conflicts of interest to this work. We declare that we do not have any commercial or associative interest that represents a conflict of interest in connection with the work submitted.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Huang, J., Zhang, T., Zhu, J. et al. A deep embedding model for knowledge graph completion based on attention mechanism. Neural Comput & Applic 33, 9751–9760 (2021). https://doi.org/10.1007/s00521-021-05742-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-021-05742-z