Abstract
Multi-hop reasoning over paths in knowledge graphs has attracted rising research interest in the field of knowledge graph completion. Entity types and relation types both contain various kinds of information content though only a subset of them are helpful in the specific triples. Although significant progress has been made by existing models, they have two major shortcomings. First, these models seldom learn an explicit representation of entities and relations with semantic information. Second, they reason without discriminating distinct role types that the same entity with multiple types plays in different triples. To address these issues, we develop a novel path-based reasoning with constrained type attention model, which tries to identify entity types by leveraging relation type constraints in the corresponding triples. Our experimental evaluation shows that the proposed model outperforms the state of the art on a real-world dataset. Further analyses also confirm that both word-level and triple-level attention mechanisms of our model are effective.
Similar content being viewed by others
References
Suchanek FM, Kasneci G, Weikum G (2007) Yago: a core of semantic knowledge. In: Proceedings of the 16th international conference on World Wide Web. ACM, pp 697–706
Bollacker K, Evans C, Paritosh P, Sturge T, Taylor J (2008) Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of the 2008 ACM SIGMOD international conference on Management of data. ACM, pp 1247–1250
Carlson A, Betteridge J, Kisiel B, Settles B, Hruschka Jr ER, Mitchell TM (2010) Toward an architecture for never-ending language learning. In: AAAI, vol 5, p 3
Lehmann J, Isele R, Jakob M, Jentzsch A, Kontokostas D, Mendes PN, Bizer C (2015) DBpediaa large-scale, multilingual knowledge base extracted from Wikipedia. Semantic Web 6(2):167–195
Bordes A, Weston J, Usunier N (2014) Open question answering with weakly supervised embedding models. In: Joint European conference on machine learning and knowledge discovery in databases. Springer, Berlin, Heidelberg, pp 165–180
Bordes A, Chopra S, Weston J (2014) Question answering with subgraph embeddings. arXiv preprint arXiv:1406.3676
Zhang F, Yuan NJ, Lian D, Xie X, Ma WY (2016) Collaborative knowledge base embedding for recommender systems. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 353–362
Catherine R, Cohen W (2016) Personalized recommendations using knowledge graphs: a probabilistic logic programming approach. In: Proceedings of the 10th ACM conference on recommender systems. ACM, pp 325–332
Lao N, Mitchell T, Cohen WW (2011) Random walk inference and learning in a large scale knowledge base. In: Proceedings of the conference on empirical methods in natural language processing. Association for Computational Linguistics, pp 529–539
Wen D, Liu Y, Yuan K, Si S, Shen Y (2017) Attention-aware path-based relation extraction for medical knowledge graph. In: International conference on smart computing and communication. Springer, Cham, pp 321–331
Neelakantan A, Roth B, Mc-Callum A (2015) Compositional vector space models for knowledge base inference. In: 2015 AAAI spring symposium series
Das R, Neelakantan A, Belanger D, McCallum A (2017) Chains of reasoning over entities, relations, and text using recurrent neural networks. In: Proceedings of the 15th conference of the European Chapter of the Association for Computational Linguistics: volume 1, Long Papers, p 141
Bordes A, Usunier N, Garcia-Duran A, Weston J, Yakhnenko O (2013) Translating embeddings for modeling multi-relational data. In: Advances in neural information processing systems, pp 2787–2795
Ji G, He S, Xu L, Liu K, Zhao J (2015) Knowledge graph embedding via dynamic mapping matrix. In: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing (volume 1: long papers), pp 687–696
Ji G, Liu K, He S, Zhao J (2016) Knowledge graph completion with adaptive sparse transfer matrix. In: AAAI, pp 985–991
Zelenko D, Aone C, Richardella A (2003) Kernel methods for relation extraction. J Mach Learn Res 3:1083–1106
Gormley MR, Yu M, Dredze M (2015) Improved relation extraction with feature-rich compositional embedding models. In: Proceedings of the 2015 conference on empirical methods in natural language processing, pp 1774–1784
Socher R, Huval B, Manning C D, Ng A Y (2012, July) Semantic compositionality through recursive matrix-vector spaces. In: Proceedings of the 2012 joint conference on empirical methods in natural language processing and computational natural language learning. Association for Computational Linguistics, pp 1201–1211
Angeli G, Manning C (2013) Philosophers are mortal: Inferring the truth of unseen facts. In: Proceedings of the seventeenth conference on computational natural language learning, pp 133–142
Lao N, Cohen WW (2010) Relational retrieval using a combination of path-constrained random walks. Mach Learn 81(1):53–67
Socher R, Chen D, Manning C D, Ng A (2013) Reasoning with neural tensor networks for knowledge base completion. In: Advances in neural information processing systems, pp 926–934
Gardner M, Talukdar P, Krishnamurthy J, Mitchell T (2014) Incorporating vector space similarity in random walk inference over knowledge bases. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 397–406
Gardner M, Talukdar P P, Kisiel B, Mitchell T (2013) Improving learning and inference in a large knowledge-base using latent syntactic cues. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp 833-838
Toutanova K, Chen D, Pantel P, Poon H, Choudhury P, Gamon M (2015) Representing text for joint embedding of text and knowledge bases. In: Proceedings of the 2015 conference on empirical methods in natural language processing, pp 1499–1509
Jiang X, Wang Q, Qi B, Qiu Y, Li P, Wang B (2017) Attentive path combination for knowledge graph completion. In: Asian conference on machine learning, pp 590–605
Chen W, Xiong W, Yan X, Wang WY (2018) Variational knowledge graph reasoning. In: Proceedings of the 2018 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (Long Papers), pp 1823–1832
Shen Y, Deng Y, Yang M, Li Y, Du N, Fan W, Lei K (2018) Knowledge-aware attentive neural network for ranking question answer pairs. In: The 41st international ACM SIGIR conference on research development in information retrieval. ACM, pp 901–904
Bahdanau D, Cho K, Bengio Y (2015) Neural machine translation by jointly learning to align and translate. In: Proceedings of international conference on learning representations
Zhou P, Shi W, Tian J, Qi Z, Li B, Hao H, Xu B (2016) Attention-based bidirectional long short-term memory networks for relation classification. In: Proceedings of the 54th annual meeting of the association for computational linguistics (volume 2: short papers), pp 207–212
Lin Y, Shen S, Liu Z, Luan H, Sun M (2016) Neural relation extraction with selective attention over instances. In: Proceedings of the 54th annual meeting of the association for computational linguistics (volume 1: long papers), pp 2124–2133
Yang Z, Yang D, Dyer C, He X, Smola A, Hovy E (2016) Hierarchical attention networks for document classification. In: Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies, pp 1480–1489
Zhang H, Li J, Ji Y, Yue H (2017) Understanding subtitles by character-level sequence-to-sequence learning. IEEE Trans Ind Inf 13(2):616–624
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780
Glorot X, Bengio Y (2010) Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp 249–256
Kingma D P, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980
Acknowledgements
This work was financially supported by the National Natural Science Foundation of China (No. 61602013), Natural Science Foundation of Guangdong (No. 2018A030313017) and the Shenzhen Fundamental Research Project (No. JCYJ20170818091546869).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that there is no actual or potential conflict of interest in relation to this article.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Lei, K., Zhang, J., Xie, Y. et al. Path-based reasoning with constrained type attention for knowledge graph completion. Neural Comput & Applic 32, 6957–6966 (2020). https://doi.org/10.1007/s00521-019-04181-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-019-04181-1