skip to main content
10.1145/3269206.3271769acmconferencesArticle/Chapter ViewAbstractPublication PagescikmConference Proceedingsconference-collections
research-article

Knowledge Graph Completion by Context-Aware Convolutional Learning with Multi-Hop Neighborhoods

Published: 17 October 2018 Publication History

Abstract

The main focus of relational learning for knowledge graph completion (KGC) lies in exploiting rich contextual information for facts. Many state-of-the-art models incorporate fact sequences, entity types, and even textual information. Unfortunately, most of them do not fully take advantage of rich structural information in a KG, i.e., connectivity patterns around each entity. In this paper, we propose a context-aware convolutional learning (CACL) model which jointly learns from entities and their multi-hop neighborhoods. Since we directly utilize the connectivity patterns contained in each multi-hop neighborhood, the structural role similarity among entities can be better captured, resulting in more informative entity and relation embeddings. Specifically, CACL collects entities and relations from the multi-hop neighborhood as contextual information according to their relative importance and uniquely maps them to a linear vector space. Our convolutional architecture leverages a deep learning technique to represent each entity along with its linearly mapped contextual information. Thus, we can elaborately extract the features of key connectivity patterns from the context and incorporate them into a score function which evaluates the validity of facts. Experimental results on the newest datasets show that CACL outperforms existing approaches by successfully enriching embeddings with neighborhood information.

References

[1]
M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, et almbox. 2016. Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467 (2016).
[2]
A. Abujabal, M. Yahya, M. Riedewald, and G. Weikum. 2017. Automated Template Generation for Question Answering over Knowledge Graphs. In WWW . 1191--1200.
[3]
J. Atwood and D. Towsley. 2016. Diffusion-convolutional neural networks. In NIPS. 1993--2001.
[4]
S. Auer, C. Bizer, G. Kobilarov, J. Lehmann, R. Cyganiak, and Z. Ives. 2007. DBpedia: A nucleus for a web of open data. The semantic web (2007), 722--735.
[5]
K. Bollacker, C. Evans, P. Paritosh, T. Sturge, and J. Taylor. 2008. Freebase: a collaboratively created graph database for structuring human knowledge. In SIGMOD. 1247--1250.
[6]
A. Bordes, N. Usunier, A. Garc'ia-Durán, J. Weston, and O. Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In NIPS . 2787--2795.
[7]
Y. Cao, L. Huang, H. Ji, X. Chen, and J. Li. 2017. Bridging text and knowledge by learning multi-prototype entity mention embedding. In ACL. 1623--1633.
[8]
R. Das, A. Neelakantan, D. Belanger, and A. McCallum. 2017. Chains of reasoning over entities, relations, and text using recurrent neural networks. In EACL . 132--141.
[9]
J. Deriu, A.n Lucchi, D. V. Luca, A. Severyn, S. Müller, M. Cieliebak, Th. Hofmann, and M. Jaggi. 2017. Leveraging Large Amounts of Weakly Supervised Data for Multi-Language Sentiment Classification. In WWW. 1045--1052.
[10]
T. Dettmers, P. Minervini, P. Stenetorp, and S. Riedel. 2018. Convolutional 2D Knowledge Graph Embeddings. In AAAI .
[11]
B. L. Douglas. 2011. The Weisfeiler-Lehman method and graph isomorphism testing. arXiv preprint arXiv:1101.5211 (2011).
[12]
J. Duchi, E. Hazan, and Y. Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, Vol. 12 (2011), 2121--2159.
[13]
A. Garc'ia-Durán, A. Bordes, and N. Usunier. 2015. Composing relationships with translations. In EMNLP. 286--290.
[14]
A. Grover and J. Leskovec. 2016. node2vec: Scalable feature learning for networks. In SIGKDD. 855--864.
[15]
K. He, X. Zhang, S. Ren, and J. Sun. 2015. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In ICCV. 1026--1034.
[16]
X. He, L. Liao, H. Zhang, L. Nie, X. Hu, and T. S. Chua. 2017. Neural collaborative filtering. In WWW. 173--182.
[17]
G. Ji, S. He, L. Xu, K. Liu, and J. Zhao. 2015. Knowledge Graph Embedding via Dynamic Mapping Matrix. In ACL. 687--696.
[18]
D. Kim, C. Park, J. Oh, S. Lee, and H. Yu. 2016. Convolutional matrix factorization for document context-aware recommendation. In RecSys . 233--240.
[19]
Y. Kim. 2014. Convolutional neural networks for sentence classification. In EMNLP . 1746--1751.
[20]
T. N. Kipf and M. Welling. 2017. Semi-supervised classification with graph convolutional networks. In ICLR .
[21]
Y. Koren, R. Bell, and C. Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer, Vol. 42, 8 (2009).
[22]
K. Lee, A. Qadir, S. A. Hasan, V. Datla, A. Prakash, J. Liu, and O. Farri. 2017. Adverse Drug Event Detection in Tweets with Semi-Supervised Convolutional Neural Networks. In WWW. 705--714.
[23]
Y. Lin, Z. Liu, H. Luan, M. Sun, S. Rao, and S. Liu. 2015a. Modeling relation paths for representation learning of knowledge bases. In EMNLP . 705--714.
[24]
Y. Lin, Z. Liu, and M. Sun. 2016. Knowledge representation learning with entities, attributes and relations. In IJCAI . 2866--2872.
[25]
Y. Lin, Z. Liu, M. Sun, Y. Liu, and X. Zhu. 2015b. Learning Entity and Relation Embeddings for Knowledge Graph Completion. In AAAI . 2181--2187.
[26]
D. Lukovnikov, A. Fischer, J. Lehmann, and S. Auer. 2017. Neural Network-based Question Answering over Knowledge Graphs on Word and Character Level. In WWW. 1211--1220.
[27]
A. L. Maas, A. Y. Hannun, and A. Y. Ng. 2013. Rectifier nonlinearities improve neural network acoustic models. In ICML .
[28]
T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. 2013. Distributed representations of words and phrases and their compositionality. In NIPS . 3111--3119.
[29]
G. A. Miller. 1995. WordNet: a lexical database for English. Commun. ACM, Vol. 38, 11 (1995), 39--41.
[30]
B. Min, R. Grishman, L. Wan, C. Wang, and D. Gondek. 2013. Distant supervision for relation extraction with an incomplete knowledge base. In HLT-NAACL . 777--782.
[31]
V. Nair and G. E. Hinton. 2010. Rectified linear units improve restricted boltzmann machines. In ICML. 807--814.
[32]
A. Neelakantan, B. Roth, and A. McCallum. 2015. Compositional Vector Space Models for Knowledge Base Completion. In ACL .
[33]
D. Q. Nguyen. 2017. An overview of embedding models of entities and relationships for knowledge base completion. arXiv preprint arXiv:1703.08098 (2017).
[34]
D. Q. Nguyen, T. D. Nguyen, D. Q. Nguyen, and D. Phung. 2017. A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network. arXiv preprint arXiv:1712.02121 (2017).
[35]
D. Q. Nguyen, K. Sirts, L. Qu, and M. Johnson. 2016a. Neighborhood mixture model for knowledge base completion. In CoNLL. 40--50.
[36]
D. Q. Nguyen, K. Sirts, L. Qu, and M. Johnson. 2016b. STransE: a novel embedding model of entities and relationships in knowledge bases. In NAACL-HLT .
[37]
M. Nickel, K. Murphy, V. Tresp, and E. Gabrilovich. 2016a. A review of relational machine learning for knowledge graphs. Proc. IEEE, Vol. 104, 1 (2016), 11--33.
[38]
M. Nickel, L. Rosasco, T. A Poggio, et almbox. 2016b. Holographic Embeddings of Knowledge Graphs. In AAAI. 1955--1961.
[39]
M. Nickel, V. Tresp, and H. P. Kriegel. 2011. A three-way model for collective learning on multi-relational data. In ICML . 809--816.
[40]
M. Niepert, M. Ahmed, and K. Kutzkov. 2016. Learning convolutional neural networks for graphs. In ICML. 2014--2023.
[41]
B. Perozzi, R. Al-Rfou, and S. Skiena. 2014. DeepWalk: Online learning of social representations. In SIGKDD . 701--710.
[42]
X. Ren, Z. Wu, W. He, M. Qu, C. R. Voss, H. Ji, T. F. Abdelzaher, and J. Han. 2017. CoType: Joint extraction of typed entities and relations with knowledge bases. In WWW . 1015--1024.
[43]
P. Ristoski and H. Paulheim. 2016. Rdf2vec: Rdf graph embeddings for data mining. In ISWC. 498--514.
[44]
M. Schlichtkrull, T. N. Kipf, P. Bloem, R. V. D. Berg, I. Titov, and M. Welling. 2018. Modeling relational data with graph convolutional networks. In ESWC. 593--607.
[45]
B. Shi and T. Weninger. 2017. ProjE: Embedding Projection for Knowledge Graph Completion. In AAAI . 1236--1242.
[46]
R. Socher, D. Chen, C. D. Manning, and A. Ng. 2013. Reasoning with neural tensor networks for knowledge base completion. In NIPS . 926--934.
[47]
Y. Sun, L. Lin, D. Tang, N. Yang, Z. Ji, and X. Wang. 2015. Modeling Mention, Context and Entity with Neural Networks for Entity Disambiguation. In IJCAI . 1333--1339.
[48]
J. Tang, M. Qu, M. Wang, M. Zhang, J. Yan, and Q. Mei. 2015. LINE: Large-scale information network embedding. In WWW. 1067--1077.
[49]
Y. Tay, A. T. Luu, and S. C. Hui. 2017a. Non-Parametric Estimation of Multiple Embeddings for Link Prediction on Dynamic Knowledge Graphs. In AAAI. 1243--1249.
[50]
Y. Tay, L. A. Tuan, M. C. Phan, and S. C. Hui. 2017b. Multi-Task Neural Network for Non-discrete Attribute Prediction in Knowledge Graphs. In CIKM. 1029--1038.
[51]
K. Toutanova and D. Chen. 2015. Observed versus latent features for knowledge base and text inference. In Proceedings of the 3rd Workshop on Continuous Vector Space Models and their Compositionality . 57--66.
[52]
K. Toutanova, D. Chen, P. Pantel, H. Poon, P. Choudhury, and M. Gamon. 2015. Representing Text for Joint Embedding of Text and Knowledge Bases. In EMNLP, Vol. 15. 1499--1509.
[53]
K. Toutanova, V. Lin, W. T. Yih, H. Poon, and C. Quirk. 2016. Compositional Learning of Embeddings for Relation Paths in Knowledge Base and Text. In ACL .
[54]
T. Trouillon, J. Welbl, S. Riedel, É. Gaussier, and G. Bouchard. 2016. Complex embeddings for simple link prediction. In ICML. 2071--2080.
[55]
D. Vrandevc ić and M. Krötzsch. 2014. Wikidata: a free collaborative knowledgebase. Commun. ACM, Vol. 57, 10 (2014), 78--85.
[56]
Z. Wang, J. Zhang, J. Feng, and Z. Chen. 2014a. Knowledge Graph and Text Jointly Embedding. In EMNLP. 1591--1601.
[57]
Z. Wang, J. Zhang, J. Feng, and Z. Chen. 2014b. Knowledge Graph Embedding by Translating on Hyperplanes. In AAAI . 1112--1119.
[58]
H. Xiao, M. Huang, L. Meng, and X. Zhu. 2017. SSP: Semantic Space Projection for Knowledge Graph Embedding with Text Descriptions. In AAAI . 3104--3110.
[59]
R. Xie, Z. Liu, J. Jia, H. Luan, and M. Sun. 2016b. Representation Learning of Knowledge Graphs with Entity Descriptions. In AAAI . 2659--2665.
[60]
R. Xie, Z. Liu, and M. Sun. 2016a. Representation Learning of Knowledge Graphs with Hierarchical Types. In IJCAI . 2965--2971.
[61]
C. Xiong and J. Callan. 2015. Esdrank: Connecting query and documents through external semi-structured data. In CIKM . 951--960.
[62]
B. Yang, W. T. Yih, X. He, J. Gao, and L. Deng. 2015. Embedding entities and relations for learning and inference in knowledge bases. In ICLR .
[63]
B. Zhang and M. Al Hasan. 2017. Name disambiguation in anonymized graphs using network embedding. In CIKM . 1239--1248.
[64]
C. Zhang, W. Fan, N. Du, and P. S. Yu. 2016. Mining User Intentions from Medical Queries: A Neural Network Based Heterogeneous Jointly Modeling Approach. In WWW. 1373--1384.
[65]
W. Zheng, H. Cheng, L. Zou, J. X. Yu, and K. Zhao. 2017. Natural Language Question/Answering: Let Users Talk With The Knowledge Graph. In CIKM . 217--226.
[66]
H. Zhong, J. Zhang, Z. Wang, H. Wan, and Z. Chen. 2015. Aligning Knowledge and Text Embeddings by Entity Descriptions. In EMNLP . 267--272.

Cited By

View all
  • (2025)Accelerating Mini-batch HGNN Training by Reducing CUDA KernelsAlgorithms and Architectures for Parallel Processing10.1007/978-981-96-1542-1_17(283-303)Online publication date: 15-Feb-2025
  • (2024)HiHGNN: Accelerating HGNNs Through Parallelism and Data Reusability ExploitationIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2024.339484135:7(1122-1138)Online publication date: Jul-2024
  • (2024)Learning Dynamic and Static Representations for Extrapolation-Based Temporal Knowledge Graph ReasoningIEEE/ACM Transactions on Audio, Speech, and Language Processing10.1109/TASLP.2024.348550032(4741-4754)Online publication date: 2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CIKM '18: Proceedings of the 27th ACM International Conference on Information and Knowledge Management
October 2018
2362 pages
ISBN:9781450360142
DOI:10.1145/3269206
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2018

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. deep learning
  2. graph embeddings
  3. joint modeling
  4. knowledge graphs
  5. link prediction

Qualifiers

  • Research-article

Funding Sources

Conference

CIKM '18
Sponsor:

Acceptance Rates

CIKM '18 Paper Acceptance Rate 147 of 826 submissions, 18%;
Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)41
  • Downloads (Last 6 weeks)10
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Accelerating Mini-batch HGNN Training by Reducing CUDA KernelsAlgorithms and Architectures for Parallel Processing10.1007/978-981-96-1542-1_17(283-303)Online publication date: 15-Feb-2025
  • (2024)HiHGNN: Accelerating HGNNs Through Parallelism and Data Reusability ExploitationIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2024.339484135:7(1122-1138)Online publication date: Jul-2024
  • (2024)Learning Dynamic and Static Representations for Extrapolation-Based Temporal Knowledge Graph ReasoningIEEE/ACM Transactions on Audio, Speech, and Language Processing10.1109/TASLP.2024.348550032(4741-4754)Online publication date: 2024
  • (2024)A Knowledge Graph Completion Model with Path-Specific Learning2024 IEEE International Conference on Medical Artificial Intelligence (MedAI)10.1109/MedAI62885.2024.00071(491-500)Online publication date: 15-Nov-2024
  • (2024)MRH-GCN: An Efficient GCN Accelerator for Multi-Relation Heterogeneous Graph2024 IEEE 32nd Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM)10.1109/FCCM60383.2024.00030(197-203)Online publication date: 5-May-2024
  • (2023)Interpretable patent recommendation with knowledge graph and deep learningScientific Reports10.1038/s41598-023-28766-y13:1Online publication date: 14-Feb-2023
  • (2022)A Dynamic Convolutional Network-Based Model for Knowledge Graph CompletionInformation10.3390/info1303013313:3(133)Online publication date: 4-Mar-2022
  • (2022)Dual Gated Graph Attention Networks with Dynamic Iterative Training for Cross-Lingual Entity AlignmentACM Transactions on Information Systems10.1145/347116540:3(1-30)Online publication date: 31-Jul-2022
  • (2022)Heterogeneous Network Representation Learning: A Unified Framework With Survey and BenchmarkIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2020.304592434:10(4854-4873)Online publication date: 1-Oct-2022
  • (2022)A Neighborhood Re-Ranking Model With Relation Constraint for Knowledge Graph CompletionIEEE/ACM Transactions on Audio, Speech and Language Processing10.1109/TASLP.2022.322553731(411-425)Online publication date: 30-Nov-2022
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media