tutorial

Deep Knowledge Graph Representation Learning for Completion, Alignment, and Question Answering

Author:

Soumen ChakrabartiAuthors Info & Claims

SIGIR '22: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

Pages 3451 - 3454

https://doi.org/10.1145/3477495.3532679

Published: 07 July 2022 Publication History

Abstract

A knowledge graph (KG) has nodes and edges representing entities and relations. KGs are central to search and question answering (QA), yet research on deep/neural representation of KGs, as well as deep QA, have moved largely to AI, ML and NLP communities. The goal of this tutorial is to give IR researchers a thorough update on the best practices of neural KG representation and inference from AI, ML and NLP communities, and then explore how KG representation research in the IR community can be better driven by the needs of search, passage retrieval, and QA. In this tutorial, we will study the most widely-used public KGs, important properties of their relations, types and entities, best-practice deep representations of KG elements and how they support or cannot support such properties, loss formulations and learning methods for KG completion and inference, the representation of time in temporal KGs, alignment across multiple KGs, possibly in different languages, and the use and benefits of deep KG representations in QA applications.

References

[1]

R. Abboud, I. I. Ceylan, T. Lukasiewicz, and T. Salvatori. BoxE: A box embedding model for knowledge base completion. NeurIPS, abs/2007.06267, 2020. URL https://arxiv.org/pdf/2007.06267.pdf.

[2]

A. Asai, K. Hashimoto, H. Hajishirzi, R. Socher, and C. Xiong. Learning to retrieve reasoning paths over wikipedia graph for question answering. ICLR, abs/1911.10470, 2020.

[3]

K. Bollacker, C. Evans, P. Paritosh, T. Sturge, and J. Taylor. Freebase: a collaboratively created graph database for structuring human knowledge. In SIGMOD Conference, pages 1247--1250, 2008. URL http://ids.snu.ac.kr/w/images/9/98/sc17.pdf.

Digital Library

[4]

A. Bordes, N. Usunier, A. Garcia-Duran, J. Weston, and O. Yakhnenko. Translating embeddings for modeling multi-relational data. In NIPS, pages 2787--2795, 2013. URL http://papers.nips.cc/paper/5071-translating-embeddings-for-modeling-multi-relational-data.pdf.

Digital Library

[5]

M. Chen, Y. Tian, M. Yang, and C. Zaniolo. Multilingual knowledge graph embeddings for cross-lingual knowledge alignment. In IJCAI, 2017. URL https://arxiv.org/pdf/1611.03954.pdf.

Digital Library

[6]

M. Cornolti, P. Ferragina, M. Ciaramita, S. Rued, and H. Schuetze. The SMAPH system for query entity recognition and disambiguation. In ERD Challenge Workshop, 2014. URL https://research.google.com/pubs/archive/42720.pdf.

Digital Library

[7]

S. S. Dasgupta, S. N. Ray, and P. P. Talukdar. HyTE: Hyperplane-based temporally aware knowledge graph embedding. In EMNLP Conference, 2018. URL https://www.aclweb.org/anthology/D18--1225.pdf.

[8]

S. Deerwester, S. T. Dumais, T. K. Landauer, G. W. Furnas, and R. A. Harshman. Indexing by latent semantic analysis. Journal of the Society for Information Science, 41 (6): 391--407, 1990. URL https://www.cs.bham.ac.uk/ pxt/IDA/lsa_ind.pdf.

[9]

J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In NAACL Conference, 2019. URL https://www.aclweb.org/anthology/N19--1423.pdf.

[10]

gneul, and Hofmann]GaneaBH2018HyperbolicO.-E. Ganea, G. Bécigneul, and T. Hofmann. Hyperbolic entailment cones for learning hierarchical embeddings. arXiv preprint arXiv:1804.01882, 2018. URL https://arxiv.org/pdf/1804.01882.pdf.

[11]

R. Goel, S. M. Kazemi, M. A. Brubaker, and P. Poupart. Diachronic embedding for temporal knowledge graph completion. In AAAI Conference, 2020. URL https://arxiv.org/pdf/1907.03143.pdf.

[12]

and Chakrabarti]JainRMC2020TimePlexP. Jain, S. Rathi, Mausam, and S. Chakrabarti. Temporal knowledge base completion: New algorithms and evaluation protocols. EMNLP Conference, abs/2005.05035, 2020. URL https://arxiv.org/abs/2005.05035.

[13]

Z. Jia, S. Pramanik, R. S. Roy, and G. Weikum. Complex temporal question answering on knowledge graphs. CIKM, 2021. URL https://dl.acm.org/doi/pdf/10.1145/3459637.3482416.

Digital Library

[14]

M. Joshi, U. Sawant, and S. Chakrabarti. Knowledge graph and corpus driven segmentation and answer inference for telegraphic entity-seeking queries. In EMNLP Conference, pages 1104--1114, 2014. URL http://www.emnlp2014.org/papers/pdf/EMNLP2014117.pdf. Data http://bit.ly/1OCKbVW.

[15]

V. Krishnan, S. Das, and S. Chakrabarti. Enhanced answer type inference from questions using sequential models. In EMNLP/HLT, pages 315--322, 2005. URL http://www.aclweb.org/anthology/H05--1040.

Digital Library

[16]

S. Kulkarni, A. Singh, G. Ramakrishnan, and S. Chakrabarti. Collective annotation of Wikipedia entities in Web text. In SIGKDD Conference, pages 457--466, 2009. URL http://www.cse.iitb.ac.in/ soumen/doc/CSAW/.

Digital Library

[17]

T. Lacroix, G. Obozinski, and N. Usunier. Tensor decompositions for temporal knowledge base completion. In ICLR, 2020. URL https://openreview.net/forum?id=rke2P1BFwS.

[18]

X. Li, D. Roth, and K. Small. The role of semantic information in learning question classifiers. In International Joint Conference on Natural Language Processing (IJCNLP), 2004. URL http://l2r.cs.uiuc.edu/ danr/Papers/LiRoSm04.pdf.

[19]

X. L. Li, L. Vilnis, D. Zhang, M. Boratko, and A. McCallum. Smoothing the geometry of probabilistic box embeddings. In ICLR, 2019. URL https://openreview.net/forum?id=H1xSNiRcF7.

[20]

J. J. Lin, R. Nogueira, and A. Yates. Pretrained transformers for text ranking: Bert and beyond. WSDM Conference, 2021. URL https://dl.acm.org/doi/pdf/10.1145/3437963.3441667.

[21]

T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. Distributed representations of words and phrases and their compositionality. In NIPS, pages 3111--3119, 2013. URL https://goo.gl/x3DTzS.

Digital Library

[22]

A. H. Miller, A. Fisch, J. Dodge, A.-H. Karimi, A. Bordes, and J. Weston. Key-value memory networks for directly reading documents. In EMNLP Conference, 2016. URL https://www.aclweb.org/anthology/D16--1147.pdf.

[23]

G. Miller, R. Beckwith, C. FellBaum, D. Gross, K. Miller, and R. Tengi. Five papers on WordNet. Princeton University, Aug. 1993. URL https://wordnet.princeton.edu/.

[24]

D. Milne and I. H. Witten. Learning to link with Wikipedia. In CIKM, pages 509--518, 2008. URL http://www.cs.waikato.ac.nz/ dnk2/publications/CIKM08-LearningToLinkWithWikipedia.pdf.

Digital Library

[25]

B. Mitra and N. Craswell. An introduction to neural information retrieval. Foundations and Trends in Information Retrieval, 13: 1--126, 2018. URL https://www.nowpublishers.com/article/Details/INR-061.

Digital Library

[26]

M. Nickel and D. Kiela. Poincaré embeddings for learning hierarchical representations. In NIPS, pages 6341--6350, 2017. URL https://arxiv.org/pdf/1705.08039.pdf.

[27]

M. Nickel, L. Rosasco, T. A. Poggio, et al. Holographic embeddings of knowledge graphs. In AAAI Conference, pages 1955--1961, 2016. URL https://arxiv.org/abs/1510.04935.

Digital Library

[28]

J. Pennington, R. Socher, and C. D. Manning. GloVe: Global vectors for word representation. In EMNLP Conference, volume 14, pages 1532--1543, 2014. URL http://www.emnlp2014.org/papers/pdf/EMNLP2014162.pdf.

[29]

S. Pramanik, J. O. Alabi, R. S. Roy, and G. Weikum. UNIQORN: Unified question answering over rdf knowledge graphs and natural language text. ArXiv, abs/2108.08614, 2021. URL https://arxiv.org/pdf/2108.08614.pdf.

[30]

D. Savenkov and E. Agichtein. When a knowledge base is not enough: Question answering over knowledge bases with external text data. In SIGIR Conference, pages 235--244, 2016. URL https://dl.acm.org/citation.cfm?id=2911536.

Digital Library

[31]

U. Sawant and S. Chakrabarti. Learning joint query interpretation and response ranking. In WWW Conference, Brazil, 2013. URL http://arxiv.org/abs/1212.6193.

Digital Library

[32]

U. Sawant, S. Garg, S. Chakrabarti, and G. Ramakrishnan. Neural architecture for question answering using a knowledge graph and Web corpus. Information Retrieval Journal, 22: 324--349, 2018. URL http://arxiv.org/pdf/1706.00973.

Digital Library

[33]

A. Saxena, A. Tripathi, and P. P. Talukdar. Improving multi-hop question answering over knowledge graphs using knowledge base embeddings. In ACL Conference, 2020. URL https://www.aclweb.org/anthology/2020.acl-main.412.pdf.

[34]

A. Saxena, S. Chakrabarti, and P. Talukdar. Question answering over temporal knowledge graphs. ACL Conference, 2021. URL https://arxiv.org/pdf/2106.01515.

[35]

Singh2021AlignKGCH. Singh, S. Chakrabarti, P. Jain, S. R. Choudhury, and Mausam. Multilingual knowledge graph completion with joint relation and entity alignment. In AKBC Conference, volume abs/2104.08804, 2021. URL https://openreview.net/forum?id=FTzU__68rp8.

[36]

S. Subramanian and S. Chakrabarti. New embedded representations and evaluation protocols for inferring transitive relations. In SIGIR Conference, 2018. URL https://gitlab.com/soumen.chakrabarti/rectangle.

Digital Library

[37]

F. M. Suchanek, G. Kasneci, and G. Weikum. YAGO: A core of semantic knowledge unifying WordNet and Wikipedia. In WWW Conference, pages 697--706. ACM Press, 2007. URL http://www2007.org/papers/paper391.pdf.

[38]

Sun, Bedrax-Weiss, and Cohen]Sun2019PullNetH. Sun, T. Bedrax-Weiss, and W. W. Cohen. PullNet: Open domain question answering with iterative retrieval on knowledge bases and text. EMNLP Conference, abs/1904.09537, 2019 a . URL https://www.aclweb.org/anthology/D19--1242.pdf.

[39]

Sun, Deng, Nie, and Tang]Sun2019RotatEZ. Sun, Z. Deng, J.-Y. Nie, and J. Tang. RotatE: Knowledge graph embedding by relational rotation in complex space. ICLR, abs/1902.10197, 2019 b . URL https://arxiv.org/pdf/1902.10197.pdf.

[40]

X. Tang, J. Zhang, B. Chen, and C. Li. BERT-INT: A BERT-based interaction model for knowledge graph alignment. In IJCAI, 2020. URL https://www.ijcai.org/proceedings/2020/0439.pdf.

[41]

ć, Schaffert, Steiner, and Pintscher]Tanon2016FreebaseToWikidataT. P. Tanon, D. Vrandevcić, S. Schaffert, T. Steiner, and L. Pintscher. From Freebase to Wikidata: The great migration. WWW Conference, 2016. URL http://dl.acm.org/ft_gateway.cfm?id=2874809&type=pdf.

Digital Library

[42]

K. Toutanova, D. Chen, P. Pantel, H. Poon, P. Choudhury, and M. Gamon. Representing text for joint embedding of text and knowledge bases. In EMNLP Conference, pages 1499--1509, 2015. URL https://www.aclweb.org/anthology/D/D15/D15--1174.pdf.

[43]

T. Trouillon, J. Welbl, S. Riedel, É. Gaussier, and G. Bouchard. Complex embeddings for simple link prediction. In ICML, pages 2071--2080, 2016. URL http://arxiv.org/abs/1606.06357.

Digital Library

[44]

I. Vendrov, R. Kiros, S. Fidler, and R. Urtasun. Order-embeddings of images and language. arXiv preprint arXiv:1511.06361, 2015. URL https://arxiv.org/pdf/1511.06361.

[45]

F. Wang, K. Sun, M. Chen, J. Pujara, and P. Szekely. Retrieving complex tables with multi-granular graph representation learning. In SIGIR Conference, pages 1472--1482, 2021. URL https://dl.acm.org/doi/pdf/10.1145/3404835.3462909.

Digital Library

[46]

Y. Wu, X. Liu, Y. Feng, Z. Wang, R. Yan, and D. Zhao. Relation-aware entity alignment for heterogeneous knowledge graphs. IJCAI, 2019. URL https://arxiv.org/pdf/1908.08210.pdf.

[47]

V. Zayats, K. Toutanova, and M. Ostendorf. Representations for question answering from documents with tables and text. In EACL Conference, 2021. URL https://www.aclweb.org/anthology/2021.eacl-main.253.pdf.

[48]

Y. Zhu, H. Liu, Z. Wu, and Y. Du. Relation-aware neighborhood matching model for entity alignment. In AAAI Conference, 2021. URL https://arxiv.org/pdf/2012.08128.pdf.

Cited By

Tang WWang YGurrin CKongkachandra RSchoeffmann KDang-Nguyen DRossetto LSatoh SZhou L(2024)Multi-modal Entity Alignment via Position-enhanced Multi-label PropagationProceedings of the 2024 International Conference on Multimedia Retrieval10.1145/3652583.3658085(366-375)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3652583.3658085
Mu LCheng YWang XLi YZhang Y(2024)MIIGraph: Multi-granularity Information Integration Graph for Document-Level Event ExtractionWeb and Big Data10.1007/978-981-97-7244-5_6(80-94)Online publication date: 28-Aug-2024
https://doi.org/10.1007/978-981-97-7244-5_6
Lu XWang LJiang ZLiu SLin J(2023)MRE: A translational knowledge graph completion model based on multiple relation embeddingMathematical Biosciences and Engineering10.3934/mbe.202325320:3(5881-5900)Online publication date: 2023
https://doi.org/10.3934/mbe.2023253
Show More Cited By

Index Terms

Deep Knowledge Graph Representation Learning for Completion, Alignment, and Question Answering
1. Information systems
  1. Information retrieval
    1. Search engine architectures and scalability
      1. Searching with auxiliary databases

Recommendations

Knowledge Graph Embedding Based Question Answering
WSDM '19: Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining

Question answering over knowledge graph (QA-KG) aims to use facts in the knowledge graph (KG) to answer natural language questions. It helps end users more efficiently and more easily access the substantial and valuable knowledge in the KG, without ...
Knowledge graph embedding methods for entity alignment: experimental review
Abstract
In recent years, we have witnessed the proliferation of knowledge graphs (KG) in various domains, aiming to support applications like question answering, recommendations, etc. A frequent task when integrating knowledge from different KGs is to ...
Simple Question Answering over Knowledge Graph Enhanced by Question Pattern Classification
Abstract
Question answering over knowledge graph (KGQA), which automatically answers natural language questions by querying the facts in knowledge graph (KG), has drawn significant attention in recent years. In this paper, we focus on single-relation ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '22: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2022

3569 pages

ISBN:9781450387323

DOI:10.1145/3477495

General Chairs:
Enrique Amigo
UNED
,
Pablo Castells
UAM and Amazon
,
Julio Gonzalo
UNED
,
Program Chairs:
Ben Carterette
Spotify
,
J. Shane Culpepper
RMIT University
,
Gabriella Kazai
Waseda University

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 July 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Tutorial

Conference

SIGIR '22

Sponsor:

SIGIR

SIGIR '22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 11 - 15, 2022

Madrid, Spain

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
405
Total Downloads

Downloads (Last 12 months)49
Downloads (Last 6 weeks)3

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Tang WWang YGurrin CKongkachandra RSchoeffmann KDang-Nguyen DRossetto LSatoh SZhou L(2024)Multi-modal Entity Alignment via Position-enhanced Multi-label PropagationProceedings of the 2024 International Conference on Multimedia Retrieval10.1145/3652583.3658085(366-375)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3652583.3658085
Mu LCheng YWang XLi YZhang Y(2024)MIIGraph: Multi-granularity Information Integration Graph for Document-Level Event ExtractionWeb and Big Data10.1007/978-981-97-7244-5_6(80-94)Online publication date: 28-Aug-2024
https://doi.org/10.1007/978-981-97-7244-5_6
Lu XWang LJiang ZLiu SLin J(2023)MRE: A translational knowledge graph completion model based on multiple relation embeddingMathematical Biosciences and Engineering10.3934/mbe.202325320:3(5881-5900)Online publication date: 2023
https://doi.org/10.3934/mbe.2023253
Hu XChen JMeng SWen LYu PChen HDuh WHuang HKato MMothe JPoblete B(2023)SelfLRE: Self-refining Representation Learning for Low-resource Relation ExtractionProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3592058(2364-2368)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3592058
Yuan XLei QYu SXu CChen Z(2022)Fine-grained relational learning for few-shot knowledge graph completionACM SIGAPP Applied Computing Review10.1145/3570733.357073522:3(25-38)Online publication date: 3-Nov-2022
https://dl.acm.org/doi/10.1145/3570733.3570735
Do NNguyen H(2022)Knowledge-based Problem Solving and Reasoning methods2022 14th International Conference on Knowledge and Systems Engineering (KSE)10.1109/KSE56063.2022.9953617(1-7)Online publication date: 19-Oct-2022
https://doi.org/10.1109/KSE56063.2022.9953617

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten