Skip to main content
SpringerLink
Log in

Improving temporal knowledge graph embedding using tensor factorization

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The approach of knowledge graph embedding (KGE) enables it possible to represent facts of a knowledge graph (KG) in low-dimensional continuous vector spaces. Consequently, it can significantly reduce the complexity of those operations performed on the underlying KG, and has attracted a lot of attention in recent years. However, most of KGE approaches have only been developed over static facts and ignore the time attribute. As a matter of effect, in some real-world KGs, a fact might only be valid for a specific time interval or point in time. For instance, the fact (Barack Obama, is president of, US, [2009-2017]) is only valid between 2009 and 2017. To conquer this issue, based on a famous tensor factorization approach, canonical polyadic decomposition, we propose two new temporal KGE models called TSimplE and TNTSimplE that integrates time information besides static facts. A non-temporal component is also added to deal with heterogeneous temporal KGs that include both temporal and non-temporal relations. We prove that the proposed models are fully expressive which has a bound on the dimensionality of their embeddings, and can incorporate several important types of background knowledge including symmetry, antisymmetry and inversion. In addition, our models are capable of dealing with two common challenges in real-world temporal KGs, i.e., modeling time intervals and predicting time for facts with missing time information. We conduct extensive experiments on three real-world temporal KGs: ICEWS, YAGO3 and Wikidata. The results indicate that our models achieve start-of-the-art performance with lower time or space complexity.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Code Availability

Some or all data, models, or code generated or used during the study are available from the corresponding author by request. (List items).

References

  1. Adam P, Sam G, Soumith C, Gregory C, Edward Y, Zachary D, Zeming L, Alban D, Luca A, Adam L (2017) Automatic differentiation in pytorch. In: Proceedings of neural information processing systems

  2. Balazevic I, Allen C, Hospedales T (2019) Tucker: Tensor factorization for knowledge graph completion. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pp 5188–5197

  3. Bollacker K, Evans C, Paritosh P, Sturge T, Taylor J (2008) Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of the 2008 ACM SIGMOD international conference on Management of data, pp 1247–1250

  4. Bordes A, Usunier N, Garcia-Duran A, Weston J, Yakhnenko O (2013) Irreflexive and hierarchical relations as translations. arXiv:1304.7158

  5. Bordes A, Usunier N, Garcia-Durán A, Weston J, Yakhnenko O (2013) Translating embeddings for modeling multi-relational data. In: Proceedings of the 26th international conference on neural information processing systems-volume 2, pp 2787–2795

  6. Dasgupta SS, Ray SN, Talukdar P (2018) Hyte: Hyperplane-based temporally aware knowledge graph embedding. In: Proceedings of the 2018 conference on empirical methods in natural language processing, pp 2001–2011

  7. Dong X, Gabrilovich E, Heitz G, Horn W, Lao N, Murphy K, Strohmann T, Sun S, Zhang W (2014) Knowledge vault: a web-scale approach to probabilistic knowledge fusion. In: Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pp 601–610

  8. Erxleben F, Günther M, Krötzsch M, Mendez J, Vrandea̧ić D (2014) Introducing wikidata to the linked data web. In: Proceedings of the 13th international semantic web conference-part I, pp 50–65

  9. Fabian M, Gjergji K, Gerhard W, et al. (2007) Yago: a core of semantic knowledge unifying wordnet and wikipedia. In: 16Th International world wide web conference, WWW, pp 697–706

  10. Garcia-Duran A, Dumančić S, Niepert M (2018) Learning sequence encoders for temporal knowledge graph completion. In: Proceedings of the 2018 conference on empirical methods in natural language processing, pp 4816–4821

  11. Goel R, Kazemi SM, Brubaker M, Poupart P (2020) Diachronic embedding for temporal knowledge graph completion. In: Proceedings of the AAAI conference on artificial intelligence, vol 34, pp 3988–3995

  12. Håstad J (1989) Tensor rank is np-complete. In: International colloquium on automata, languages, and programming, pp 451–460. Springer

  13. Hitchcock FL (1927) The expression of a tensor or a polyadic as a sum of products. J Math Phys 6(1-4):164–189

    Article  MATH  Google Scholar 

  14. Huang X, Zhang J, Li D, Li P (2019) Knowledge graph embedding based question answering. In: Proceedings of the twelfth ACM international conference on web search and data mining, pp 105–113

  15. Ji G, He S, Xu L, Liu K, Zhao J (2015) Knowledge graph embedding via dynamic mapping matrix. In: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing (volume 1: Long papers), pp 687–696

  16. Kazemi SM, Poole D (2018) Simple embedding for link prediction in knowledge graphs. In: Advances in neural information processing systems, pp 4284–4295

  17. Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv:1412.6980

  18. Kruskal J, Rank D (1989) Uniqueness for 3-way and n-way arrays. Multiway Data Analysis. Elsevier, Amsterdam, pp 7–18

    Google Scholar 

  19. Lacroix T, Obozinski G, Usunier N (2020) Tensor decompositions for temporal knowledge base completion. arXiv:2004.04926

  20. Lacroix T, Usunier N, Obozinski G (2018) Canonical tensor decomposition for knowledge base completion. In: Proceedings of the 35th international conference on machine learning, pp 2863–2872. PMLR

  21. Lautenschlager J, Shellman S, Ward M (2015) Icews events and aggregations. Harvard Dataverse 3

  22. Leblay J, Chekol MW (2018) Deriving validity time in knowledge graph. In: Companion proceedings of the the web conference 2018, pp 1771–1776

  23. Leetaru K, Schrodt P A (2013) Gdelt: Global data on events, location, and tone, 1979–2012. In: ISA Annual convention, vol 2, pp 1–49. Citeseer

  24. Lin Y, Liu Z, Luan H, Sun M, Rao S, Liu S (2015) Modeling relation paths for representation learning of knowledge bases. In: Proceedings of the 2015 conference on empirical methods in natural language processing, pp 705–714

  25. Ma Y, Tresp V, Daxberger EA (2019) Embedding models for episodic knowledge graphs. J Web Semantics 59(100):490

    Google Scholar 

  26. Mahdisoltani F, Biega J, Suchanek F (2014) Yago3: a knowledge base from multilingual wikipedias. In: 7Th biennial conference on innovative data systems research. CIDR conference

  27. Nickel M, Murphy K, Tresp V, Gabrilovich E (2015) A review of relational machine learning for knowledge graphs. Proc IEEE 104(1):11–33

    Article  Google Scholar 

  28. Shi B, Weninger T (2018) Open-world knowledge graph completion. In: Proceedings of the AAAI conference on artificial intelligence, vol 32

  29. Sun Z, Deng ZH, Nie JY, Tang J (2019) Rotate: Knowledge graph embedding by relational rotation in complex space. In: International conference on learning representations

  30. Trouillon T, Welbl J, Riedel S, Gaussier É, Bouchard G (2016) Complex embeddings for simple link prediction. In: Proceedings of the 33rd international conference on international conference on machine learning-volume 48, pp 2071–2080

  31. Tucker LR (1966) Some mathematical notes on three-mode factor analysis. Psychometrika 31 (3):279–311

    Article  MathSciNet  Google Scholar 

  32. Wang X, He X, Cao Y, Liu M, Chua TS (2019) Kgat: Knowledge graph attention network for recommendation. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp 950–958

  33. Wang Z, Zhang J, Feng J, Chen Z (2014) Knowledge graph embedding by translating on hyperplanes. In: Proceedings of the twenty-eighth AAAI conference on artificial intelligence, pp 1112–1119

  34. Xiong C, Power R, Callan J (2017) Explicit semantic ranking for academic search via knowledge graph embedding. In: Proceedings of the 26th international conference on world wide web, pp 1271–1279

  35. Yang B, Yih WT, He X, Gao J, Deng L (2014) Embedding entities and relations for learning and inference in knowledge bases. arXiv:1412.6575

  36. Zhang Z, Cai J, Zhang Y, Wang J (2020) Learning hierarchy-aware knowledge graph embeddings for link prediction. In: Proceedings of the AAAI conference on artificial intelligence, vol 34, pp 3065–3072

  37. Zhang Z, Han X, Liu Z, Jiang X, Sun M, Liu Q (2019) Ernie: Enhanced language representation with informative entities. In: Proceedings of the 57th annual meeting of the association for computational linguistics, pp 1441–1451

Download references

Acknowledgements

This is a part research accomplishment of the project “the Science and Technology Research Project of Henan Province (No. 192102210129)”, which is supported by Henan Provincial Department of Science and Technology.

Funding

This study is funded by the project “the Science and Technology Research Project of Henan Province (No. 192102210129)”, which is supported by Henan Provincial Department of Science and Technology.

Author information

Authors and Affiliations

  1. Information Engineering University, Zhengzhou, China

    Peng He, Gang Zhou, Mengli Zhang, Jianghong Wei & Jing Chen

  2. Zhengzhou University of Technology, Zhengzhou, China

    Peng He

Authors
  1. Peng He
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gang Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mengli Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jianghong Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jing Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gang Zhou.

Ethics declarations

Conflict of Interests

All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript.

Additional information

Availability of data and material

Some or all data, models, or code generated or used during the study are available from the corresponding author by request. (List items).

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

He, P., Zhou, G., Zhang, M. et al. Improving temporal knowledge graph embedding using tensor factorization. Appl Intell 53, 8746–8760 (2023). https://doi.org/10.1007/s10489-021-03149-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-03149-w

Keywords

Search

Navigation