Skip to main content
SpringerLink
Log in

MPNet: temporal knowledge graph completion based on a multi-policy network

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Temporal knowledge graphs completion (TKGC) is a critical task that aims to forecast facts that will occur in future timestamps. It has attracted increasing research interest in recent years. Among the many approaches, reinforcement learning-based methods have gained attention due to their efficient performance and interpretability. However, these methods still face two challenges in the prediction task. First, a single policy network lacks the capability to capture the dynamic and static features of entities and relationships separately. Consequently, it fails to evaluate candidate actions comprehensively from multiple perspectives. Secondly, the composition of the action space is incomplete, often guiding the agent towards distant historical events and missing the answers in recent history. To address these challenges, this paper proposes a Temporal Knowledge Graph Completion Based on a Multi-Policy Network(MPNet). It constructs three policies from the aspects of static entity-relation, dynamic relationships, and dynamic entities, respectively, to evaluate candidate actions comprehensively. In addition, this paper creates a more diverse action space that guides the agent in investigating answers within historical subgraphs more effectively. The effectiveness of MPNet is validated through an extrapolation setting, and extensive experiments conducted on three benchmark datasets demonstrate the superior performance of MPNet compared to existing state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data Availability

This model’s codes and data are available at https://github.com/Mike-RF/MPNet.

Notes

  1. https://github.com/liu-yushan/TLogic

References

  1. Guo Q, Zhuang F, Qin C, Zhu H, Xie X, Xiong H, He Q (2020) A survey on knowledge graph-based recommender systems. IEEE Trans Knowl Data Eng 34(8):3549–3568

    Article  Google Scholar 

  2. Moon S, Shah P, Kumar A, Subba R (2019) Opendialkg: Explainable conversational reasoning with attention-based walks over knowledge graphs. In: Proceedings of the 57th annual meeting of the association for computational linguistics, pp 845–854

  3. Zhang Y, Dai H, Kozareva Z, Smola A, Song L (2018) Variational reasoning for question answering with knowledge graph. In: Proceedings of the AAAI conference on artificial intelligence, vol 32,

  4. Zhu C, Chen M, Fan C, Cheng G, Zhang Y (2021) Learning from history: Modeling temporal knowledge graphs with sequential copy-generation networks. In: Proceedings of the AAAI conference on artificial intelligence, vol 35, pp 4732–4740

  5. Jin W, Qu M, Jin X, Ren X (2020) Recurrent event network: autoregressive structure inferenceover temporal knowledge graphs. In: Empirical methods in natural language processing

  6. Wang J, Lin X, Huang H, Ke X, Wu R, You C, Guo K (2023) Glanet: temporal knowledge graph completion based on global and local information-aware network. Appl Intell 1–17

  7. Zhang J, Liang S, Sheng Y, Shao J (2022) Temporal knowledge graph representation learning with local and global evolutions. Knowl Based Sy 251:109234

    Article  Google Scholar 

  8. Han Z, Chen P, Ma Y, Tresp V (2021) Explainable subgraph reasoning for forecasting on temporal knowledge graphs. In: International conference on learning representations

  9. Liu Y, Ma Y, Hildebrandt M, Joblin M, Tresp V (2022) Tlogic: Temporal logical rules for explainable link forecasting on temporal knowledge graphs. In: Proceedings of the AAAI conference on artificial intelligence, vol 36, pp 4120–4127

  10. Xiong W, Hoang T, Wang WY (2017) Deeppath: A reinforcement learning method for knowledge graph reasoning. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp 564–573

  11. Das R, Dhuliawala S, Zaheer M, Vilnis L, Durugkar I, Krishnamurthy A, Smola A, McCallum A (2017) Go for a walk and arrive at the answer: reasoning over paths in knowledge bases with reinforcement learning. arXiv preprint arXiv:1711.05851

  12. Liao J, Zhao X, Tang J, Zeng W, Tan Z (2021) To hop or not, that is the question: towards effective multi-hop reasoning over knowledge graphs. World Wide Web 24:1837–1856

    Article  Google Scholar 

  13. Lin XV, Socher R, Xiong C (2018) Multi-hop knowledge graphreasoning with reward shaping. In: Riloff E, Chiang D, Hockenmaier J, Tsujii J (eds) Proceedings of the 2018 conference on empirical methods in natural language processing, association for computational linguistics, Brussels, Belgium, pp 3243–3253. https://doi.org/10.18653/v1/D18-1362. https://aclanthology.org/D18-1362

  14. Zhu A, Ouyang D, Liang S, Shao J (2022) Step by step: A hierarchical framework for multi-hop knowledge graph reasoning with reinforcement learning. Knowl Based Syst 248:108843

    Article  Google Scholar 

  15. Sun H, Zhong J, Ma Y, Han Z, He K (2021) TimeTraveler:Reinforcement learning for temporal knowledge graph forecasting. In: Moens M-F, Huang X, Specia L, Yih SW-t (eds) Proceedings of the 2021 conference on empirical methods in natural language processing, Association for Computational Linguistics, pp 8306–8319. https://doi.org/10.18653/v1/2021.emnlp-main.655. https://aclanthology.org/2021.emnlp-main.655

  16. Jain P, Rathi S, Chakrabarti S et al (2020) Temporal knowledge base completion: new algorithms and evaluation protocols. In: Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), pp 3733–3747

  17. Bordes A, Usunier N, Garcia-Duran A, Weston J, Yakhnenko O (2013) Translating embeddings for modeling multi-relational data. Adv Neural Inf Process Syst 26

  18. Lin Y, Liu Z, Sun M, Liu Y, Zhu X (2015) Learning entity and relation embeddings for knowledge graph completion. In: Proceedings of the AAAI conference on artificial intelligence, vol 29

  19. Schlichtkrull M, Kipf TN, Bloem P, Van Den Berg R, Titov I, Welling M (2018) Modeling relational data with graph convolutional networks. In: The Semantic Web: 15th International Conference, ESWC 2018, Heraklion, Crete, Greece, June 3–7, 2018, Proceedings 15, Springer, pp 593–607

  20. Vashishth S, Sanyal S, Nitin V, Talukdar P (2020) Composition-based multi-relational graph convolutional networks

  21. Leblay J, Chekol MW (2018) Deriving validity time in knowledgegraph. In: Companion proceedings of the the web conference 2018, WWW ’18, International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, pp 1771-1776. https://doi.org/10.1145/3184558.3191639

  22. García-Durán A, Dumančić S, Niepert M (2018) Learning sequence encoders for temporal knowledge graph completion. In: Conference on empirical methods in natural language processing, vol abs/1809.03202, pp 4816–4821

  23. Goel R, Kazemi SM, Brubaker M, Poupart P (2020) Diachronic embedding for temporal knowledge graph completion 34:3988–3995

    Google Scholar 

  24. Lacroix T, Obozinski G, Usunier N (2020) Tensor decompositions for temporal knowledge base completion, vol abs/2004.04926

  25. Trivedi R, Dai H, Wang Y, Song L (2017) Know-evolve: deep temporal reasoning for dynamic knowledge graphs. In: International conference on machine learning, PMLR, pp 3462–3471

  26. Han Z, Ma Y, Wang Y, Günnemann S, Tresp V (2020) Graph hawkes neural network for forecasting on temporal knowledge graphs. Preprint arXiv:2003.13432

  27. Han Z, Ding Z, Ma Y, Gu Y, Tresp V (2021) Temporal knowledge graph forecasting with neural ode. Preprint arXiv:2101.05151

  28. Li Z, Guan S, Jin X, Peng W, Lyu Y, Zhu Y, Bai L, Li W, Guo J, Cheng X (2022) Complex evolutional pattern learning fortemporal knowledge graph reasoning. In: Proceedings of the 60th annual meeting of the association for computational linguistics (Volume 2: Short Papers), Association for Computational Linguistics, Dublin, Ireland, pp 290–296. https://doi.org/10.18653/v1/2022.acl-short.32. https://aclanthology.org/2022.acl-short.32

  29. Lee D-H, Ahrabian K, Jin W, Morstatter F, Pujara J (2023) Temporal knowledge graph forecasting without knowledge usingin-context learning. In: The 2023 conference on empirical methods in natural language processing (EMNLP). https://openreview.net/forum?id=wpjRa3d9OJ

  30. Lin Q, Liu J, Mao R, Xu F, Cambria E (2023) Techs: Temporal logical graph networks for explainable extrapolation reasoning. In: Proceedings of the 61st annual meeting of the association for computational linguistics (vol 1: Long Papers), pp 1281–1293

  31. Williams RJ (1992) Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach Learn 8(3–4):229–256

    Article  Google Scholar 

  32. Boschee E, Lautenschlager J, O’Brien S, Shellman S, Starz J, Ward M (2015). ICEWS Coded Event Data. https://doi.org/10.7910/DVN/28075

  33. Leblay J, Chekol MW (2018) Deriving validity time in knowledge graph. In: Companion of the the web conference, pp 1771–1776

  34. Mahdisoltani F, Biega J, Suchanek F (2013) Yago3: A knowledge base from multilingual wikipedias

  35. Han Z, Wang Y, Ma Y, Guünnemann, Tresp V (2020) Graph hawkes network for reasoning on temporal knowledge graphs

Download references

Funding

This work was supported by the Natural Science Foundation of Fujian, China(No. 2021J01619), and the National Natural Science Foundation of China(No.61672159).

Author information

Authors and Affiliations

  1. College of Computer and Data Science, Fuzhou University, 350108, Fujian, China

    Jingbin Wang, RenFei Wu, YuWei Wu, FuYuan Zhang, SiRui Zhang & Kun Guo

Authors
  1. Jingbin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. RenFei Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. YuWei Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. FuYuan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. SiRui Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kun Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to this paper. The design and implementation of the model algorithm was mainly completed by JingBin Wang. The first draft of the manuscript and part of the algorithm design were completed by RenFei Wu. Kun Guo participated in the discussion, proposed the model and completed the review and editing of the manuscript. Material preparation, data collection and analysis were performed by YuWei Wu, FuYuan Zhang, SiRui Zhang. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Kun Guo.

Ethics declarations

Competing Interests

The authors declare that there is no conflict of interest regarding the publication of this paper.

Ethic and informed consent for data used

− Our paper has not been submitted or published elsewhere. − The paper won’t be submitted anywhere else till the program with the journal editing department is finished. − Our research will not be divided into several parts to increase submission volume and submitted to different journals or a single journal over time. − The relevant dataset utilized in the experiment as well as our model code have been made available online for public access.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, J., Wu, R., Wu, Y. et al. MPNet: temporal knowledge graph completion based on a multi-policy network. Appl Intell 54, 2491–2507 (2024). https://doi.org/10.1007/s10489-024-05320-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-024-05320-5

Keywords

Search

Navigation