Skip to main content
SpringerLink
Log in

DAuCNet: deep autoregressive framework for temporal link prediction combining copy mechanism network

  • Regular Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

The research on temporal knowledge graphs (TKGs) has received increasing attention. Since knowledge graphs are always incomplete, knowledge reasoning problems are crucial. However, knowledge reasoning is challenging due to the temporal evolution of TKGs. Yet, most existing approaches focus on knowledge graph inference within past timestamps and cannot predict upcoming facts. There is evidence that when temporal facts in TKGs are evolving and interacting, most of them exhibit repeating patterns along the historical timeline. This observation indicates that forecasting models may predict upcoming facts based on history. To this end, this paper proposes a novel temporal representation learning model for predicting future facts named DAuCNet, which applies a Deep Autoregressive structure as the main framework and combines it with a time-aware Copy-based mechanism Network. Specifically, our model proposes a Temporal Fact Encoder to encode historical facts and a Duplicate Fact Collector to collect historically relevant events and identify repetitive events. It employs a multi-relation Neighborhood Aggregator based on graph-attention networks to model the connection of facts at the concurrent window. Finally, DAuCNet integrates these three modules to forecast future facts. Experimental results show that DAuCNet performs significantly better at temporal link prediction and inference for future timestamps than other baselines using five public knowledge graph datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. https://github.com/aning-mark/temporal_link_prediction_daucnet_master

  2. https://github.com/ttrouill/complex

  3. https://github.com/MichSchli/RelationPrediction

  4. https://github.com/TimDettmers/ConvE

  5. https://github.com/DeepGraphLearning/KnowledgeGraphEmbedding

  6. https://github.com/malllabiisc/HyTE

  7. https://github.com/mniepert/mmkb

  8. https://github.com/rstriv/Know-Evolve

  9. https://github.com/mdeff/paper-cnn-graph-recurrent-iclr2017

References

  1. Berahmand K, Nasiri E, Rostami M et al (2021) A modified DeepWalk method for link prediction in attributed social network. Computing 103(10):2227–2249

    Article  MathSciNet  Google Scholar 

  2. Xiong C, Merity S, Socher R (2016) Dynamic memory networks for visual and textual question answering, Proceedings of the 2016 International Conference on Machine Learning, pp 2397–2406

  3. Chen J, Chen J, Yu Z (2019) Incorporating structured commonsense knowledge in story completion, Proceedings of the 2019 AAAI Conference on Artificial Intelligence. pp 6244–6251

  4. Lü L, Medo M, Yeung CH et al (2012) Recommender systems. Phys Rep 519(1):1–49

    Article  Google Scholar 

  5. Liu ZH, Xiong C, Sun M, et al. (2018) Entity-Duet neural ranking: understanding the role of knowledge graph semantics in neural information retrieval. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pp 2395–2405

  6. Leetaru K, Schrodt PA (2013) Gdelt: Global data on events, location, and tone, ISA Annual Convention

  7. Ward MD, Beger A, Cutler J et al (2013) Comparing GDELT and ICEWS event data. Analysis 21(1):267–297

    Google Scholar 

  8. Jiang T, Liu T, Ge T et al. (2016) Encoding temporal information for time-aware link prediction. In: Proceedings of the 2016 conference on empirical methods in natural language processing, pp 2350–2354

  9. Dasgupta SS, Ray SN, Talukdar PH (2018) Hyperplane-based temporally aware knowledge graph embedding. In: Proceedings of the 2018 conference on empirical methods in natural language processing, pp 2001–2011

  10. Garcia-Duran A, Dumančić S, Niepert M (2018) Learning sequence encoders for temporal knowledge graph completion. In: Proceedings of the 2018 conference on empirical methods in natural language processing, pp 4816–4821

  11. Goel R, Kazemi SM, Brubaker M et al. (2020) Diachronic embedding for temporal knowledge graph completion. In: Proceedings of the 2020 AAAI conference on artificial intelligence, pp 3988–3995

  12. Cao Y, Wang X, He X, et al. (2019) Unifying knowledge graph learning and recommendation: towards a better understanding of user preferences. Proceedings of the 2019 world wide web conference, pp 151–161

  13. Zhou L, Yang Y, Ren X et al. (2018) Dynamic network embedding by modeling triadic closure process. In: Proceedings of the 2018 AAAI conference on artificial intelligence, pp 571–578

  14. Sankar A, Wu Y, Gou L et al. (2020) Dysat: Deep neural representation learning on dynamic graphs via self-attention networks. In: Proceedings of the 13th international conference on web search and data mining, pp 519–527

  15. Trivedi R, Dai H, Wang Y et al. (2017) Know-evolve: Deep temporal reasoning for dynamic knowledge graphs. In: Proceedings of the 34th international conference on machine learning, pp 3462–3471

  16. Trivedi R, Farajtabar M, Biswal P, et al. (2019) Dyrep: learning representations over dynamic graphs. In: Proceedings of the 7th international conference on learning representations

  17. Korotayev AV, Tsirel SV (2010) A spectral analysis of world GDP dynamics: Kondratieff waves, Kuznets swings, Juglar and Kitchin cycles in global economic development, and the 2008–2009 economic crisis. Struct Dyn 4 (1)

  18. Nasiri E, Berahmand K, Li Y (2021) A new link prediction in multiplex networks using topologically biased random walks. Chaos Solitons Fractals 151:111230

    Article  Google Scholar 

  19. Nasiri E, Berahmand K, Samei Z et al (2022) Impact of centrality measures on the common neighbors in link prediction for multiplex networks. Big Data 10(2):138–150

    Article  Google Scholar 

  20. Gu J, Lu Z, Li H et al. (2016) Incorporating copying mechanism in sequence-to-sequence learning. In: Proceedings of the 54th annual meeting of the association for computational linguistics

  21. Wang Q, Mao Z, Wang B et al (2017) Knowledge graph embedding: A survey of approaches and applications. IEEE Trans Knowl Data Eng 29(12):2724–2743

    Article  Google Scholar 

  22. Ji S, Pan S, Cambria E et al (2022) A survey on knowledge graphs: representation, acquisition, and applications. IEEE Trans Neural Netw Learn Syst 33(2):494–514

    Article  MathSciNet  Google Scholar 

  23. Wang M, Qiu L, Wang X (2021) A survey on knowledge graph embeddings for link prediction. Symmetry 13(3):485

    Article  Google Scholar 

  24. Bordes A, Usunier N, Garcia-Duran A, et al. (2013) Translating embeddings for modeling multi-relational data. In: Proceedings of the 27th annual conference on neural information processing systems, pp 2787–2795

  25. Wang Z, Zhang J, Feng J, et al. (2014) Knowledge graph embedding by translating on hyperplanes. In: Proceedings of the 2014 AAAI conference on artificial intelligence, pp 1112–1119

  26. Lin Y, Liu Z, Sun M, et al. (2015) Learning entity and relation embeddings for knowledge graph completion. In: Proceedings of the 29th AAAI conference on artificial intelligence, pp 2181–2187

  27. Nickel M, Tresp V, Kriegel HP (2011) A three-way model for collective learning on multi-relational data. In: Proceedings of the 28th international conference on machine learning, pp 809–816

  28. Sun Z, Deng ZH, Nie JY et al. (2019) Rotate: knowledge graph embedding by relational rotation in complex space. In: Proceedings of the 7th international conference on learning representations

  29. Nickel M, Rosasco L, Poggio T (2016) Holographic embeddings of knowledge graphs. In: Proceedings of the 2016 AAAI conference on artificial intelligence, pp 1955–1961

  30. Yang B, Yih SW, He X et al. (2015) Embedding entities and relations for learning and inference in knowledge bases. In: Proceedings of the 2015 international conference on learning representations

  31. Trouillon T, Welbl J, Riedel S et al. (2016) Complex embeddings for simple link prediction. In: Proceedings of the 2016 international conference on machine learning, pp 2071–2080

  32. Dai Quoc Nguyen TDN, Nguyen DQ, Phung D (2018) A novel embedding model for knowledge base completion based on convolutional neural network. In: Proceedings of the 2018 conference of the North American chapter of the association for computational linguistics, pp 327–333

  33. Dettmers T, Minervini P, Stenetorp P et al. (2018) Convolutional 2D knowledge graph embeddings. In: Proceedings of the 32nd AAAI conference on artificial intelligence, pp 1811–1818

  34. Balažević I, Allen C, Hospedales TM (2019) Tucker: Tensor factorization for knowledge graph completion. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing, pp 5184–5193

  35. Ma Y, Tresp V, Daxberger EA (2019) Embedding models for episodic knowledge graphs. J Web Semantics 59:100490

    Article  Google Scholar 

  36. Bai L, Yu W, Chen M et al (2021) Multi-hop reasoning over paths in temporal knowledge graphs using reinforcement learning. Appl Soft Comput 103:107144

    Article  Google Scholar 

  37. Perozzi B, Al-Rfou R, Skiena S (2014) Deepwalk: Online learning of social representations. In: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp 701–710

  38. Chen J, Zhang J, Xu X et al (2021) E-lstm-d: a deep learning framework for dynamic network link prediction. IEEE Trans Syst Man Cyber Syst 51(6):3699–3712

    Article  Google Scholar 

  39. Schlichtkrull M, Kipf TN, Bloem P et al. (2018) Modeling relational data with graph convolutional networks. In: 15th Proceedings of the 2018 European semantic web conference, pp 593–607

  40. Cai L, Yan B, Mai G et al. (2019) TransGCN: Coupling transformation assumptions with graph convolutional networks for link prediction. In: Proceedings of the 10th international conference on knowledge capture, pp 131–138

  41. Veličković P, Cucurull G, Casanova A et al. (2018) Graph attention networks. In: Proceedings of the 6th international conference on learning representations

  42. Pareja A, Domeniconi G, Chen J et al. (2020) Evolvegcn: Evolving graph convolutional networks for dynamic graphs. In: Proceedings of the 2020 AAAI conference on artificial intelligence, pp 5363–5370

  43. Singer U, Guy I, Radinsky K (2019) Node embedding over temporal graphs. In: Proceedings of the 2019 international joint conferences on artificial intelligence, pp 4605–4612

  44. Goyal P, Kamra N, He X et al. (2018) Dyngem: deep embedding method for dynamic graphs. arXiv preprint arXiv:1805.11273

  45. Cho K, van Merriënboer B, Gu̇lçehre Ç, et al. (2014) Learning phrase representations using RNN encoder–decoder for statistical machine translation. In: Proceedings of the 2014 conference on empirical methods in natural language processing, pp 1724–1734

  46. Jin W, Qu M, Jin X, et al. (2020) Recurrent event network: autoregressive structure inferenceover temporal knowledge graphs. In: Proceedings of the 2020 conference on empirical methods in natural language processing, pp 6669–6683

  47. Vaswani A, Shazeer N, Parmar N, et al. (2017) Attention is all you need. In: Proceedings of the 2017 annual conference on neural information processing systems, pp 5998–6008

  48. Nathani D, Chauhan J, Sharma C, et al. (2019) Learning attention-based embeddings for relation prediction in knowledge graphs. In:Proceedings of the 57th annual meeting of the association for computational linguistics, pp 4710–4723

  49. Vinyals O, Fortunato M, Jaitly N (2015) Pointer networks. In: Proceedings of the 2015 annual conference on neural information processing systems, pp 2692–2700

  50. Gu J, Lu Z, Li H, et al. (2016) Incorporating copying mechanism in sequence-to-sequence learning. ACL (1)

  51. Zhou Q, Yang N, Wei F, et al. (2018) Sequential copying networks. In: Proceedings of the 2018 AAAI conference on artificial intelligence, pp 4987–4995

  52. Seo Y, Defferrard M, Vandergheynst P, et al. (2018) Structured sequence modeling with graph convolutional recurrent networks. In: Proceedings of the 25th international conference on neural information processing, pp 362–373

Download references

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing, 211106, China

    Xiangning Hou, Li Yan & Zongmin Ma

  2. Department of Computer Science, University of Massachusetts Lowell, Lowell, MA, 01854, USA

    Ruizhe Ma

Authors
  1. Xiangning Hou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ruizhe Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Li Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zongmin Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

XH: Methodology, Writing—Original draft preparation. RM: Validation, Writing—Reviewing and editing. LY: Data curation, Investigation. ZM: Validation, Writing—Reviewing and editing.

Corresponding authors

Correspondence to Ruizhe Ma or Zongmin Ma.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hou, X., Ma, R., Yan, L. et al. DAuCNet: deep autoregressive framework for temporal link prediction combining copy mechanism network. Knowl Inf Syst 65, 2061–2085 (2023). https://doi.org/10.1007/s10115-022-01823-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-022-01823-0

Keywords

Search

Navigation